Start an AI security assessment by naming the decision it needs to inform. Are you deciding whether to release an assistant, expand its access, adopt a model, or change a fine-tuning process? That decision helps separate the application risks from the model questions and keeps the engagement focused on evidence the owner can use. An AI application includes more than its model. It may have users, APIs, retrieved documents, memory, business tools, approval steps, and access controls. A model assessment examines agreed security-relevant properties or behaviours of the model within the access and testing conditions available. The scopes can overlap, but the findings should identify the layer involved.
Define the application boundary
Map how a request moves through the system and where authority is checked. Identify which information a user may retrieve, which actions an agent can request, and which decisions need human approval. Include conventional application and API controls alongside AI-specific scenarios. OWASP’s 2025 LLM application risk list includes prompt injection, sensitive-information disclosure, improper output handling, excessive agency, and vector or embedding weaknesses. It provides useful prompts for deciding which scenarios belong in the assessment; it does not replace a threat model for the particular application. For a retrieval assistant, an illustrative question is whether authorised users remain confined to the information they may access when the retrieval path changes. For an agent, it may be whether a proposed action receives the required approval before execution. Agree representative test identities, safe fixtures, and clear stopping conditions with the system owner.
Define the model boundary
Record what the testing team can access: a hosted interface, evaluation outputs, configuration, fine-tuning information, datasets, weights, or an isolated model environment. Do not imply that a black-box review can inspect unavailable training data or internals. NIST’s adversarial machine-learning taxonomy distinguishes threats using factors including lifecycle stage, attacker objectives, capabilities, and knowledge. Those distinctions help teams express a testing assumption precisely and avoid treating every model as the same target. Select scenarios that matter to the intended use and are authorised for the environment. Agree which behaviours will be measured, the test budget, the treatment of sensitive data, and how results will be reproduced. Keep privacy, robustness, misuse, and broader safety or quality evaluations clearly labelled; they are not interchangeable claims.
Specify the evidence and the limits
The report should explain the tested version, conditions, observed behaviour, business relevance, and proposed response. It should also name the areas not tested. Avoid an undifferentiated “AI passed” result: readers need to know which question the evidence answered.
Plan for change and retesting
Plan how your team will address findings and when targeted retesting is useful. A new model, data source, tool permission, or workflow may change the assumptions behind the assessment. Record review triggers with the owner so the report remains connected to the system’s evolution. Secureline can help define the appropriate engagement, with application and model work scoped separately.
A starting checklist.
- State the release, access, or model-adoption decision the assessment must inform.
- Map users, retrieval sources, tools, permissions, and approval boundaries.
- Confirm authorised model access, test data, query budget, and stop conditions.
- Define reproducible evidence, version records, evaluation categories, and exclusions.
- Assign remediation owners, retest scope, and review triggers for system changes.
This guide provides general planning questions. Adapt them to your environment and agreed assessment requirements.