AI security starts where new capabilities create unfamiliar attack surfaces

AI security starts where new capabilities create unfamiliar attack surfaces

AI does not replace familiar security risk; it links untrusted content, probabilistic behavior and privileged action. A drafting model differs from an agent that reads email, calls tools, stores memory and changes production data. Security begins by mapping capability, not by applying one control to every use.

Trace the full system: training and retrieval data, model and provider, prompts, tools, identities, outputs, logs and human decisions. Mark trust boundaries and assets . NIST�s 2025 adversarial-ML taxonomy covers evasion, poisoning, privacy and misuse; agent systems also face indirect prompt injection, where malicious instructions arrive through data the agent consumes.

Limit consequence before optimizing detection. Give agents narrow, task-specific identities; separate read from write; constrain tools and destinations; require confirmation for irreversible or high-impact actions. Treat model output as untrusted input to downstream systems. Validate parameters, enforce business rules outside the model and prevent secrets from entering contexts that providers or users should not see.

Evaluate the deployed workflow, not only the base model. Test hostile documents, compromised tools, privilege escalation, data exfiltration, memory poisoning and plausible user mistakes. Measure attack success and impact. NIST�s 2026 agent red-teaming work found security performance varies sharply across frontier models and does not uniformly track general capability.

Operate security as continuous evidence. Monitor tool calls, abnormal data access, policy overrides and model or prompt changes; preserve traceability and rehearse containment. Update evaluations as adversaries and capabilities evolve. AI security becomes credible when the maximum harm of a surprising output is bounded by architecture�and when the organization can detect, stop and learn from attempted abuse.