Neural
Prompt injection has become the shorthand for AI security testing in much of the industry. Ask the model to ignore its instructions. Try to get it to produce prohibited content. See if you can make it reveal its system prompt. This is a useful starting point. It is also a dangerously incomplete picture of the adversarial surface of a deployed AI system.
Organizations that equate AI red-teaming with prompt injection testing have mapped a small corner of their risk exposure and left the rest unexplored. This article describes what a comprehensive AI red-team engagement looks like — the attack vectors it covers, the methodologies it employs, and the findings that typically surface when the scope expands beyond the obvious.
The Full Adversarial Surface
A production AI system is not just a model receiving user prompts. It is a system with a training pipeline, a deployment infrastructure, an access control layer, integration points with other services, and an operational context that changes over time. Adversarial testing that covers only the inference interface is testing one surface of a multi-surface system.
The full adversarial surface of a typical enterprise AI deployment includes: the training data supply chain, the model weights and their storage, the inference API and its authentication mechanisms, the system prompt and any retrieval-augmented context, the downstream services that consume model outputs, and the monitoring infrastructure itself. Each of these is a potential entry point for an attacker with different objectives and capabilities.
Model-Level Testing Beyond Prompt Injection
At the model level, a comprehensive red team runs tests across several attack categories that extend well beyond direct prompt injection.
Jailbreak evaluation. Systematic testing of the model's resistance to current jailbreak techniques — not just the naive "ignore previous instructions" but multi-step reasoning chains, role-playing scaffolds, adversarial suffixes, and token manipulation approaches. Jailbreak resistance is a measurable property that degrades with model updates; regression testing is required.
Indirect injection testing. If the model processes external content — documents, web pages, retrieved knowledge base entries — the red team tests whether adversarial content injected into those sources can override model behavior. Indirect injection is often more dangerous than direct injection because it reaches the model through channels that users and operators do not directly control.
Extraction testing. Attempts to extract training data, system prompt contents, or internal knowledge representations through carefully crafted queries. This covers both active extraction (direct queries for sensitive content) and passive extraction (statistical analysis of output distributions to infer training data characteristics).
Behavioral boundary testing. Systematic probing of the model's boundaries — what content categories it will and will not produce, what instructions it will and will not follow, what reasoning chains it can be induced to make. This produces a behavioral map that reveals inconsistencies and edge-case vulnerabilities that are difficult to predict from first principles.
System-Level Testing: Where Most Findings Live
In practice, the most consequential findings from AI red-team engagements come not from the model level but from the system level. The model itself may be well-aligned and robustly tested. The infrastructure around it often is not.
In the majority of AI red-team engagements we have conducted, the highest-severity findings were system-level vulnerabilities — authentication gaps, privilege escalation paths, and data exposure through logging infrastructure — not model-level vulnerabilities.
Authentication and authorization. API endpoints for production models frequently have weaker authentication than the sensitivity of their outputs warrants. Testing includes credential brute-forcing, token manipulation, privilege escalation through API parameter manipulation, and unauthorized cross-tenant access in multi-tenant deployments.
Logging and monitoring exposure. Inference logs often capture full request and response content. If these logs are accessible to users with API credentials — a common misconfiguration — an attacker can learn what inputs other users are submitting and reconstruct proprietary prompt templates. The monitoring infrastructure becomes an intelligence source for the attacker.
Tool and function calling abuse. Models with tool-calling capability can be induced to make unauthorized calls to integrated services. Red-team testing of function-calling interfaces includes attempts to escalate tool permissions, invoke tools outside their defined scope, and chain tool calls to produce effects that no individual call would permit.
Planning and Scoping a Red-Team Engagement
Effective AI red-teaming requires explicit scoping decisions before the engagement begins. What is the threat model — insider, external attacker, or both? What access level does the red team assume — black-box API access only, or gray-box access including system prompts and model architecture information? What are the objectives — demonstrating data exfiltration potential, identifying jailbreak vulnerabilities, or stress-testing the monitoring infrastructure?
Different threat models and access assumptions produce different findings. A black-box engagement may not surface vulnerabilities that require knowledge of the system prompt. A gray-box engagement may reveal vulnerabilities that a black-box attacker would spend weeks trying to discover. Choosing the right engagement model depends on who your actual adversaries are and what they are likely to know about your system.
From Red Team to Continuous Adversarial Testing
A point-in-time red-team engagement produces a findings report. That report has a shelf life measured in weeks for fast-moving AI systems, not years as with traditional software audits. New attack techniques are published continuously. Model updates introduce new vulnerabilities. The threat landscape evolves.
The transition from periodic red-teaming to continuous adversarial probe testing addresses this limitation. Rather than scheduling engagements quarterly or annually, continuous probing runs a defined set of adversarial test cases against production model endpoints on a daily or per-update cadence. When the probe suite is updated with new attack variants, the updated tests run automatically against existing deployments.
This does not replace periodic red-team engagements — novel attack techniques and creative exploitation often require human adversarial researchers, not automated probes. But it ensures that the window between engagements is covered by systematic testing, not by hope that attackers have not discovered what your red team has not yet tested.
