AI Security for Enterprise: The Vulnerabilities Your DevSecOps Team Doesn't Know to Look For

The problem with applying traditional security frameworks to AI

The standard enterprise security frameworks, from the classic OWASP Top 10 to NIST CSF and ISO 27001, were built around a threat model where software processes inputs according to deterministic logic. The vulnerabilities they address are mostly about: unauthorized access, unvalidated input being interpreted as executable code (SQL injection, XSS), data exfiltration through unprotected channels, and authentication bypass.

AI systems break most of these assumptions. The "logic" is a statistical model whose behavior emerges from training data. The "inputs" include natural language that the model interprets rather than executes literally. The "outputs" are generated rather than retrieved. Your existing DAST scanner will not find a prompt injection vulnerability. Your WAF will not block a training data poisoning attempt. The attack surface is genuinely different, and treating it as a subset of normal application security is how you get eleven-day incidents.

Vulnerability 1: prompt injection

Prompt injection is the AI equivalent of SQL injection: an attacker embeds instructions in content that the AI processes, and those instructions override or supplement the system prompt. The attack vector is any external content the model reads: documents, emails, web pages, database records, tool outputs.

The logistics example above was an indirect prompt injection through a supplier document. Direct prompt injection, a user typing "ignore your previous instructions," is easy to detect. Indirect injection through processed content is much harder, because the model has no reliable way to distinguish between content it should process and instructions it should follow when both arrive in the same context.

It comes in several variants. One is the invoice case above. Another is a customer support agent reading email threads: a customer submits a ticket with hidden text telling the agent to mark all of that customer's later tickets as high priority. A third is a document analysis agent, where a counterparty slips a note into a contract telling the agent to flag the document as "approved" whatever the clause review finds.

Every AI system that reads external content is vulnerable to prompt injection. The mitigations are: strict output schema validation (the agent can only produce outputs in defined formats, not free-form text with embedded instructions), sandboxed tool execution (the agent cannot take privileged actions based on content alone without a human checkpoint), and content sanitization that strips or escapes instruction-like patterns before processing. None of these are complete solutions. They reduce the blast radius.

Vulnerability 2: training data poisoning

If you fine-tune models on enterprise data, or if you are building a RAG system whose retrieval corpus can be written to by untrusted parties, you have a training data poisoning risk.

The most realistic attack scenario for enterprise AI is not someone poisoning the foundation model. You are not training GPT-4. The realistic scenarios are:

• →A RAG knowledge base that any employee can write to. A disgruntled employee adds a document stating that a specific supplier has been approved for payment without PO, which becomes embedded in the retrieval index and gets cited in automated payment approvals.
• →Fine-tuning on customer interaction data that includes adversarial examples injected by customers who know the training pipeline exists.
• →A knowledge base that pulls from external sources (supplier portals, shared drives, email archives) where those sources are partially controlled by external parties.

Vulnerability 3: model inversion and membership inference

Model inversion attacks attempt to extract training data from a model by carefully querying it. Membership inference attacks determine whether a specific piece of data was in the training set. Both are primarily relevant if you fine-tune on sensitive data: employee records, customer PII, financial projections, proprietary process documentation.

A successful model inversion attack on an enterprise system is rare. Membership inference is easier to picture. A contractor who knows the training pipeline exists queries the fine-tuned model with phrases they know appeared in confidential documents, and watches how the completions change to confirm the document was in the training data.

The mitigation: do not fine-tune on data you cannot afford to have extracted. For most enterprise use cases (document processing, operations automation, customer support), fine-tuning is unnecessary and adds risk without proportional benefit. Use prompting and RAG instead of fine-tuning for workflow-specific knowledge.

Vulnerability 4: insecure tool execution

Agents with tool access (the ability to call APIs, write to databases, send emails, execute code) introduce an authorization surface that traditional IAM does not cover well. The agent authenticates with a service key that has specific permissions. The question is whether the agent's decision to use that tool should always be trusted.

The standard failure pattern: an agent has write access to a CRM because it needs to update contact records after calls. A prompt injection attack (or a jailbreak) causes the agent to bulk-delete records instead. The API key has the permission. The agent used it. Nothing in the standard security model prevented it.

We design around this with three controls:

• 1.Least-privilege tool scoping. The agent key for each tool has the minimum permissions the task requires. A summarization agent does not need write access to the CRM, only read. We audit this per agent per deployment.
• 2.Irreversibility checkpoints. Any tool call that cannot be undone (send email, delete record, post invoice, transfer funds) requires a human confirmation step before execution, regardless of agent confidence score.
• 3.Rate limits and anomaly detection on tool calls. If an agent that normally makes 5–10 database writes per session suddenly makes 200 in 90 seconds, that should fire an alert. Tool-use frameworks do not give you this by default. You have to build it.

Vulnerability 5: output leakage through AI interfaces

AI systems that generate text have a less obvious data leakage vector: they may include information from their context that a particular user should not have access to.

The concrete scenario: a customer-facing support agent has access to the full customer record, including internal notes, escalation history, and account health scores. A customer asks a question that causes the agent to surface internal information it should not share externally. The agent was not compromised. The system prompt simply never defined the boundary between internal and external knowledge.

Test for this explicitly. Before any customer-facing AI deployment, run a set of adversarial prompts designed to draw out internal information, phrased as innocent questions, requests for context, or corrections. This is the testing that catches a system handing over internal account notes in answer to a leading question.

The fix is always the same: the system prompt must explicitly enumerate what the agent is not allowed to share, not just what it is supposed to do. "You help customers with billing questions" is not enough. "You must never include internal account notes, health scores, escalation flags, or agent-facing remarks in responses to customers" is the version that works.

What an AI security audit looks like

Run five test categories on any AI system before production:

• →Prompt injection suite. ~40 test cases across direct injection, indirect injection via processed documents, and role-confusion attacks (telling the model it is in test mode, or that the previous system prompt has been replaced).
• →Information boundary testing. ~20 adversarial prompts designed to elicit information the agent should not share with the requesting user role.
• →Tool abuse simulation. Attempts to cause the agent to use tools in unintended ways: bulk operations, operations outside the stated scope, operations on entities the user should not control.
• →RAG corpus audit. If the system uses retrieval, we check what can be written to the corpus, who can write it, and whether adversarially crafted documents change agent behavior.
• →Output logging review. Every AI output should be logged with enough context to reconstruct why the model said what it said. If you cannot audit outputs post-hoc, you will not detect incidents until they become visible through other means.

None of this is in a standard penetration testing scope. You need to add it explicitly. The vendors running your annual pentest are not thinking about indirect prompt injection through supplier invoice content. You have to ask for it by name.

The one opinion I know people will push back on

Much enterprise AI security guidance says "use a guardrails layer," meaning a content filter that reviews both inputs and outputs for policy violations. These are useful. They are not sufficient and they are not a security boundary.

Guardrails catch obvious violations: explicit instructions to do harmful things, clear attempts to bypass the system. They do not catch subtle prompt injection that uses business-like language. They do not catch information leakage that sounds like a helpful response. They do not prevent tool misuse when the request pattern looks legitimate. Guardrails are a last filter, not a primary control. If your AI security strategy is "we have guardrails," the strategy is not complete.