AI Security for Enterprise: The Vulnerabilities Your DevSecOps Team Doesn't Know to Look For

The problem with applying traditional security frameworks to AI

Standard enterprise security frameworks — OWASP, NIST CSF, ISO 27001 — were built around a threat model where software processes inputs according to deterministic logic. The vulnerabilities they address are mostly about: unauthorized access, unvalidated input being interpreted as executable code (SQL injection, XSS), data exfiltration through unprotected channels, and authentication bypass.

AI systems break most of these assumptions. The "logic" is a statistical model whose behavior emerges from training data. The "inputs" include natural language that the model interprets rather than executes literally. The "outputs" are generated rather than retrieved. Your existing DAST scanner will not find a prompt injection vulnerability. Your WAF will not block a training data poisoning attempt. The attack surface is genuinely different, and treating it as a subset of normal application security is how you get eleven-day incidents.

Vulnerability 1: prompt injection

Prompt injection is the AI equivalent of SQL injection: an attacker embeds instructions in content that the AI processes, and those instructions override or supplement the system prompt. The attack vector is any external content the model reads — documents, emails, web pages, database records, tool outputs.

The logistics example above was an indirect prompt injection through a supplier document. Direct prompt injection — a user typing "ignore your previous instructions" — is easy to detect. Indirect injection through processed content is much harder, because the model has no reliable way to distinguish between content it should process and instructions it should follow when both arrive in the same context.

We have seen three variants in production. The first was the logistics invoice case above. The second was a customer support agent that was reading email threads — a customer submitted a support ticket containing hidden text that instructed the agent to mark all subsequent tickets from that customer as high priority. The third was a document analysis agent where a third party included a note in a contract that told the agent to flag the document as "approved" regardless of clause review.

Every AI system that reads external content is vulnerable to prompt injection. The mitigations are: strict output schema validation (the agent can only produce outputs in defined formats, not free-form text with embedded instructions), sandboxed tool execution (the agent cannot take privileged actions based on content alone without a human checkpoint), and content sanitization that strips or escapes instruction-like patterns before processing. None of these are complete solutions. They reduce the blast radius.

Vulnerability 2: training data poisoning

If you fine-tune models on enterprise data, or if you are building a RAG system whose retrieval corpus can be written to by untrusted parties, you have a training data poisoning risk.

The most realistic attack scenario for enterprise AI is not someone poisoning the foundation model — you are not training GPT-4. The realistic scenarios are:

• →A RAG knowledge base that any employee can write to. A disgruntled employee adds a document stating that a specific supplier has been approved for payment without PO, which becomes embedded in the retrieval index and gets cited in automated payment approvals.
• →Fine-tuning on customer interaction data that includes adversarial examples injected by customers who know the training pipeline exists.
• →A knowledge base that pulls from external sources (supplier portals, shared drives, email archives) where those sources are partially controlled by external parties.

Vulnerability 3: model inversion and membership inference

Model inversion attacks attempt to extract training data from a model by carefully querying it. Membership inference attacks determine whether a specific piece of data was in the training set. Both are primarily relevant if you fine-tune on sensitive data — employee records, customer PII, financial projections, proprietary process documentation.

We have not seen a successful model inversion attack against a client system in production. We have seen membership inference used in a different way: a contractor who knew the training pipeline existed queried the fine-tuned model with specific phrases they knew had appeared in confidential documents, observing that the model's completion behavior changed in ways that confirmed the document was in training data.

The mitigation is not to fine-tune on data you cannot afford to have extracted. For most enterprise use cases — document processing, operations automation, customer support — fine-tuning is not necessary and introduces risk without proportional benefit. Use prompting and RAG instead of fine-tuning for workflow-specific knowledge.

Vulnerability 4: insecure tool execution

Agents with tool access — the ability to call APIs, write to databases, send emails, execute code — introduce an authorization surface that traditional IAM does not cover well. The agent authenticates with a service key that has specific permissions. The question is whether the agent's decision to use that tool should always be trusted.

The standard failure pattern: an agent has write access to a CRM because it needs to update contact records after calls. A prompt injection attack (or a jailbreak) causes the agent to bulk-delete records instead. The API key has the permission. The agent used it. Nothing in the standard security model prevented it.

We design around this with three controls:

• 1.Least-privilege tool scoping. The agent key for each tool has the minimum permissions the task requires. A summarization agent does not need write access to the CRM, only read. We audit this per agent per deployment.
• 2.Irreversibility checkpoints. Any tool call that cannot be undone (send email, delete record, post invoice, transfer funds) requires a human confirmation step before execution, regardless of agent confidence score.
• 3.Rate limits and anomaly detection on tool calls. If an agent that normally makes 5–10 database writes per session suddenly makes 200 in 90 seconds, that should fire an alert. This is not a default capability in any tool-use framework we have used — you have to build it.

Vulnerability 5: output leakage through AI interfaces

AI systems that generate text have a less obvious data leakage vector: they may include information from their context that a particular user should not have access to.

The concrete scenario: a customer-facing support agent has access to the full customer record, including internal notes, escalation history, and account health scores. A customer asks a question that causes the agent to surface internal information it should not share externally — not because the agent was compromised, but because the system prompt did not clearly define the information boundary between internal and external knowledge.

We test for this explicitly. Before any customer-facing AI deployment, we run a set of adversarial prompts designed to elicit internal information — phrased as innocent questions, as requests for context, as corrections. On two of nine deployments, this testing caught the system outputting internal account notes in response to leading questions.

The fix is always the same: the system prompt must explicitly enumerate what the agent is not allowed to share, not just what it is supposed to do. "You help customers with billing questions" is not enough. "You must never include internal account notes, health scores, escalation flags, or agent-facing remarks in responses to customers" is the version that actually works.

What an AI security audit actually looks like

For any AI system we build or audit, we run five test categories before production:

• →Prompt injection suite. ~40 test cases across direct injection, indirect injection via processed documents, and role-confusion attacks (telling the model it is in test mode, or that the previous system prompt has been replaced).
• →Information boundary testing. ~20 adversarial prompts designed to elicit information the agent should not share with the requesting user role.
• →Tool abuse simulation. Attempts to cause the agent to use tools in unintended ways — bulk operations, operations outside stated scope, operations targeting entities the user should not control.
• →RAG corpus audit. If the system uses retrieval, we check what can be written to the corpus, who can write it, and whether adversarially crafted documents change agent behavior.
• →Output logging review. Every AI output should be logged with enough context to reconstruct why the model said what it said. If you cannot audit outputs post-hoc, you will not detect incidents until they become visible through other means.

None of this is in a standard penetration testing scope. You need to add it explicitly. The vendors running your annual pentest are not thinking about indirect prompt injection through supplier invoice content. You have to ask for it by name.

The one opinion I know people will push back on

Most enterprise AI security guidance says "use a guardrails layer" — a content filter that reviews both inputs and outputs for policy violations. These are useful. They are not sufficient and they are not a security boundary.

Guardrails catch obvious violations — explicit instructions to do harmful things, clear attempts to bypass the system. They do not catch subtle prompt injection that uses business-like language. They do not catch information leakage that sounds like a helpful response. They do not prevent tool misuse when the request pattern looks legitimate. Guardrails are a last filter, not a primary control. If your AI security strategy is "we have guardrails," the strategy is not complete.