Integrating AI with Legacy Systems: The Patterns That Actually Work

We integrate AI systems with legacy ERP platforms, accounting systems, and operations software that was not designed for API-driven automation. The read side is usually solvable. With enough patience you can extract data from almost anything. The write side is where integrations die. Posting a payment or creating a supplier record requires the system to accept structured input, validate it against business rules, and persist it correctly.

Legacy systems fail at writes for predictable reasons: brittle validation logic, undocumented field requirements, session-based authentication that times out, and no support for the idempotency that modern integration patterns depend on. Here is what we have learned to do instead of fighting these systems directly.

Never write directly to a legacy system in real-time

This is the most important rule we have. Never write directly to a legacy system in real-time from an AI pipeline. Sometimes it is technically impossible anyway. The bigger reason is that a real-time write couples your AI pipeline to the legacy system's availability, performance and error behavior.

Legacy systems go down for maintenance. They return cryptic errors at 3pm on a Friday when the network team made a firewall change. They time out under load. They fail silently on validation errors that should be exceptions. If your AI pipeline writes directly to the legacy system and the legacy system is slow, your pipeline is slow. If it returns an undocumented error, your pipeline fails.

The pattern we use instead: the AI pipeline writes to a shadow table in a database we control. A separate, durable process reads from that table and writes to the legacy system asynchronously, with retry logic and error handling that we own. The AI pipeline is decoupled from the legacy system entirely. We control the retry semantics. We control the error handling. The legacy system can be unavailable for four hours without stopping the pipeline.

Shadow tables and event sourcing at the boundary

A shadow table is a database table in your integration layer that mirrors the data model of the target system, owned by you rather than by the legacy system. When the AI pipeline produces a result, such as an approved invoice or a new supplier record, it writes to the shadow table first. The shadow table is your source of truth for what the AI decided.

From the shadow table, a separate synchronization process handles the actual write to the legacy system. It can retry on failure without re-running the AI pipeline. It can batch writes to reduce load on the legacy system. It can pause writes during a maintenance window and resume without losing data. And critically: it records the result of each write attempt, including errors, so you have a complete history of what was attempted and what succeeded.

We extend this with event sourcing at the boundary: instead of updating shadow table rows, we append events such as InvoiceApproved, SupplierCreated and POLineUpdated. Each event is immutable. The sync process processes each event once, marks it consumed, and handles the legacy write. On failure, the event stays unconsumed and retries. On success, it is marked consumed and the result is recorded alongside it. This gives you a complete, auditable history of everything the AI decided and everything that was written to the legacy system.

SOAP, SFTP, and batch imports: making each tolerable

SOAP APIs from the mid-2000s are unpleasant to work with. They use XML, WSDL schemas that are often partially wrong, authentication mechanisms that were designed before service accounts, and error responses that return HTTP 200 with a fault buried in the XML body. The shadow table pattern handles most of this. The sync layer translates your clean internal event into whatever XML the SOAP endpoint requires. We keep the SOAP-specific code in one place, isolated from the AI pipeline.

SFTP drops and batch file imports are different. Some legacy systems have no API at all. The only way to push data in is to drop a formatted file in a specific directory at a specific time and wait for a batch process to pick it up. This is still a viable integration pattern, just a slower one. The AI pipeline writes to the shadow table as normal. The sync process formats the data into the required file format (often fixed-width text, occasionally CSV with encoding requirements nobody documented) and drops it to the SFTP server.

The hard part of batch integration is confirmation. The file drops. Did the legacy system process it? Did it error? Most legacy batch systems do not return a structured confirmation. They write a log file to a different SFTP directory. Sometimes. The integration pattern here: poll the output directory for a result file after the expected processing window, parse whatever output format the legacy system generates, and map it back to the shadow table events you submitted. Then alert on anything that does not have a confirmed result within 2x the expected processing time.

Screen scraping is the last resort, for something like a 2004 procurement system with no API and no batch export, where the only write path is the browser UI. Playwright against a dedicated service account session works, with a rate limiter so you don't trip the session management. It is fragile. When the system's UI changes, the scraper breaks. The shadow table architecture earns its keep here, because the scraper is a thin adapter: if it breaks, you fix the adapter and leave the AI pipeline alone.

Polling when the legacy system supports neither webhooks nor real-time API

Modern integration assumes the source system can notify you when something changes. Legacy systems do not. You have two options: poll on a schedule, or query a change log table if the system maintains one.

Most legacy ERP systems keep some form of change tracking: a modified timestamp on records, a transaction log table, an audit history. If it exists, use it. A query for "records modified since last poll" is dramatically more efficient than a full table scan, and it captures the window of changes cleanly. Look for it before you build anything. It is common to spend weeks polling the main tables and then find a change log table that was already doing the job.

If there is no change log: poll on a schedule with a high-water mark. Record the timestamp of the last successful poll. On each run, query for records modified after that timestamp. Use the system's own timestamps, not a clock you control, because clock skew between systems causes missed records. Keep the polling interval realistic for what the legacy system can take. Poll an expensive query every 30 seconds on a system already busy with batch processing and it will start returning errors.

Treat the legacy system as a slow, unreliable external dependency, never as a database you control. Shadow tables give you something fast and reliable to build on. The legacy system is just a sync target.