When you connect two tools, an ERP and an online shop, a CMS and a website, or payroll software and a government service, the question always comes up in a meeting: real-time API vs batch, which one? Quite often someone says "real time" because it sounds modern, and the discussion ends there.
I ask the question differently, because I have seen this choice cost a lot in both directions. On this site I use both, each for a specific piece of data. Here is the grid I apply, one piece of data at a time, and what I write into the integration specification so the choice still holds when something breaks.
01What real time and batch mean in an integration
A batch feed groups data and processes it in one go, at a set time or by hand. In an integration, real time nearly always means "triggered by an event": an order is placed, a document is published, and a message goes straight to the other tool.
Andreas Kretz gives a simple rule of thumb in a video on stream and batch processing. A fixed schedule or a manual start means batch. A trigger on an event means streaming. He adds a point I find useful in meetings: having a message queue does not make processing real time. If you empty it every ten minutes, you are doing batch with one more tool.
Both forms already exist in the tools you use. Odoo offers scheduled actions that run at a fixed interval, in minutes, hours, days, weeks or months. Sanity sends webhooks when a document is created, updated or deleted. So the business need decides, and the tool follows.
02The first question: when does the data have to be right?
I do not ask "do we need real time?". I ask: at what moment does this data have to be right in the other tool, and what happens if it is not?
Stock shown on an online shop has to be right when the customer clicks "buy". If it is an hour late, you sell what you no longer have. An enriched product page can wait until the night without anyone noticing. Consolidated revenue for the leadership team only needs to be right at the time of the meeting.
Sometimes an outside rule sets the pace. Moonworkers, my company, is a payroll API for small UK businesses. On the HMRC side, the employer sends its Full Payment Submission on or before payday, and the same page warns against sending too early: if the information changes, a corrected submission is needed. It also says the submission can be split into batches. The pace is set by a deadline, with a risk in sending too late as well as too early.
03What each option costs when something breaks
On paper, both work. The difference shows on the day the target tool is down, a record arrives malformed, or the volume doubles.
Real time: fast, but it has to cope
Sanity's documentation is candid about what a webhook asks of you. Delivery is "at least once": the same message can arrive twice, and each request carries a key that lets the receiver ignore duplicates. On failure, Sanity retries twice, 30 seconds apart, with one request at a time. If your processing is slower than the pace of changes, the queue never empties.
In practice, real time forces you to plan three things: a receiver that answers quickly, a way to ignore a message already handled, and a way to catch up on what was lost during an outage longer than the retries.
Batch: simple, but a missed night shows late
A scheduled job is easier to rerun. If last night's file failed, you fix it and rerun it in the morning. Grouping also helps with cost: Google explains that each HTTP connection has a cost, and its Gmail API accepts batch requests so you open fewer of them.
The risk runs the other way: a silent failure. If nobody is told, the data stays wrong until the next run, or for days. Andreas Kretz points to another trap in his video: emptying a queue in one go every hour creates load peaks, where a steady flow leaves headroom.
04Choosing data by data, in five questions
I do not choose for the whole integration. I write one line per piece of data exchanged (stock, catalogue, order, customer, invoice, content) and answer five questions.
- When does the data have to be right in the target tool? At the customer's click, before a deadline, before a meeting?
- What does an hour's delay cost? A day's? If the answer is "nothing visible", batch is nearly always enough.
- Who is told when it fails, and through which channel? An integration without alerts is one you discover broken.
- How do you recover after an outage? You need to know where processing stopped, and be able to rerun it without creating duplicates.
- What is the reference data? When two tools disagree, which one is right, and in which unit?
The last question comes from a project lesson. On one project, a mix of centimetres and millimetres between two tools distorted an entire render. Neither real time nor batch would have changed anything. What was missing was a sentence in the specification saying which data was authoritative, and in which unit.
What I write into the specification
For each line, I add to the integration document: the direction of the flow, the trigger (event or schedule, with the time zone), the acceptable delay, the recovery rule, the person alerted and the source of truth. On an integration between an ERP and an online shop, the specification moved at almost every meeting. I kept it evolving version after version, so everyone was talking about the same document. I describe the method in the article on the ERP and online shop integration specification, and a practical case in connecting Odoo and PrestaShop.
05What I do on this site, and what I take from it
This blog runs on Next.js and Sanity, and I built it myself (more on that in the article on Sanity as a headless CMS). It uses both modes, for two different needs.
When my content machine publishes an article in Sanity, it immediately calls a revalidation route on the site, protected by a secret, then loads the page once. That is an event trigger: a published article has to be visible straight away, otherwise I end up reviewing an out-of-date version.
A monthly report that goes out on its own on a set date, on the other hand, has no reason to be real time. It is a scheduled job, of the same kind as Vercel cron jobs, which call an address on the site with an HTTP request at the planned time, always in UTC. I describe an example in the article on automated SEO reporting.
What I take from it comes down to a lesson learnt elsewhere: a site can look fine to the logged-in team and be broken for visitors, because of caching. A "real-time" data sync that writes correctly to the database but does not refresh what the visitor sees is not real time for that visitor. So I always check as a visitor, with no session.
06The limits of this grid
This grid helps you decide. It does not replace an architect when volumes are very high or when several systems write the same data. In those cases the question is no longer real time or batch, but who is allowed to write what, and I prefer to say so early.
It does not settle the choice of tool either. Between a no-code connector and custom development, the criteria are different (I cover them in the article Zapier vs Make). And I do not do ERP overhauls: when the problem lies in the ERP itself, an integration will not fix it.
Finally, mixing the two modes has a cost. Andreas Kretz shows that architectures often combine streaming and batch, with intermediate storage between the two. That is often the right answer, but it is one more component to monitor.
07Where to start on Monday
Take your integration, or the one being planned, and build the table: one line per piece of data exchanged, the five questions as columns. You will quickly see that only two or three pieces of data really need an immediate trigger, and the rest can go through a monitored batch feed.
Then write down, for each line, who is told when it fails. If that box stays empty, that is where to start, before any discussion of technology.
If you would like an outside view of that table, it is the kind of short diagnostic I do in my technical direction work, and you can see examples of projects in the portfolio. Happy to talk it through.