Acceptance testing is the moment you check that software does what you expect before you open it to its real users. On paper, everyone agrees with that. In practice it is the phase that gets squeezed when the schedule slips, where everyone tests their own corner a little, and the one people regret the week after go-live.
I am writing about it because I have run acceptance testing through to go-live, including on an integration between an ERP and an e-commerce site, and because I spend a good part of my time building automatic checks for my content machines. The two are more alike than you might think. Here is how I organise acceptance testing today: a test document people actually read, business flows tested end to end, a check done as a visitor would see the site, and a checklist that says clearly when you can open the doors.
01What acceptance testing is, and what it is not
The word covers several things. The French Wikipedia page on acceptance testing separates factory testing, run by the supplier before delivery, user acceptance testing, run by the client and its business owners, and then a check of regular service in production, with real data, often on a pilot site.
That distinction clears up a common misunderstanding. The supplier's tests are not the client's acceptance testing. They show that the software works the way the supplier understood it. Business acceptance testing shows whether it works the way you need it to, which is not the same thing, especially when the specification has moved a lot. That is also why I version an integration specification: you test against a specific version, not against your memory of the last meeting.
I also keep in mind what the Scrum Guide says about the Definition of Done: a formal description of the expected state, and anything that does not meet it is not released. Acceptance testing is where that definition becomes concrete.
02A test document people actually read
A forty-page test document that nobody opens is useless. Mine fits in a spreadsheet, one row per scenario, with few columns:
- the business flow and the step being tested;
- the starting data (which customer, which product, which account);
- the action to take, in one sentence;
- the expected result, written before the test;
- the actual result, the date and the person who tested;
- the status: passed, blocking defect, minor defect.
If you only look after one column, make it the expected result. When it is empty, the tester decides on the spot whether "it looks fine", and two people will not decide the same way. Writing it beforehand forces you to reread the specification, and it often shows gaps in it, which is already a useful outcome.
At the top of the document sits a page of test plan: who tests what, in which environment, on which dates, and above all the list of cases that block the go-live. That list is written with the business, before you start.
Three levels, decided before testing
For my content machines, I have a check that rereads every article before publication and returns a three-level verdict. Red: a single point is enough to block. Amber: it may be legitimate, but the article only goes out with a written justification for each point. Green: it gets published. I describe how this works in the note on my AI content machine.
I use the same scale for the status of an acceptance test scenario. Red is a blocking defect from the list. Amber is a defect you agree to take into production, with a sentence that says why and who follows it up. Without that sentence, amber quietly becomes green, and that is exactly how a list of "minor" issues keeps growing until opening day.
03Test the whole flow, not one screen at a time
On an integration between an ERP and an e-commerce site, connected through a real-time API and batch feeds, testing each screen on its own told us almost nothing. What had to be checked was that an order placed on one side arrived correctly on the other, with the right data, at the right time, including when the overnight feed ran after a change made during the day.
The Odoo upgrade documentation says the same in its own words: test as many business flows as possible, from purchasing to invoicing, as well as integrations with external software and automated actions. It also points out that the test database is neutralised (emails, payments and scheduled actions are switched off). That is reassuring, but it means some production behaviour simply does not show up in testing.
How I write an end-to-end scenario
- Start from a real business case, not one invented for the demo.
- Follow the data from one system to the next: creation, transfer, receipt, final document.
- Check the result where it is used (the invoice, the stock, the customer's screen), not only where it was entered.
- Replay the same scenario after the batch feeds have run.
- Record the exact gap when there is one, with the starting data, so the person fixing it can reproduce it.
Test the data, not just the screens
I have learnt to test the reference data, not just the screens. A mix of units between two tools, centimetres on one side and millimetres on the other, can distort an entire render without any screen test noticing: every screen shows a plausible value. The only defence is explicit reference data, and a check that compares it from end to end.
When a calculation matters, I go further. At Moonworkers, which does payroll for small UK businesses, we wrote an "oracle" from the HMRC texts, independent of the code under test, and had it replay 896 contribution cases. It found four one-penny errors, all caused by a rounding rule. No tester would have spotted them by eye, and yet a payslip that is a penny out is still a wrong payslip.
04Check as a visitor, not as the team
A site can look normal to the logged-in team and be broken for visitors. I have seen it happen: the team, logged in to the admin, was not served the same page as visitors because of the cache. Your team sees the site logged in; your customers do not. Since then I check a site as a visitor would: private browsing, no session, on a phone, from a network that is not the office one.
It has become a fixed step in my content machines: after publishing, I check the public page, not the CMS form. A 200 response, the image and its alt text, the headings, internal links that respond. A form can keep the content you pushed into it while nothing has actually gone live.
For public pages I add one more outside view. The live test in the Search Console URL Inspection tool shows a screenshot of the page as Google renders it and the list of resources it loaded. If a stylesheet or a script does not get through, it shows up there, and the logged-in team would not have noticed.
05The go-live checklist
The day before, and on the day, I go back to the same list. I give it to you as it is, to adapt to your project. It owes a lot to the Odoo documentation, which recommends rehearsing the whole procedure the day before and warns that once the upgrade has started there is no going back, and to the GOV.UK Service Manual page on the live phase, which asks, among other things, that you can monitor the service's availability and that support staff have been trained.
- Every case on the blocking list has passed, and the decision to go live is written down, with the name of whoever takes it.
- Every defect taken into production has its written justification and an owner.
- A test that could not be run counts as a failure, not a pass.
- The procedure has been rehearsed end to end on a recent copy of production.
- You know what happens if you have to roll back, or you know you cannot, and everyone has heard it.
- Data entry in production is frozen during the switch, and users know when.
- Feeds and scheduled tasks that were switched off in testing are switched back on and checked.
- Someone watches the first orders, the first invoices and the first feed exchanges.
- The people who answer users know what is changing and who to escalate a problem to.
- A visitor check is done straight after opening.
The third line comes straight from my own tools. The trap with any check is the default green when it never ran: an environment that was down, a tester who was off, a scenario "we will do afterwards". On my strictest machine, a check that did not happen is a red. In acceptance testing, I do the same.
06What acceptance testing will not tell you
Well-run acceptance testing reduces surprises; it does not remove them, and I would rather say so before the date than after. Real volumes, users' data entry habits and partners' behaviour are never fully reproduced in testing, and the Odoo documentation says as much when it explains what is switched off in a test database.
It also costs time for business people who already have their own jobs. If nobody can be freed up, I prefer to say so early and move the date, rather than have the project team test what it built itself. And I do not test everything: I pick the flows that are expensive when they break, and I accept finding the rest in production, with someone watching.
An oracle like the one at Moonworkers has its limits too: it is only as good as its reading of the official texts. If it gets something wrong in the same way as the code, it sees nothing. That is why it has to be written from the source, by someone who did not write the code under test.
07Where to start next week
If your go-live is coming up, start with two things: write, with the business, the list of cases that block the opening, and choose three end-to-end flows to test in full, data included. The spreadsheet can wait an hour; those two decisions cannot.
This is the kind of work I do in my project and technical direction engagements. If you are a few weeks away from a go-live, we can talk about it.