How to measure the return on an AI project

When AI projects disappoint, it is often not because the technology failed. It is because nobody wrote down what success would look like, so after six months there is no way to tell whether the money was well spent. Measuring return does not need a finance team. It needs a baseline, a few honest estimates and the discipline to check them.

Start with a baseline

You cannot measure improvement without knowing where you started. Before the project begins, record for the process you are changing:

• Volume: how many items per month (invoices, enquiries, quotations)
• Time per item: measured over a sample, not guessed
• Error rate: how often mistakes happen, and what they cost to fix
• Turnaround time: how long the customer or next department waits
• Who does the work and their approximate hourly cost

Two weeks of simple measurement is usually enough. A spreadsheet where staff note the time spent on 30 to 50 items gives a far better baseline than a manager's estimate.

Count all the costs

Business cases tend to include the vendor's quote and stop there. A fuller picture includes:

If government support applies, include it, but evaluate the project without it as well. A project that only pays back because of a grant is fragile.

Estimate the benefits honestly

The most common benefits, in rough order of how easy they are to measure:

1. Time saved on routine work: hours per month multiplied by the cost of those hours
2. Errors avoided: fewer re-issued invoices, wrong shipments or missed follow-ups
3. Faster turnaround: quotes sent the same day, enquiries answered after hours
4. Capacity gained: more volume handled without hiring
5. Revenue effects: better conversion of enquiries, more upsell

Be careful with time saved. If an employee saves five hours a week but there is nothing more valuable for them to do, the saving is real on paper but not in cash. The benefit becomes real when the capacity is used, for example to absorb growth without hiring or to move someone to sales.

A worked example

A trading company processes 1,200 supplier invoices a month. Baseline measurement shows each takes about four minutes to key in and check, with around 3% needing correction later.

• Current effort: 1,200 × 4 minutes = 80 hours a month
• After automation, staff review only flagged invoices and spot-check the rest: roughly 20 hours a month
• Time saved: about 60 hours a month

At a loaded staff cost of, say, S$30 an hour, that is roughly S$1,800 a month, before counting fewer corrections. If the project costs S$18,000 to build and S$400 a month to run, the simple payback is around 13 months. That is a reasonable but not spectacular project, and a useful number to have before signing.

Track results after launch

Agree on three to five measures and check them monthly for the first six months:

• The same metrics as your baseline, measured the same way
• The share of items the system handles without intervention
• Quality: errors caught in review, and complaints
• Actual running costs compared with the estimate

Expect performance to improve over the first two or three months as content is refined and staff get used to the new process. If it is not improving by month three, find out why before spending more.

When not to proceed

Sometimes the honest calculation says no. Common signs:

• The volume is too low. Automating a task done 20 times a month rarely pays back.
• The process changes frequently, so the system would need constant rework
• A simpler fix exists, such as a better form, a template or a rule in your existing software

Turning down a weak AI project is a good outcome. It frees budget for one that works.