How to measure AI time savings: baseline, pilot and proof
Most AI business cases rest on a time-saved percentage that nobody measured. Here is a straightforward way to measure it properly — without a data science team — so your ROI is built on evidence rather than enthusiasm.
Why self-reported savings are not enough
When people are asked how much time a new tool saves them, their answers are often wrong — sometimes badly. In a 2025 randomised trial by METR, experienced software developers expected AI tools to make them 24% faster and, after the study, believed they had been about 20% faster. Measured against the clock, they were 19% slower. Perception is not measurement.
Surveys are still useful for spotting where people think AI helps. But the number that goes into a business case should come from timing real work.
Step 1: Define the unit of work
Pick something countable with a clear start and end: one invoice entered, one customer email answered, one report drafted, one meeting summarised. Avoid vague units like "admin time". If you cannot count it, you cannot measure time per unit.
Step 2: Measure the baseline
- Duration: one to two normal weeks. Avoid month-end, holidays or unusual peaks unless they are typical.
- Method: simple time logging — start and end time per unit, in a spreadsheet or time-tracking tool. Where the system already timestamps work (ticketing, CRM, accounting), use that.
- Sample: aim for at least 30–50 units per person, so a few unusual cases do not distort the average.
- Quality: record the error or rework rate as well — how many units needed correction.
Step 3: Run a fair pilot
- Same people, same work: ideally the same people do the same type of work with the tool, so differences reflect the tool, not the people.
- Allow learning time: discard the first few days, or run the pilot for long enough (two to four weeks) that the learning curve flattens.
- Include the whole task: time from receiving the work to finishing it, including prompting, waiting, reviewing and correcting the AI's output.
- Keep a comparison group if you can: a few people continuing without the tool over the same period controls for seasonal changes in workload.
Step 4: Compare speed and quality
| Measure | Baseline | Pilot | Change |
|---|---|---|---|
| Average minutes per unit | 12.0 | 8.4 | −30% |
| Units per person per day | 28 | 37 | +32% |
| Rework rate | 4% | 6% | +2 points |
| Minutes of rework per unit | 0.5 | 0.8 | +0.3 |
| Net minutes per unit | 12.5 | 9.2 | −26% |
Illustrative figures. The useful number for the business case is the net change including rework — here 26%, not the headline 30%.
Step 5: Turn the result into an ROI input
Enter the net percentage as "time saved on these tasks" in the ROI calculator, and use the share of pilot participants who actually kept using the tool as your adoption rate. If the result still clears your hurdle, scale with confidence; if not, you have saved the cost of a wider rollout.
Common measurement traps
- Measuring only easy cases. If the pilot gets the simple work and the baseline included the hard work, the saving is inflated.
- Ignoring review time. Time spent checking AI output belongs in the pilot figure.
- The novelty effect. People often work harder when they know they are being observed or are trying something new. Longer pilots reduce this.
- Counting saved time as saved money. Track what the freed time was actually used for — see why AI projects fail to deliver ROI.