DIGNALEGI

Worth reading. A personal index selected by Ratipcan Uysal.

Digna Legi guide / AI ROI

How to Measure AI ROI Without Counting Imaginary Savings

Measure AI ROI without counting imaginary savings: identify captured value, include total system cost, test assumptions, and calculate payback.

By Ratipcan Uysal11 min read

Paper work units pass through a mechanical bottleneck toward a balance, where one yellow unit reaches the value side.

An AI proposal reaches the finance meeting with two numbers: the price of the tool and the hours it supposedly saves.

The spreadsheet multiplies those hours by salaries. A cheap subscription becomes a large return. Nobody has to show that payroll fell, hiring was avoided, valuable output increased, or loss declined. The calculation prices capacity that the organisation has not used.

Ask a harder question:

Which economic consequence changes because this time was released?

If the answer is unknown, the organisation may have evidence of a faster task. It does not yet have evidence of a return.

Released time is capacity. It becomes a return only when the organisation captures it.

Measuring AI ROI therefore requires two linked claims. The first is causal: the AI-assisted workflow changed an outcome relative to a credible alternative. The second is economic: that change produced an incremental benefit after every incremental cost required to build, operate, verify, govern, and maintain the workflow.

The companion guides cover which AI productivity metrics can support the causal claim and how to run an experiment that can reject it. This guide begins after that evidence exists. Its subject is the investment decision.

Define the return before using the formula

ROI is a comparison between options over a stated period. The relevant alternative may be the current process, a conventional software improvement, additional staff, outsourcing, or doing nothing. AI does not deserve a privileged comparison against an imaginary world in which every other option stands still.

The NIST AI Risk Management Framework explicitly asks decision makers to consider viable non-AI alternatives and the resources needed to manage risk. That requirement belongs in the economic case. A technically impressive AI workflow can still lose to a simpler queue redesign.

Four calculations answer different questions:

Net benefit
Captured economic benefit minus total incremental cost. This shows how much value remains after the investment.
ROI
Net benefit divided by total incremental cost. This expresses the return relative to what was spent.
Payback period
Initial investment divided by the rate at which net benefit arrives. This shows how long capital remains exposed, assuming the benefit arrives as forecast.
Net present value
Future incremental benefits and costs discounted to their present value. This is more informative than a single-period ROI when the workflow requires substantial upfront work or produces benefits over several years.

Each calculation needs the same foundations: a fixed time horizon, a credible alternative, incremental benefits, incremental costs, and uncertainty visible enough to change the decision. Comparing a three-year benefit with one month of cost is not optimism. It is a broken denominator.

Although written for public investment rather than corporate AI budgets, HM Treasury's 2026 Green Book supplies useful discipline. It separates monetisable and unmonetisable effects, requires alternatives and uncertainty to remain visible, and recommends sensitivity analysis and switching values instead of hiding fragile assumptions inside one forecast.

Count value through one economic channel

The same operational improvement can enter the business case through several economic channels. Choose the channel that will actually be used and prevent the same capacity from appearing twice.

Cost removed
A current cash expense disappears. Examples include retired contractor spend, lower processing fees, fewer outsourced reviews, or infrastructure that can be decommissioned. The saving should match an identifiable budget line.
Cost avoided
A planned expense no longer needs to occur. Avoided hiring belongs here only when demand would otherwise have required the hire and the hiring plan genuinely changes.
Contribution generated
The workflow produces additional accepted output that customers buy or that improves retention. Use incremental contribution after variable costs, rather than headline revenue.
Expected loss reduced
The intervention changes the probability or impact of fraud, defects, downtime, regulatory exposure, or another costly event. Estimate the difference between the alternatives, then expose the uncertainty in both probability and impact.
Capacity deliberately reassigned
People use released time on named work that produces a measured result. The value belongs to the result, not automatically to the salary cost of the time.

Suppose a workflow releases 2,000 hours. Finance cannot claim the salary value of all 2,000 hours, the value of extra output produced during those hours, and an avoided hire that uses the same capacity. Those are competing explanations of one benefit unless they apply to separate portions of the capacity.

This gives the business case a simple audit question: where will the gain appear, who owns that line, and what evidence will show that it moved?

Every monetised benefit needs an owner, a destination, and a comparison.

Build the full cost boundary

Model fees are visible because they arrive on an invoice. The larger cost boundary often crosses several teams and several stages of the system.

Build costs
Workflow design, engineering, integration, data preparation, evaluation, security review, procurement, and initial training.
Operating costs
Licences, inference, retrieval, storage, observability, support, human review, exception handling, and incident response.
Change costs
Process redesign, documentation, management time, temporary productivity loss, adoption support, and work abandoned during transition.
Risk costs
Expected errors, customer remediation, legal or compliance work, control failures, and the safeguards used to reduce them.
Lifecycle costs
Model migrations, repeated evaluations, prompt or tool maintenance, vendor changes, retraining, decommissioning, and exit from a failed design.

The FinOps Foundation's guidance on AI tooling calls the useful unit “use case economics”: the total cost of achieving a specific business outcome per unit of that outcome. It also notes that data preparation, retraining frequency, serving architecture, and ongoing operation can outweigh the visible model charge. This is practitioner guidance, not evidence that every AI project has the same cost structure. Its value is the boundary it forces teams to draw.

NIST adds another cost that spreadsheets commonly omit: maintaining trustworthiness after deployment. Monitoring, appeals, overrides, incident response, recovery, change management, and decommissioning require people and systems. They are part of operating the intervention, not optional governance overhead.

Past spending should not keep a weak workflow alive. The Green Book excludes sunk costs from the next decision and instead asks what future resources could do elsewhere. A failed integration may explain the history of the project. It does not improve the economics of another year.

Cheap AI does not make the return automatic

The strongest counterargument is that modern AI is so inexpensive that almost any genuine time saving clears the hurdle. A Social Capital analysis makes this case by comparing a low per-employee AI cost with average employee cost and calculating that only a few productive minutes per week would cover the subscription.

That is a useful price comparison and an incomplete investment case. The minutes have value only if they change output, spending, loss, or a constrained decision. The comparison must also include integration, review, failure, governance, and adoption costs when those costs are material. A low licence price reduces the hurdle; it does not prove that the organisation crossed it.

There is good reason to expect complementary work. Brynjolfsson, Rock, and Syverson argue that general-purpose technologies can show implementation lags because organisations need new processes, skills, and complementary innovations before productivity gains spread. Their 2017 paper is a macroeconomic explanation, not a forecast of returns from current generative-AI products. It identifies a mechanism: organisational change can consume resources before the economic benefit appears.

A more recent working paper by Kim, Kim, and Koning provides firmer but narrower evidence. In a field experiment with 515 high-growth startups, firms shown how other companies had reorganised production around AI discovered more use cases and achieved economically meaningful performance gains. The paper supports the importance of mapping AI into production. It remains a working paper in a selected startup population, so its effect sizes should not be imported into a mature enterprise business case.

Taken together, the evidence supports a practical inference: access price determines how high the hurdle is, while workflow design and organisational change determine whether there is a benefit available to clear it.

Run the numbers on one workflow

Consider a hypothetical customer-support workflow. The numbers are illustrative and produce no claim about typical AI performance.

The organisation processes 30,000 eligible cases a year. The current workflow takes 15 minutes per accepted case, including review and reopenings. A controlled pilot estimates that the AI-assisted workflow takes 11 minutes at the same quality threshold.

Baseline effort
30,000 cases multiplied by 15 minutes equals 450,000 minutes, or 7,500 hours.
AI-assisted effort
30,000 cases multiplied by 11 minutes equals 330,000 minutes, or 5,500 hours.
Capacity released
The difference is 2,000 hours a year.

The first-year incremental costs are £30,000 for integration, £12,000 for licences and inference, £8,000 for training and review operations, and £4,000 for evaluation and maintenance. Total first-year cost is £54,000.

Now change only the capture mechanism.

No capture
Staffing, contractor spend, accepted output, and customer outcomes remain unchanged. Captured benefit is £0. Net benefit is negative £54,000 and ROI is negative 100 per cent. The workflow released capacity but did not produce a financial return.
Avoided spend
An approved demand plan would otherwise require 1,200 contractor hours at £45 an hour. The workflow avoids £54,000 of spending. Net benefit is £0 and first-year ROI is 0 per cent. The investment reaches break-even; the remaining 800 released hours are still unpriced capacity.
Incremental contribution
The organisation reassigns the capacity to retention work and a credible comparison attributes £90,000 of incremental contribution to that work. Net benefit is £36,000. First-year ROI is roughly 67 per cent. If all first-year costs were paid upfront and benefit arrived evenly, simple payback would occur after about 7.2 months.

The calculation also exposes a switching value. With £54,000 of cost and 2,000 released hours, the investment needs £27 of captured benefit per released hour to break even. At £45 an hour, at least 1,200 hours must be captured. Below that point, the first-year case fails.

The arithmetic is elementary. The difficult work is proving the inputs: eligible volume, complete time per accepted case, quality parity, capture rate, value per captured unit, and total cost. That is why the economic case must remain attached to the experiment rather than appearing after it as a separate finance exercise.

Stress-test the assumptions that decide the result

One ROI percentage hides which assumptions carry the decision. Use at least a downside, base, and upside case, then calculate the switching value for the variables most likely to fail.

For an AI workflow, the decisive variables often include:

  • eligible work volume;
  • adoption and compliance;
  • complete time per accepted outcome;
  • review and exception rates;
  • the share of released capacity actually captured;
  • value per captured outcome;
  • recurring model and infrastructure cost;
  • maintenance, monitoring, and migration effort;
  • the probability and cost of material failure.

Do not vary every assumption by the same arbitrary percentage. Use pilot evidence, historical forecast errors, contract terms, operational ranges, and explicit expert judgment. HM Treasury's Green Book recommends adjusting for optimism bias with evidence from past forecasts where possible, then using sensitivity analysis to show when the preferred option stops representing value for money.

The decision should follow the fragility of the case. If a small rise in review time destroys the return, redesign before scale. If the case remains positive under conservative volume and capture assumptions, the organisation has room to learn. If an unmeasured benefit must carry the entire forecast, describe that dependency instead of converting it into invented precision.

The AI productivity experiment guide covers comparison design, guardrails, segmentation, and kill conditions. The finance model should consume that evidence, not replace it.

Keep valuable effects visible without forcing a price

Some benefits matter before they can be monetised credibly. Faster access to knowledge, employee learning, service resilience, strategic option value, accessibility, and better decision quality may justify investment. Some risk reductions are too sparse or consequential for a stable expected-loss estimate.

Record these effects beside the financial case with a measure, owner, evidence standard, and decision relevance. Do not assign a convenient currency value merely to make the spreadsheet complete.

For some investments, ROI is the wrong primary test. A mandatory compliance control may require cost-effectiveness analysis: which viable option achieves the required outcome at the lowest acceptable cost and risk? An exploratory prototype may be purchased for information: what decision will the learning unlock, and what is the maximum rational price of that information? A reversible personal tool with low cost and low downside may need only a short threshold test.

HM Treasury's guidance on evaluating AI interventions recommends matching evaluation effort to risk and learning potential. The principle travels well beyond government: analytical effort should be proportionate, while the economic claim should never exceed the evidence.

Our view: finance the capture mechanism

An AI budget should not be approved because a model is impressive, a task is faster, or the subscription is cheap. It should be approved when one record makes six elements inspectable:

  1. Decision and alternative. What is being funded, for how long, compared with which viable option?
  2. Causal evidence. What changed in the complete workflow, with what uncertainty and for which population?
  3. Value channel. Which cost, contribution, expected loss, or named use of capacity will change?
  4. Cost boundary. Which build, operating, human, risk, and lifecycle costs are incremental?
  5. Sensitivity. Which assumptions determine break-even, and where are their switching values?
  6. Accountability. Who owns benefit capture, and which result triggers scale, redesign, or stop?

This record prevents finance from valuing a technical gain that operations never captures. It also prevents operations from dismissing benefits that are real but not yet suitable for a financial headline.

Fund the mechanism that captures value, not the hours a tool claims to save.

What would change our view

The standard should become lighter when simpler evidence proves predictive.

We would give self-reported hours or isolated task-speed measures direct financial weight if they repeatedly predicted later budget reduction, avoided spending, incremental contribution, or expected-loss reduction across comparable workflows, without a separate capture plan.

We would also relax the full investment record for classes of AI intervention whose total cost, failure exposure, and reversibility remain reliably trivial. In that setting, the cost of elaborate analysis could exceed the value of the decision.

Until those conditions hold, salary multiplied by reported hours is a capacity estimate. Calling it ROI does not make it money.

Sources and further reading