Direct answer
AI strategy ROI should compare the incremental value of a use case with its full economic and control cost. The calculation is incomplete if it ignores adoption, data work, review time, integration, errors, change management and risk treatment.
The purpose of a scorecard is not to make uncertain benefits look precise. It is to improve investment, scaling and stop decisions.
Begin with the decision and baseline
Define the workflow, user, current method, volume, quality, cycle time, cost and business outcome. Without a baseline, the organization can measure activity but not improvement.
State the AI intervention precisely. "Use generative AI" is not a use case. "Draft a first version of a weekly account-risk summary from approved CRM fields for human review" is testable.
The six-part scorecard
| Dimension | Core question | Example evidence |
|---|---|---|
| Business value | What economic or service outcome changes? | Revenue, cost, loss avoided, cycle time, quality |
| Adoption | Do intended users use it correctly? | Eligible users, active use, completion, abandonment |
| Output quality | Is work accurate and useful? | Review score, error rate, rework, acceptance |
| Control quality | Are data, privacy and approvals working? | Exceptions, incidents, prohibited-input checks |
| Total cost | What does the complete system consume? | Licences, integration, data, review, training, support |
| Learning | What evidence changed our decision? | Confirmed assumptions, failed hypotheses, next test |
ROI structure
Use a range rather than one heroic number:
ROI = (incremental benefit - full incremental cost) / full incremental cost
Benefits may include additional contribution margin, productive capacity released, reduced external spend, faster cash, improved service or expected loss reduction. Time saved is not automatically cash saved. Explain whether capacity is redeployed, avoided or monetized.
Adoption is part of economics
A technically successful pilot can fail economically if users do not trust it, the workflow adds review burden or managers continue to require the old process. Track eligible users, sustained use, correct use and the outcome achieved by adopters compared with a suitable baseline.
Avoid rewarding raw prompt or token volume. High usage can indicate value, friction or waste.
Quality and human review
Define acceptable output before the trial. Use representative cases, named reviewers and error categories. Record false confidence, omissions, unsupported claims, privacy issues and harmful edge cases. The control cost belongs in the business case.
Portfolio decisions
Classify use cases as:
- scale now;
- continue controlled experiment;
- redesign workflow or data;
- retain as a limited assistive tool;
- stop.
Stopping a weak use case is evidence of governance, not failure of innovation.
A 30-60-90 day measurement sequence
First 30 days: baseline the workflow, define evidence, test data and controls, and run a small representative sample.
Days 31-60: observe real adoption, rework, exceptions and outcome quality. Compare against the baseline and refine the process.
Days 61-90: assess economics, control sustainability and scaling constraints. Make an explicit portfolio decision.
Finance partnership
Finance should challenge the baseline, benefit recognition, cost completeness, timing and confidence range. Technology should challenge architecture and operating cost. Risk, privacy and legal teams should challenge permitted data and consequences. The business owner remains accountable for the outcome.
Common failure modes
- counting pilots or generated outputs as value;
- multiplying minutes saved by salaries without testing redeployment;
- ignoring review and correction time;
- measuring only enthusiastic early adopters;
- excluding integration, data and change costs;
- treating vendor benchmarks as internal evidence;
- scaling before defining stop conditions.
Related resources
The AI Transformation research hub connects governance, data, operating model and adoption. Use the AI business prompts library for bounded decision tasks and the AI Digital Transformation program for a professional learning pathway.
The prompt library publishes reusable machine-readable exports as CSV and JSON.