Skip to content
All writing

How to run a four-week AI assessment before you buy anything

An AI assessment should end with two or three uses worth building, a list of ideas not worth the money, and a costed plan you keep whether or not the assessor builds it. Here is how to run one in four weeks.

Intelligent systems2 min read

An AI assessment is worth paying for when it ends in a decision. The useful output is short: the two or three uses of AI worth building in your operation, the ideas that are not worth the money and why, and a costed plan your team can take to procurement. Four weeks is enough time to get there if the work starts from the operation rather than from the technology.

It is built around what our own fixed-fee assessment delivers, and written so you can run it yourself or hold a vendor to it.

Start with where the work piles up

Spend the first week with the people who do the work, not the people who buy software. Sit with the loan desk, the claims team, the call center, the planners. Ask where requests wait, where the same document is read twice, where a decision depends on one person who is always busy. Write each one down with its volume and who owns it.

By the end of the week you should have a long list of candidate uses, each tied to a queue or a decision someone can point to. A candidate that nobody can attach to a real queue usually came from a vendor demo.

Check the data and the system each use would live in

The second week tests each candidate against two questions. Is the data the model needs captured, reachable and allowed to be used for this purpose? And which existing system would the result have to appear in for anyone to act on it?

The second question removes more candidates than the first. A fraud score that sits in a separate dashboard is ignored by the people who approve transactions. A readmission risk that does not appear in the clinical record before discharge arrives too late. Write down the system of record for each use and whether it can accept a new field, a new screen or an API call.

Rank on value, risk and the cost to run it

Week three is ranking. For each surviving use, estimate the value from the volume and the cost of the current process, the risk if the model is wrong, and what it will cost to operate after launch: monitoring, retraining, human review, audit. Teams often price the build and forget the running cost, which is where many business cases come apart after launch.

Risk decides the design. A use where a wrong answer is cheap and reversible can act on its own. A use that affects a customer's money or health needs a person approving each action and a log an auditor can read.

Write the plan for a procurement file

The last week turns the ranking into a plan: scope, sequence, the integration work in each system, the controls, the team, the cost and the measures that decide whether to continue after a pilot. Write the scope so it can go straight into a tender or a purchase order. Include the ideas you rejected and the reason for each, because the same ideas will come back from the next vendor.

The plan should belong to you. If the assessor will only release it with a build contract attached, you paid for a sales document.

Questions

How long should an enterprise AI assessment take?
Four weeks is enough for one business area: a week of discovery with the people doing the work, a week checking data and target systems, a week ranking uses on value, risk and running cost, and a week writing a costed plan.
What should an AI assessment deliver?
The two or three uses worth building, the ideas not worth the money with reasons, and a costed plan with scope written for procurement. You should keep the plan whether or not the assessor builds it.
Why do AI use cases fail after the assessment?
Often because the result does not appear in the system where people make the decision, or because the business case priced the build and left out the cost of monitoring, retraining and human review.

Tell us which system the AI has to run inside

If that system is not ready for AI yet, tell us that too. An engineer will reply and say plainly whether we are the right firm for it.