How to evaluate a kidney-risk tool in your population

Start with intended use

Ask what decision the tool is meant to support, in which population, and what it explicitly does not do. A tool that flags patients for review is not the same as one that diagnoses or rules out disease.

• Which patients is it for, and which are excluded?
• What output does the clinician actually see?
• What does the vendor say the tool must not be used for?

Check the data fit

A tool can only work from information it can see. Confirm that the inputs exist in your records, at the time the tool would run, with acceptable completeness. Missing data is a performance question, not an implementation detail.

Ask for the comparator

A risk tool is only useful if it improves on what your organization already does. The relevant comparison is usual care or a simple rule using the same information, evaluated in a population like yours — not a comparison with no triage at all.

Look at errors and subgroups

Ask how the tool behaves when it is wrong in each direction, and whether performance holds across the groups your site serves. A single overall figure can hide a subgroup where the tool is unreliable.

Workflow and follow-up

A flag is only worth what happens after it. Ask who reviews it, how long that takes, and whether confirmatory testing and referral actually get completed. Review burden and follow-up completion belong in the evaluation plan, not in a footnote.

Governance and decision criteria

Agree in advance who owns the evaluation, what data access is permitted, how results will be reported whatever they show, and what outcome would lead you to continue, change or stop. Write those criteria down before the first result arrives.