Separate the evidence

Keep evidence layers visible

Ask what changed in the product, for users and for the organisation. State the hypothesised links and alternative explanations. Usage is not automatically value; revenue movement is not automatically caused by the feature.

UK Government guidance recommends choosing measures for the question and combining performance data with user research. The Magenta Book adds theory-of-change, context and attribution discipline. Both sources have public-sector scope.

Failure mode: every completed story is labelled value. Demonstrate the output, then test the user and outcome assumptions separately.

Worked example and transfer practice

All examples below are fictional teaching material. Use synthetic notes; no workplace or personal data is required or uploaded.

Three evidence layers in one record. Synthetic library evidence: confirmations were sent for 40 reservations (output); 30 of 36 responding borrowers identified the collection date (user evidence); failed collections fell from 8 of 40 to 4 of 40 (service outcome). These are two comparison windows, each with 40 reservations. Volunteer staffing also changed, so the decline does not establish attribution. The four nonrespondents also limit the understanding claim.

Try before reading the guide: Calculate the two failed-collection shares, name a competing explanation, and propose evidence needed before expanding the trial.

Practice review guide

The shares are 20% and 10%, a 10-percentage-point difference, not proof of causation. Check comparable windows, borrower mix, staffing and missing responses. More messages sent alone cannot establish service value.