What should change after AI training?

Did people attend? Did they enjoy the programme? Do they feel more confident? Can they do useful work differently?

All four questions can matter. They are not interchangeable.

A well-received session is a good sign about the experience. It does not, by itself, establish a change in capability. A more confident participant may still need help recognising an unreliable answer.

The measurement should match the claim you intend to make.

Begin with a real task

Choose a task the programme is meant to support. Define the information available, the expected output and what makes that output usable.

Ask participants to complete a comparable task before and after the programme under documented conditions. Keep the tasks similar enough to compare, while acknowledging that repetition and familiarity may also affect performance.

The purpose is not to create a perfect research experiment inside a short course. It is to avoid claiming improvement without looking at the work.

Look at quality before speed

A faster draft is not necessarily a better completed task. Ask whether the result is accurate enough for its purpose, complete, appropriately sourced and usable by the next person.

Make the review criteria specific. “Good quality” is difficult to apply consistently. “Identifies unsupported claims and leaves them as questions for review” is something a reviewer can observe.

Include examples that should be rejected. Knowing when not to use an output is part of practical capability.

Count the whole task

Measure the time spent preparing information, generating an output, checking it, correcting it and putting it where it is needed.

For an explicitly illustrative example, imagine a task that used to take 40 minutes. The AI-assisted draft takes 15 minutes, but checking takes 10 and transfer takes another 5. The observed difference is 10 minutes, not 25.

That still might be useful. It simply describes a different result from the faster-looking generation step.

Potential time release should not be presented as cash savings unless the organisation has evidence of how that time was actually used or costs changed.

Test whether the method transfers

Can a colleague use the documented method without its author standing beside them? Do they know which information is permitted, what to check and when to ask for help?

A short teach-back or repeatability exercise can expose unclear instructions before they become a permanent part of the process. It also turns the playbook into a working resource rather than a course souvenir.

Review after the novelty

Agree a follow-up point to see which methods are still in use, what has been adapted and what has been abandoned. Ask why.

Keep the review proportionate and transparent. Participants should understand what information is collected, who sees it and how it will be used. A learning programme should not quietly become an employee-surveillance system.

Report the result honestly

Separate participant feedback, observed task performance and any measured operational change. State the sample, conditions and limitations. Leave unmeasured outcomes unclaimed.

A modest, well-described result is more useful than an impressive number nobody can explain.

Discuss a team programme to agree relevant tasks, a practical learning method and a review that matches the work.