Use case

Did your training work? How to measure training effectiveness on the job

0.07. That's the measured link between “I liked this training” and “I apply it at work,” on a scale from 0 to 1. Yet at the end of the year, it's often what we report to leadership: a completion rate and a satisfaction score.

The use case on video, in French.

Situation
An L&D manager is preparing the year-end review of their training programs for leadership
For
L&D managers, team managers, sales and customer service leaders
Duration
One measurement before the training, another one a month later
What's evaluated
One specific behavior, observed in ten people with the same rubric
In Engramm
The same scored scenario before and after training, dashboard by skill

The year-end review.

Take a sales training program delivered in the spring. By December, its review comes down to two numbers: 92% of participants completed it, and they rated it 4.6 out of 5. Those numbers say the training was taken and liked. They say nothing about what the reps have done differently in meetings since.

In 2019, ATD surveyed 779 talent development professionals about how their organizations evaluate learning programs.

8 in 10measure participant reactions and the knowledge they gained
54%measure what participants apply on the job
38%measure the impact on business results

Respondents named concrete barriers: 41% find it hard to isolate a program's influence on results, 39% lack access to the data they need, and 33% say higher-level evaluation costs too much. So L&D teams measure what's easy to measure: a survey at the end of the session and a quiz in the learning platform.

What satisfaction predicts.

In 1997, George Alliger and his colleagues pooled 34 studies and 115 correlations between training evaluation criteria. A correlation of 0 means no link, and 1 means a perfect one. Between liking a training program and applying it on the job afterward, it's 0.07. The end-of-module quiz does barely better, at 0.11.

Participants who found the training useful for their job applied it more than those who simply enjoyed it, and perceived usefulness predicted on-the-job application better than knowledge tests did. The evaluation levels and their limits are covered in the guide to the Kirkpatrick model.

Evaluating behavior on the job.

In 2012, Alan Saks and Lisa Burke surveyed 150 training professionals in Canada. The more often their organization evaluated its training, the more participants applied what they had learned. Of the four evaluation levels, only evaluations of on-the-job behavior and results went with higher transfer.

The study relies on what respondents reported and doesn't show which way the link runs. Evaluation may push people to apply what they learned, or organizations where training carries over to the job may simply evaluate more.

Medical schools watch students do it.

French medical schools settled the question differently. On May 28 and 29, 2024, across all 32 medical schools, 7,879 sixth-year students sat the first national OSCEs, objective structured clinical examinations. Each student runs a consultation with a simulated patient, and an examiner scores them on a rubric.

Knowledge is still assessed by national written exams, the EDN, which make up 60% of the score used to rank students for residency. The OSCEs count for 30%.

The same method at work.

At work, the logic is the same: one specific situation, an observation rubric, one measurement before the training and another a month later. For training on discovery, the behavior could be this one: the rep asks at least three questions about the customer's situation before presenting the offer.

You can observe it by sitting in on meetings, by listening back to recorded calls, or by having everyone take the same simulation before and after the training. Ride-alongs and call reviews show what happens with real customers. A scored simulation gives a measurement you can compare from one person to the next and from one month to the next; it shows what someone can do in a role-played situation, and the field then tells you whether they do it with customers.

Comparing ten people before and after isn't enough to prove the training caused the change. ATD points to a simple way to isolate its effect: train a third of the team this month and the next third the following month, which gives a month of comparison between people doing the same job for the same manager.

Before-and-after measurement in Engramm.

In Engramm, the situation to observe becomes a scenario: the person on the other side, their context, and what they'll say if asked the right questions. Each participant plays it out loud before the training, then a month later, and gets a scorecard built on your criteria. The dashboard compares results by team and by skill, down to each person.

The manager keeps observing in the field, through ride-alongs or call reviews, to check that what improves in simulation shows up in real meetings.

An exercise for Monday.

Take a training program you've just delivered, pick a single behavior to observe, and look at it in ten people, now and again in a month. If you'd like to talk it through on a specific case, message me.

Template to download: before-and-after observation sheet

One page to choose the behavior to observe and where to observe it, then the rubric for ten people, before the training and a month later, as a two-page PDF.

Frequently asked questions.

How do you measure training effectiveness?

Pick one specific behavior the training should change, and observe it with the same rubric before the training and a few weeks after, in the same people. Add one or two business metrics chosen up front. Satisfaction scores and quizzes complement this measurement: they tell you whether people liked the training and what they remembered.

What is delayed training evaluation?

It's an evaluation carried out several weeks or months after the training to see what participants do with it at work. Immediate evaluation happens at the end of the session and covers satisfaction or knowledge.

Which training KPIs should you track?

Completion rate and satisfaction describe participation. For effectiveness, track the share of participants who apply the target behavior, measured before and after, then the business metric the training is meant to move, such as the rate at which meetings turn into proposals.

Sources.

  • George M. Alliger, Scott I. Tannenbaum, Winston Bennett Jr., Holly Traver and Allison Shotland (1997), “A meta-analysis of the relations among training criteria,” Personnel Psychology, 50(2), 341-358: doi.org
  • Alan M. Saks and Lisa A. Burke (2012), “An investigation into the relationship between training evaluation and the transfer of training,” International Journal of Training and Development, 16(2), 118-127: doi.org
  • ATD (2019), Effective Evaluation: Measuring Learning Programs for Success, summarized by Shauna Robinson, “L&D's Struggle With Learning Evaluation,” December 6, 2019: td.org
  • Le Quotidien du Médecin, “Ecos nationaux 2024 : un succès pour les doyens,” May 31, 2024 (in French): lequotidiendumedecin.fr
  • Le Quotidien du Médecin, “Accès à l'internat : les premiers Ecos nationaux, une réussite et des critiques,” September 6, 2024 (in French): lequotidiendumedecin.fr
  • University of Strasbourg, call for simulated patients for the 2024 OSCEs (in French): unistra.fr

Geoffrey Mathern

Director of Operations, Engramm

Geoffrey Mathern has been Director of Operations at Engramm since October 2025. He works with the training teams and sales leaders of Engramm's customers, from scenario design to tracking results. Before Engramm, he spent more than ten years in digital marketing and SEO, including almost six at Fabernovel.

LinkedIn profile
Newsletter

Get the next resources.

An email every two weeks with new guides, customer stories and webinars.

See Engramm on one of your topics.

In 45 minutes, we show you the platform on a topic your teams work on.

Book a demo
Book a demo