Posts

Sports is sometimes a great crucible for life lessons. Players learn teamwork, the benefits of hard work and practice, and how to act in times of success and failure.

Learning professionals can learn a lot from sports as well. The 2015 Superbowl is a case in point.

Interception

With 27 seconds to go, the Seattle Seahawks were on the New England Patriots one yard line. Only one more yard to go for victory. They called a pass play, rather controversial in the court of public opinion, but not a bad call according to statisticians.

The Seahawks quarterback, Russell Wilson, thought he had a touchdown. “I thought it was going to be a touchdown when I threw it.” Unfortunately for Wilson and the Seahawks, Malcolm Butler, a Patriots rookie cornerback, was prepared.

This is where the science of learning comes in. Butler was prepared for a number of reasons–many having to do with the science of learning. For an explanation of the 12 most important learning factors, you can review my work on the Decisive Dozen.

  1. Butler, despite being a rookie, had played a lot of football before. He had a lot of prior knowledge, which enable him to quickly learn what to do.
  2. He was given tools and resources to help him learn. He got a playbook, he was able to view videotape of Seahawks' plays, he was surrounded by experienced players and coaches, he was motivated and encouraged.
  3. He was given feedback on his performance–but not just general feedback, very specific feedback on what to do.
  4. He got many practice opportunities to refine his knowledge and performance.
  5. Perhaps most importantly, Butler was prompted to make a link between a particular situation and a particular action to take.

Here's the formation prior to the interception. Notice on the bottom of the image that the receivers for Seattle are "stacked" two deep–that is, one is lined up on the line of scrimmage, one is behind the other.

Interception-Set

  

Here is what Butler saw just as the play was getting started.

What Butler Saw

Here's what Butler said:

“I saw Wilson looking over there. He kept his head still and just looked over there, so that gave me a clue. And the stacked receivers; I just knew they were going to throw. I don’t know how I knew. I just knew. I just beat him to the point and caught the ball.”

In a separate interview he restated what he saw:

“I remembered the formation they were in, two receivers stacked, I just knew they were going to [a] pick route.”

From a science of learning perspective, what Butler did was link a particular SITUATION (two receivers stacked) with a particular ACTION he was supposed to take (move first to where the ball would be thrown). It's this cognitive linking that was so crucial to the Superbowl victory–and to human performance more generally.

While we human beings like to think of ourselves as proactive–and we are sometimes–most of our moment-to-moment actions are triggered by environmental cues. SITUATION-ACTION! It's the way the world works. When we are served food on smaller plates, we eat less–because the small plates make the food look bigger, triggering us to feel full more quickly. When we drive on a narrow street, we drive more slowly. When we see someone dressed in a suit, we think more highly of that person than if they were dressed shabbily. We can't help ourselves. What's more, these reactions are largely automatic, unintended, subconsciously triggered. Indeed, notice Butler's first quote. He wasn't sure what made him react as he did.

In the Decisive Dozen, I refer to this phenomenon as "Context Alignment." The notion is that the learning situation ought to mimic or simulate the anticipated performance situation. Others have similar notions about the importance of context, including Bransford's transfer-appropriate processing, Tulving's encoding-specificity, and Smith's context-dependent memory.

Indeed, recently a meta-analysis (a study of many studies) by Gollwitzer and Sheeran  found that "implementation intentions"–what I prefer to call "triggers"–had profound effects, often improving performance compliance by twice as much as having people set a goal to accomplish something. That is, creating a cognitive link between SITUATION and ACTION was often twice as potent as prompting people to have a goal to take a particular action.

Butler was successful because he had a trigger linking the SITUATION of stacked receivers with the ACTION of bolting to the point where the ball would be thrown.

Situation-Action

Listen to football players talk and you'll know that the best teams understand this phenomenon deeply. They talk about "picking up the keys," which is really a way of saying noticing what situation they're in on the field. Once they understand the situation, then they know what action to take. Moreover, if they can automate the situation-action link–through repeated practice–they can take actions more quickly, which can make all the difference!!

Here's how Butler talks about his preparation. When asked in an interview, "You said you knew that play was coming. How did you know that play was coming?" Butler said:

"Preparation in the [fan?] room, looking over my play book, looking over their plays, studying my opponent. I got beat on it at practice … last week, and Bill [Coach of New England Patriots] told me I got to be on it. And what I did wrong at practice I gave ground instead of just planting and going. And during game time I just put my foot in the ground, broke on the ball and beat him to the point."

For those of us working in the learning field, we should use this truth regarding the human cognitive architecture to design our learning programs.

  1. Don't just teach content.
  2. Give them tools to help them link situations and actions.
  3. Give your learners realistic practice, that is practice set in real-world situations.
  4. Give them feedback, then give them additional practice.
  5. Continually emphasize the noticing of situations, and the actions to be taken.
  6. Provide varied practice situations, without hints, to simulate real-world conditions.
  7. For critical situations, give additional practice to automate your learners' responses.
  8. Collect Lombardi Trophy or similar…

As a resident of New England, I have to add one more nugget of wisdom…

 

  

  

   

Go Patriots!

Patriots

  

Sources of Football Information, Images, Videos:

  1. Reuters
  2. NBC
  3. Boston Globe
  4. New York Times
  5. http://nflbreakdowns.com/malcolm-butlers-interception-wilson-superbowl/
  6. http://www.nfl.com/videos/new-england-patriots/0ap3000000467843/Butler-and-Edelman-go-to-Disneyland

 

 

In an article by Farhad Manjoo in the New York Times reports on Google's efforts to improve diversity. This is a compendable effort.

I was struck that while Google was utilizing scientists to devise the content of a diversity training program, it didn't seem to be utilizing research on the learning-to-performance process at all. It could be that Manjoo left it out of the article, or it could be that Google is missing the boat. Here's my commentary:

Dear Farhad,

Either this article is missing vital information–or Google, while perhaps using research on unconscious biases, is completely failing to utilize research-based best practices in learning-to-performance design. Ask almost any thought leader in the training-and-development field and they'll tell you that training by itself is extremely unlikely to substantially change behavior on its own, without additional supports.

By the way, the anecdotes cited for the success of Google's 90-minute training program are not persuasive. It's easy to find some anecdotes that support one's claims. Scientists call this "confirmation bias."

Believe it or not, there is a burgeoning science around what successful learning-to-performance solutions look like. This article, unfortunately, encourages the false notion that training programs alone will be successful in producing behavior change.

About two years ago, four enterprising learning researchers reviewed the research on training and development and published their findings in a top-tier refereed scientific journal. They did a really nice job!

Unfortunately, a vast majority of professionals in the workplace learning-and-performance field have never read the research review, nor have they even heard about it.

As a guy whose consulting practice is premised on the idea that good learning research can be translated into practical wisdom for instructional designers, trainers, elearning developers, chief learning officers and other learning executives, I have been curious to see to what extent this seminal research review has been utilized by other learning professionals. So, for the last year and a half or so, I’ve been asking the audiences I encounter in my keynotes and other conference presentations whether they have encountered this research review.

Often I use the image below to ask the question:

Click here to see original research article…

 

What would be your guess as to the percentage of folks in our industry who have read this?

10%

30%

50%

70%

90%

Sadly, in almost all of the audiences I’ve encountered, less than 5% of the learning professionals have read this research review.

Indeed, usually more than 95% of workplace learning professionals have “never heard of it” even two years after it was published!!!

THIS IS DEEPLY TROUBLING!

And the slur this dumps on our industry’s most potent institutions should be self-evident. And I, too, must take blame for not being more successful in getting these issues heard.

A Review of the Review

People who are subscribed to my email newsletter (you can sign up here), have already been privy to this review many months ago.

I hope the following review will be helpful, and remember, when you’re gathering knowledge to help you do your work, make sure you’re gathering it from sources who are mindful of the scientific research. There is a reason that civilization progresses through its scientific efforts–science provides a structured process of insight generation and testing, creating a self-improving knowledge-generation process that maximizes innovation while minimizing bias.

——————————-

Quotes from the Research Review:

“It has long been recognized that traditional,
stand-up lectures are an inefficient and
unengaging strategy for imparting
new knowledge and skills.” (p. 86)

 

“Training costs across organizations remain
relatively constant as training shifts from
face-to-face to technology-based methods.” (p. 87)

 

“Even when trainees master new knowledge and
skills in training, a number of contextual factors
determine whether that learning is applied
back on the job…” (p. 90)

 

“Transfer is directly related to opportunities
to practice—opportunities provided either by
the direct supervisor or the organization
as a whole.” (p. 90)

 

“The Kirkpatrick framework has a number of
theoretical and practical shortcomings…” (p. 91)

Introduction

I, Will Thalheimer, am a research translator. I study research from peer-reviewed scientific journals on learning, memory, and instruction and attempt to distill whatever practical wisdom might lurk in the dark cacophony of the research catacomb. It’s hard work—and I love it—and the best part is that it gives me some research-based wisdom to share with my consulting clients. It helps me not sound like a know-nothing. Working to bridge the research-practice gap also enables me to talk with trainers, instructional designers, elearning developers, chief learning officers, and other learning executives about their experiences using research-based concepts.

 

It is from this perspective that I have a sad, and perhaps horrifying, story to tell. In 2012—an excellent research review on training was published in a top-tier journal. Unbelievably, most training practitioners have never heard of this research review. I know because when I speak at conferences and chapters in our field I often ask how many people have read the article. Typically, less than 5% of experienced training practitioners have! Less than 1 in 20 people in our field have read a very important review article.

 

What the hell are we doing wrong? Why does everyone know what a MOOC is, but hardly anyone has looked at a key research article?

 

You can access the article by clicking here. You can also read my review of some of the article’s key points as I lay them out below.

 

Is This Research Any Good?

Not all research is created equal. Some is better than others. Some is crap. Too much “research” in the learning-and-performance industry is crap so it’s important to first acknowledge the quality of the research review.

The research review by Eduardo Salas, Scott Tannenbaum, Kurt Kraiger, and Kimberly Smith-Jentsch from November 2012 was published in the highly-regarded peer-reviewed scientific journal, Psychological Science in the Public Interest, published by the Association for Psychological Science, one of the most respected social-science professional organizations in the world. The research review not only reviews research, but also utilizes meta-analytic techniques to distill findings from multiple research studies. In short, it’s high-quality research.

 

The rest of this article will highlight key messages from the research review.

 

Training & Development Gets Results!

The research review by Salas, Tannenbaum, Kraiger, and Smith-Jentsch shows that training and development is positively associated with organizational effectiveness. This is especially important in today’s economy because the need for innovation is greater and more accelerated—and innovation comes from the knowledge and creativity of our human resources. As the researchers say, “At the organizational level, companies need employees who are both ready to perform today’s jobs and able to learn and adjust to changing demands. For employees, that involves developing both job-specific and more generalizable skills; for companies, it means taking actions to ensure that employees are motivated to learn.” (p. 77). Companies spend a ton of money every year on training—in the United States the estimate is $135 billion—so it’s first important to know whether this investment produces positive outcomes. The bottom line: Yes, training does produce benefits.

 

To Design Training, It Is Essential to Conduct a Training Needs Analysis

“The first step in any training development effort ought to be a training needs analysis (TNA)—conducting a proper diagnosis of what needs to be trained, for whom, and within what type of organizational system. The outcomes of this step are (a) expected learning outcomes, (b) guidance for training design and delivery, (c) ideas for training evaluation, and (d) information about the organizational factors that will likely facilitate or hinder training effectiveness. It is, however, important to recognize that training is not always the ideal solution to address performance deficiencies, and a well-conducted TNA can also help determine whether a non-training solution is a better alternative.” (p. 80-81) “In sum, TNA is a must. It is the first and probably the most important step toward the design and delivery of any training.” (p. 83) “The research shows that employees are often not able to articulate what training they really need” (p. 81) so just asking them what they need to learn is not usually an effective strategy.

 

Learning Isn’t Always Required—Some Information can be Looked Up When Needed

When doing a training-needs analysis and designing training, it is imperative to separate information that is “need-to-know” from that which is “need-to-access.” Since learners forget easily, it’s better to use training time to teach the need-to-know information and prepare people on how to access the need-to-access information.

 

Do NOT Offer Training if It is NOT Relevant to Trainees

In addition to being an obvious waste of time and resources, training courses that are not specifically relevant to trainees can hurt motivation for training in general. “Organizations are advised, when possible, to not only select employees who are likely to be motivated to learn when training is provided but to foster high motivation to learn by supporting training and offering valuable training programs.” (p. 79) This suggests that every one of the courses on our LMS should have relevance and value.

 

It’s about Training Transfer—Not Just about Learning!

“Transfer refers to the extent to which learning during training is subsequently applied on the job or affects later job performance.” (p. 77) “Transfer is critical because without it, an organization is less likely to receive any tangible benefits from its training investments.” (p. 77-78) To ensure transfer, we have to utilize proven scientific research-based principles in our instructional designs. Relying on our intuitions is not enough—because they may steer us wrong.

 

We must go Beyond Training!

“What happens in training is not the only thing that matters—a focus on what happens before and after training can be as important. Steps should be taken to ensure that trainees perceive support from the organization, are motivated to learn the material, and anticipate the opportunity to use their skills once on (or back on) the job.” (p. 79)

 

Training can be Designed for Individuals or for Teams

“Today, training is not limited to building individual skills—training can be used to improve teams as well.” (p. 79)

 

Management and Leadership Training Works

“Research evidence suggests that management and leadership development efforts work.” (p. 80) “Management and leadership development typically incorporate a variety of both formal and informal learning activities, including traditional training, one-on-one mentoring, coaching, action learning, and feedback.” (p. 80)

 

Forgetting Must Be Minimized, Remembering Must Be Supported

One meta-analysis found that one year after training, “trainees [had] lost over 90% of what they learned.” (p. 84) “It helps to schedule training close in time to when trainees will be able to apply what they have learned so that continued use of the trained skill will help avert skill atrophy. In other words, trainees need the chance to ‘use it before they lose it.’ Similarly, when skill decay is inevitable (e.g., for infrequently utilized skills or knowledge) it can help to schedule refresher training.” (p. 84)

 

Common Mistakes in Training Design Should Be Avoided

“Recent reports suggest that information and demonstrations (i.e., workbooks, lectures, and videos) remain the strategies of choice in industry. And this is a problem [because] we know from the body of research that learning occurs through the practice and feedback components.” (p. 86) “It has long been recognized that traditional, stand-up lectures are an inefficient and unengaging strategy for imparting new knowledge and skills.” (p. 86) Researchers have “noted that trainee errors are typically avoided in training, but because errors often occur on the job, there is value in training people to cope with errors both strategically and on an emotional level.” (p. 86) “Unfortunately, systematic training needs analysis, including task analysis, is often skipped or replaced by rudimentary questions.” (p. 81)

 

Effective Training Requires At Least Four Components

“We suggest incorporating four concepts into training: information, demonstration, practice, and feedback.” (p. 86) Information must be presented clearly and in a way that enables the learners to fully understand the concepts and skills being taught. Skill demonstrations should provide clarity to enable comprehension. Realistic practice should be provided to enable full comprehension and long-term remembering. Proving feedback after decision-making and skill practice should be used to correct misconceptions and improve the potency of later practice efforts.

The bottom line is that more realistic practice is needed. Indeed, the most effective training utilizes relatively more practice and feedback than is typically provided. “The demonstration component is most effective when both positive and negative models are shown rather than positive models only.” (p. 87)

Will’s Note: While these four concepts are extremely valuable, personally I think they are insufficient. See my research review on the Decisive Dozen for my alternative.

 

E-Learning Can Be Effective, But It May Not Lower the Cost of Training

“Both traditional forms of training and technology-based training can work, but both can fail as well. (p. 87) While the common wisdom argues that e-learning is less costly, recent “survey data suggest that training costs across organizations remain relatively constant as training shifts from face-to-face to technology-based methods.” (p. 87) This doesn’t mean that e-learning can’t offer a cost savings, but it does mean that most organizations so far haven’t realized cost savings. “Well-designed technology-based training can be quite effective, but not all training needs are best addressed with that approach. Thus, we advise that organizations use technology-based training wisely—choose the right media and incorporate effective instructional design principles.” (p. 87)

 

Well-Designed Simulations Provide Potent Learning and Practice

“When properly constructed, simulations and games enable exploration and experimentation in realistic scenarios. Properly constructed simulations also incorporate a number of other research-supported learning aids, in particular practice, scaffolding or context-sensitive support, and feedback. Well-designed simulation enhances learning, improves performance, and helps minimize errors; it is also particularly valuable when training dangerous tasks. (p. 88)

 

To Get On-the-Job Improvement, Training Requires After-Training Support

“The extent to which trainees perceive the posttraining environment (including the supervisor) as supportive of the skills covered in training had a significant effect on whether those skills are practiced and maintained.” (p. 88) “Even when trainees master new knowledge and skills in training, a number of contextual factors determine whether that learning is applied back on the job: opportunities to perform; social, peer, and supervisory support; and organizational policies.” (p. 90) A trainee’s supervisor is particularly important in this regard. As repeated from above, researchers have “discovered that transfer is directly related to opportunities to practice—opportunities provided either by the direct supervisor or the organization as a whole.” (p. 90)

 

On-the-Job Learning can be Leveraged with Coaching and Support

“Learning on the job is more complex than just following someone or seeing what one does. The experience has to be guided. Researchers reported that team leaders are a key to learning on the job. These leaders can greatly influence performance and retention. In fact, we know that leaders can be trained to be better coaches…Organizations should therefore provide tools, training, and support to help team leaders to coach employees and use work assignments to reinforce training and to enable trainees to continue their development.” (p. 90)

 

Trainees’ Supervisors Can Make or Break Training Success

Researchers have “found that one misdirected comment by a team leader can wipe out the full effects of a training program.” (p. 83) “What organizations ought to do is provide leaders with information they need to (a) guide trainees to the right training, (b) clarify trainees’ expectations, (c) prepare trainees, and (d) reinforce learning…” (p. 83) Supervisors can increase trainees’ motivation to engage in the learning process. (p. 85) “After trainees have completed training, supervisors should be positive about training, remove obstacles, and ensure ample opportunity for trainees to apply what they have learned and receive feedback.” (p. 90) “Transfer is directly related to opportunities to practice—opportunities provided either by the direct supervisor or the organization.” (p. 90)

 

Will’s Note: I’m a big believer in the power of supervisors to enable learning. I’ll be speaking on this in an upcoming ASTD webinar.

 

Basing Our Evaluations on the Kirkpatrick 4 Levels is Insufficient!!!

“Historically, organizations and training researchers have relied on Kirkpatrick’s [4-Level] hierarchy as a framework for evaluating training programs…[Unfortunately,] The Kirkpatrick framework has a number of theoretical and practical shortcomings. [It] is antithetical to nearly 40 years of research on human learning, leads to a checklist approach to evaluation (e.g., ‘we are measuring Levels 1 and 2, so we need to measure Level 3’), and, by ignoring the actual purpose for evaluation, risks providing no information of value to stakeholders… Although the Kirkpatrick hierarchy has clear limitations, using it for training evaluation does allow organizations to compare their efforts to those of others in the same industry. The authors recommendations for improving training evaluation fit into two categories. First, [instead of only using the Kirkpatrick framework] “organizations should begin training evaluation efforts by clearly specifying one or more purposes for the evaluation and should then link all subsequent decisions of what and how to measure to the stated purposes.” (p. 91) Second, the authors recommend that training evaluations should “use precise affective, cognitive, and/or behavioral measures that reflect the intended learning outcomes.” (p. 91)

 

This is a devastating critique that should give us all pause. Of course it is not the first such critique, nor will it have to be the last I’m afraid. The worst part about the Kirkpatrick model is that it controls the way we think about learning measurement. It doesn’t allow us to see alternatives.

 

Leadership is Needed for Successful Training and Development

“Human resources executives, learning officers, and business leaders can influence the effectiveness of training in their organizations and the extent to which their company’s investments in training produce desired results. Collectively, the decisions these leaders make and the signals they send about training can either facilitate or hinder training effectiveness…Training is best viewed as an investment in an organization’s human capital, rather than as a cost of doing business. Underinvesting can leave an organization at a competitive disadvantage. But the adjectives “informed” and “active” are the key to good investing. When we use the word “informed,” we mean being knowledgeable enough about training research and science to make educated decisions. Without such knowledge, it is easy to fall prey to what looks and sounds cool—the latest training fad or technology.”  (p. 92)

Thank you!

I’d like to thank all my clients over the years for hiring me as a consultant, learning auditor, workshop provider, and speaker–and thus enabling me to continue in the critical work of translating research into practical recommendations.

If you think I might be able to help your organization, please feel free to contact me directly by emailing me at “info at worklearning dot com” or calling me at 617-718-0767.

 

On January 30th, there will be an online conference, costing only $65, hosted in Second Life (so you can learn about Second Life as well), and it sounds intriguing with some solid speakers/hosts.

Check it out. It's called Stepping Into Virtual Worlds.