This Friday online, I’m leading a discussion on measurement best-practices during my Brown Bag Learning event. It’s a 30-minute review of the a new job aid I’ve created to help you do better measurement. As usual, it’s research-based!!

Click to get the job aid.

Click to register (or learn more) about the webinosh.

I’ve developed three separate several job aids in regards to learning measurement over the last few years, and I decided recently that 1 was enough, so I’ve integrated the wisdom from all three into one job aid, which I now make available to you for free.

Measurementjobaid

This job aid has several advantages:

  1. It’s inspired by honest-to-goodness learning research.
  2. It fits onto one page.
  3. It provides a brief rationale for each point.
  4. It prompts users to audit their current practices.
  5. It prompts users to take action for improvement.
  6. It includes contact information for further inquiries.
  7. It covers critical measurement-design issues.
  8. It’s free.

Click here to download the job aid now.

In a provocative article today in the New York Times (June 6, 2008), Paul Krugman makes the case that all media (books, music, articles, software, etc.) providers/creators will be forced to lower prices significantly or give away their products for free. The new business model will involve selling ancillary services or products.

This will not only produce a profound shift in how the world works, but it will affect the learning-and-performance industry as well. For example, no longer will I be able to earn practically nothing for producing some of the best research-to-practice stuff on the planet. Instead, I’ll be able to earn nothing at all for the research reports I produce. No worries, I’m all right with that. I’ve got my consulting and speaking to support me.

But really, all you training providers ought to peer deeply into the future. How will this affect you?

Here’s some wild ideas:

  1. Off-the-shelf e-learning courses will get really good and be sold really cheaply to wide audiences and all the small e-learning shops across the world will collapse into three to five big powerhouses, at least for generic topics like customer-service basics, leadership basics, Microsoft Excel, etc. (maybe just within countries at first because of the need to be culturally appropriate)
  2. Companies that sell vast collections of mediocre e-learning are doomed.
  3. Custom training vendors will have to focus on learning that is strategic to their clients success and/or tailored. We’ll all have to get better talking to the business people.
  4. Training will have low profit margins, but training with consulting will have sustainable profit margins.
  5. LMS’s (and the coming talent-development upgrades) will be given away cheap, but installation, training, and maintenance will be sold for reasonable margins. Moodle will win. Well, Moodle could get other open-source competition.
  6. Training departments (or talent-development departments) will be able to purchase some things more cheaply.
  7. All the e-learning development tools that proliferate today will disappear or be reduced to niche segmentation as two to five dominant players provide relatively inexpensive yet powerful authoring tools. Think Captivate at $259 in five years, $79 in ten years.
  8. Training vendors may actually be forced to prove their value as the market shrinks and competition heats up. Can you say control-group studies done by independent unbiased evaluators. (I’m probably biased in wishing for this).
  9. Training vendors may actually be forced to pay attention to the learning research to enable them to actually get better results so they will look good on valid evaluations. (I’m probably biased in wishing for this).

     

Okay, what do you think?

 

As for my research/consulting practice, don’t worry, I’m already there. I basically make 80% of my money doing consulting, speaking and keynotes, workshops, learning audits, and providing instructional-design help. My research is done for love, because it’s the right thing to do, and because it makes me—in my not-so-humble opinion, extraordinarily good at providing consulting, workshops, etc.

Latest Update (October 2019): Here is the newest version, which I use in my presentations.

Update: Check out The Learning Landscape Video by clicking here.

Original Post Is Below:

This is the first draft of a chapter from my forthcoming book (don’t ask when, it’s a labor of love), modified somewhat to avoid references to other parts of the book (to make it whole here). Your comments, criticisms, and good ideas will be gratefully explored.

The Genesis of the Model

It’s helpful to have an overall understanding of what we’re trying to do in the learning-and-performance profession. I offer the following model as a way to frame our discussion, as a way to provide a deep conceptual map of our world—at least the world to which we should be aspiring.

I’ve been in the learning-and-performance field for almost a quarter century. I’ve been an instructional designer, a trainer, a university adjunct, a simulation architect, a project manager, a project manager, a business leader, a researcher, and a consultant. And yet, even recently, I have found myself wanting to build a better model of what we do. In the book, I’m going to offer several models that provide a good starting place for deep understanding. The first of these models I call, The Learning Landscape. I’ve been gradually building this model for years, and I recently added some additional complexity that completes the picture of what we do—or what we should aim to do.

Of course, all models are simplifications in the interest of understanding and usability. Early versions of The Learning Landscape have resonated with my clients, and I think this latest version provides additional value.

I’m going to unveil the model a piece at a time, adding complexity as this blog post progresses.

The Model’s Phases

Look at the bottom of the following diagram. You’ll notice three labels there. The learning landscape I’m describing is one in which we build a learning intervention to help our learners perform in their future performance situations (for example on the job) in an attempt to create certain beneficial learning outcomes. So for example, if we build a course to teach creative-thinking skills, we do it to help learners be more creative in their jobs and produce more innovations for their organizations.

 

From Learning Intervention to Learning Outcomes 

Look at the diagram below. It shows how a learning intervention creates performance that lead to results. In the learning intervention (box A), the learners learn—they build an understanding of the learning content. Later, in the performance situation (box C), the learner retrieves from memory the information that they learned. They also apply what they learned (box E). This successful retrieval and application enable the learner to get from the learning what they hoped to get (box F) and the organization gets the learning results it wanted to get (box G) from the investment it made.

The diagram above shows the minimum requirements for a successful learning intervention. The learners have to learn (box A), retrieve (box C), and apply (box E) what they’ve learned in order to create beneficial learning outcomes.

It would be pollyannaish for us to believe that this process always works as diagrammed above. When our learners fail to learn (box A), the whole process breaks down. You can’t retrieve what you never learned and you can’t apply what you can’t retrieve. Even when our learners fully learn a topic, at a later time they may fail to retrieve what they learned (box C). People forget information. They may forget something permanently or they may suffer temporary or contextually-induced forgetting. Learners can also learn and retrieve successfully, but not apply what they’ve learned (box E). There are many reasons that learners fail to apply what they are able to retrieve. The learning intervention might not have been sufficiently motivating to prompt the learners to apply what they learned. The incentives in the performance situation may discourage application. The learners may not have time, resources, or other competencies that enable them to be successful.

The diagram above is very helpful in understanding how our learning interventions create learning results. It highlights the obvious importance of the quality of the learning intervention itself. It also highlights the criticality of retrieval. Later we will explore in-depth how we can build learning interventions to specifically support retrieval. Finally, the diagram above highlights the importance of the performance context in supporting learners in applying what they learned. Later we will talk about how we can gain more influence in the performance situation. We’ll also discuss how to enroll learners’ managers to improve the likelihood of successful application.

Adding Other Working-Memory Processing (besides retrieval of what was learned)

The above diagram is missing a few key elements. While it nicely highlights the importance of creating retrieval, it doesn’t account for other working-memory triggers. In the diagram below, I’ve added another box (box D “Learner Responds”) to represent working-memory processing not directly related to what was previously learned in our original learning intervention.

While working-memory “retrieval” and “responding” are overlapping and often interdependent processes, I wanted to distinguish between the retrieval of information from the learning intervention and responding generated by other means, for example job aids, performance support, management prompting, and other guidance mechanisms.

While often box C and box D work in concert (as when a job aid provides guidance and also supports retrieval of what was previously learned), we need to remember that we can get our learners thinking productive thoughts without necessarily relying on course-like learning interventions.

To reiterate, there are two ways to trigger working-memory processing related to our learning efforts. First, retrieval cues can trigger our learners to remember what they’ve learned in the original learning intervention. Second, other triggers (i.e., performance-support tools) can also stimulate working-memory responding. As you probably know, sometimes it is more effective to rely on retrieval, while other times it is more effective to rely on performance support interventions. Sometimes it is better to utilize both in concert.

The Full Model 

We also should recognize that learners can learn in their performance situations. The full model below adds this performance-situation learning, what some people call informal learning.

Performance-situation learning (box B) belongs in the model because it is a powerful force in real-world performance. Our learners do a large part of their learning on the job. Whether they receive formal training or not, learners learn through their experiences at work, at play, just living their lives.

Performance-situation learning (box B) can support the learning-intervention learning (box A) by reinforcing what was originally learned, taking the learning deeper, determining real-world contingencies, and creating fluency in retrieval, among other things. One thing we need to realize as learning professionals is that we don’t necessarily have to get learners up to speed completely in our formal learning interventions—in fact it is difficult to do so. Instead of trying to cram every bit of information into our training programs, we would be far better off to design our programs with an eye to what can be learned on the job and a plan for how to support that later on-the-job learning.

From the opposite direction, learning-intervention learning (box A) can be designed specifically to help learners learn in their performance situations. We’ll talk more about this later, but briefly, we can help our learners learn by helping them notice cues they might not readily notice, by providing them with relevant mental models of how the world works, and by influencing the performance-situation itself—for example by providing reminding mechanism and getting learners’ managers involved. To highlight this point again, box A learning can influence and support box B learning and box B learning can reinforce and extend box A learning.

The Learning Landscape Summary

All models and all metaphors have limitations, even if they are brilliantly clear in simplifying complex realities into workable conceptual maps (think e=mc2). I’ve been working on this learning landscape model for years and though it seems complete and potent to me now, I imagine that in the years to come I and others will find chinks in its armor or improvements that can be made.

I admit the possibility of the model’s limitation partly because it is true (that it is likely to have limitations) and partly to model useful thought processes in learning design. Too often, our instructional-design programs have taught models as gospel, unfortunately creating instructional designers who are only able to follow recipes—and who particularly (a) are unable to be creative when unusual situations confront them, (b) are unable to create new models of increasing or context-specific usefulness, and (c) are unable/unwilling to listen and learn from the wisdom of other people and other disciplines.

The learning landscape model is intended to make the following points:

  1. Our ultimate goal should be to create beneficial learning outcomes.
  2. There are two aspects of learning outcomes—the fulfillment the learner gets from undertaking a learning effort and the learning results the organization gets from investing in learning.
  3. For formal learning interventions to produce their benefits, they must ultimately produce appropriate behaviors in some future performance situation (often a workplace on-the-job situation).
  4. For formal learning interventions to produce their benefits, they must support the learners in being able to retrieve what they’ve learned. This means specifically, that our learning interventions have to be designed to minimize forgetting and elicit spontaneous remembering.
  5. There are two ways to generate appropriate behaviors, retrieval of previously learned information and triggering of appropriate responding. These working-memory processes can work in concert. Triggering appropriate working-memory responding is an underutilized tool. We need to more aggressively look to utilize performance support tools, reminding mechanisms, and management oversight.
  6. Formal learning interventions are not the only means to produce appropriate behaviors.
  7. Learners do a lot of their learning in their performance situations. We ought to leverage that learning to reinforce and extend any formal learning that was utilized. We ought to design our formal learning interventions to improve our learners’ informal-learning opportunities.

I’ve decided to offer free webinars from time to time on very short subjects. I did my first one last week, and it went pretty well, except for a bit of operator error on my part.

In these 30-minute sessions, I’m going to focus on some of the most important issues in learning design.

Each session includes:

  • Learning News

    (brief humorous take on the week’s news)
  • Stone Pebble Nugget
    (exploration of key issue in learning design)
  • Ask Dr. Thalheimer
    (Ask me anything — think free consulting)

 
 

Upcoming Online Sessions:
(Click below to sign up)

Friday June 6th
Feedback on Correct Answers

Friday June 20th
Minimizing Forgetting 

 
 


Previous Online Sessions:

Friday May 30, 2008
The Learning Landscape