I had the great pleasure of being interviewed recently by Brent Schlenker, long-time elearning advocate. We not only had a ton of fun talking, but Brent steered us into some interesting discussions.

———-

He's created a three-part video series of our discussion:

———-

Brent is a great interviewer–and he gets some top-notch folks to join him. Check out his blog.

 

The world’s best writers use editors to improve their work. Architectural engineers have their calculations reviewed to ensure structural integrity. Doctors seek second opinions on complex cases.

Unfortunately, we in the workplace learning field–and often in the education field–are often left in the dark about the strengths and weaknesses of our learning interventions. Mostly we use smile sheets (learner response forms) to get feedback even though hundreds of research studies show that smile sheets are not correlated with learning results. When we measure learning, we often do it in a way that incorporates severe forms of bias, providing ourselves with false data that pushes us into faulty decision-making.

Learning Audits enable learning professionals to get valid feedback about the strengths and weaknesses of their learning interventions. With the feedback they produce, learning audits help organizations maximize the benefits of their learning.


What is a Learning Audit?

“A learning audit is a systematic review of a learning program to determine the program’s strengths and weaknesses—with the aim to guide subsequent improvements of that learning program and/or other learning programs. Learning audits are conducted in a high-integrity manner to ensure validity and limit bias.”

Learning audits can examine classroom training, elearning, mobile learning, on-the-job learning, self-initiated learning, and academic learning. They can be relatively quick-and-dirty or they can be highly exhaustive. They can cost a little or cost a lot.

Learning audits can utilize the following data-gathering techniques:

  1. Interview learners, learners’ supervisors, learning designers, learning developers, learning deliverers, and other organizational stakeholders.
  2. Focus-group learning stakeholders—most likely in groups of similar individuals.
  3. Survey learning stakeholders.
  4. Job-shadow people as they learn and work on the job.
  5. Research-benchmark the learning program or prototype (or design intentions) based on a validated list of key learning factors (such as the Decisive Dozen).
  6. Analyze organizational artifacts (like company newsletters and bulletin boards) and other communication devices from a learning perspective.
  7. Create a list of the learning media that have been utilized.
  8. Create a list of the available learning media.
  9. Analyze the learning measurement approaches utilized.
  10. Review smile sheet results.
  11. Review results of learning assessments—especially scenario-based decisions, case studies, simulations, and realistic hands-on exercises.
  12. In addition to reviewing results that are assessed during or immediately after learning, seek to review results that assess learning after a delay of a week or more.
  13. Review on-the-job performance results that are routinely captured.
  14. Review business results, especially those that are linked to learning.
  15. Review the quality and use of prompting mechanisms (like job aids).
  16. Review the quality and use of on-the-job learning affordances, including coaching, social media, knowledge-management systems, team learning, etc.
  17. Review the supports in place for creativity-based insight learning.
  18. Review the supports in place for after-learning application.
  19. Develop and deploy improved smile sheets, learning assessments, and performance assessments.
  20. Conduct A-B testing on different versions of the same learning program.


What is Research Benchmarking?

Research Benchmarking is the process by which your learning interventions are benchmarked against research-based best practices.

Work-Learning Research excels in research benchmarking because of exhaustive work we’ve done over more than a decade compiling scientific research on learning, memory, and instruction. By combining research-based knowledge and practical wisdom, our learning audits lead the world in providing leverable recommendations for improvement.

Your learning programs will be benchmarked against the Decisive Dozen, the 12 most important learning factors. In addition, your learning ecosystem will be reviewed to look for learning and application support, learning measurement issues, and business or organizational considerations.


Want to Know More?

If you’d like to discuss learning audits further, contact me, Dr. Will Thalheimer, at 1-617-718-0767 or email me by clicking here.

If you’d like to consider conducting your own learning audits, I encourage you to view this web page.

 

Last week I launched the website LearningAudit.com to promote the practice of learning audits.

LearningAudit_Banner_2014

It is my passionate belief that our learning interventions would be tremendously improved if we took a research-based systematic approach to reviewing them. LearningAudit.com is dedicated to the proposition that we can all do this.

On the site there is the research-to-practice report, "How to Conduct a Learning Audit" and a job aid to support the learning-audit process.

I'm a bad blogger. I don't analyze my site traffic. I don't drive my readers down a purchase funnel. I don't sell advertising on my blog site. I'm a bad, bad blogger.

Two months ago, I set up Google Analytics to capture my blog's traffic. Holy heck batman! I found out something amazing, and I'm not really sure how to think about it.

Over April and May 2014, my most popular blog post–that is, the one most visited–was a blog post I published in 2006. How popular was this 2006 blog post? It accounted for 50% of all my blog traffic! Fifty freakin' percent! And April and May have been relatively busy blog posting months for me, so it wasn't like I wasn't creating new traffic.

What blog post was the envy of all the others?

It was this one, on one of the biggest myths in the learning field.

I guess this makes sense, (1) it's important, (2) the myth keeps resurfacing, and (3) by now the link has been posted in hundreds of places.

If I die today, at least I have made a bit of a difference, just in this blog post.

I'm a bad, bad blogger.  <<<<<WINK>>>>>

If you're going to the eLearning Guild's Learning Solutions Conference this coming week in Orlando, come join me–and say hello!

I'll be speaking in three sessions:

Featured Session (F2)
Subscription Learning: A Fundamentally Different Form of eLearning

Time: Wednesday March 19, 10:45AM

Details on the session

Slides for the session

Over 300 people are expected to attend. Get there early for a good seat!

Concurrent Session (105)
Serious eLearning Manifesto (Also with Clark Quinn and Michael Allen)

Time: Wednesday March 19, 1:00PM

Details on the session

We will hand out paper version of the Manifesto at the session (there are no slides)

Morning Buzz (MB31)

Time: Thursday March 20, 7:15AM

A casual conversation about the eLearning Manifesto and Instructional Design

Note: Look for Clark Quinn, or Michael Allen's name (as mine is not listed), but I'll be there!

 

Sign up today for the eLearning Guild's Thought-Leaders series — where they've asked me to reflect on my 15 years bridging the gap between research and practice. It's not until September, but sign up is now open.

Click here to view details and to sign up…

The description begins this way:

As workplace learning-and-performance professionals, we live in world of shiny toys, blinding clouds of floating ash, and darkness. While we have passion and good intentions, we are unable to maximize performance because we are infected with misinformation about how learning really works.

Should be fun!

The Work-Learning Research website is ranked as follows:

  • #4 on Google
  • #4 on Bing
  • #7 on Yahoo

When searching for "learning research."

Interestingly, we hardly ever get paid to do research. Mostly we get paid to use research wisdom to make practical recommendations, for example in the following areas:

  1. Learning Design
  2. E-Learning
  3. Training
  4. Onboarding
  5. Safety
  6. Learning Evaluation
  7. Organizational Change
  8. Leadership Development
  9. Improving the Learning Department's Results
  10. Making Fundamental Change in Your Organization's Learning Practices

Research for me is a labor of love, and also, a way to help clients cut through opinions and get to practical truths that will actually make a difference.

But still, we are happy that the world-according-to-search-engines (WOTSE) values the research perspective we offer.

And here's a secret. We don't pay any search-optimizer companies, nor do we do anything intentional to raise our search profile (who has time or money for that?). Must be our dance moves or something…

Research Benchmarking

Research Benchmarking is the process by which your learning interventions are benchmarked against research-based best practices.

While random-assignment between-group research is likely to be too expensive and time-consuming for most of us, and benchmarking our work against other industry players is likely to push us toward mediocrity, Research Benchmarking offers a potent alternative.

Learning programs are examined to determine how well they (a) create understanding, (b) support long-term remembering (minimize forgetting), and (c) motivate on-the-job performance. They will be research-benchmarked against the 12 most decisive factors in learning design.

If you’d like to discuss research benchmarking further, contact me, Dr. Will Thalheimer, at 1-888-579-9814 or email me.

 

March 2010 Speaking Events
 
If you want to see me speak, there are lots of opportunities in March. Also, I've got a screen cast you can watch below introducing my four sessions at the eLearning Guild Learning Solutions Conference.
 

Tuesday, March 2
ISPI Massachusetts Chapter
Newton, Massachusetts
TITLE: Learning Measurement: Overcoming Myths, Research Wisdom, and Full-Source Evaluation
LINK: http://www.mass-ispi.org/public/event-details.asp?ID=180

Tuesday and Wednesday March 23 and 24th
eLearning Guild, Learning Solutions Conference

Orlando, Florida

SESSION (FOUNDATION INTENSIVE): Research Answers: What is the Value of e-Learning? What Designs Work Best? Tuesday, March 23, 11:30a-12:15p
 
SESSION (ID ZONE):  Nine Biggest Mistakes in Learning Measurement. Wednesday March 24, 11:45-12:30 at the ID Zone.
 
SESSION 203: Assessing e-Learning Results: Fundamentals, Myths, and Special Opportunities. Wednesday March 24, 1:00p – 2:00p
 
SESSION 302: Improving Systems Training by Adding Informal Learning to the Blend. Wednesday March 24, 2:30p – 3:30p
 
SESSION (Free Consulting and Question-Asking at Speaker Clinic):  Wednesday March 24, 4:00p – 5:00p

Watch my brief (3 minute) introduction: http://screenr.com/5Vx

Sign up to attend the conference: http://tinyurl.com/ydyzn9o

It’s not every day that a valued client “retires” from a
company to which she has devoted a good part of her career. I’ve known Annie
Laures for almost half a decade in my role as an outside consultant and
work-learning auditor. I’d like to take a moment and acknowledge her work at
Walgreens—and wish her well as she begins a new adventure as a consultant to
chief learning officers, training directors, and learning professionals.

Annie retired as Walgreens Director of Learning Services
after a 32-year career helping Walgreens’ pharmacists, store managers, service
clerks, corporate managers and others learn and succeed in their jobs. 

Here's Annie's picture from a relaxed informal moment.

AnnieLaures

Annie’s accomplishments are too many and varied to recite
here. Here’s a short list. In her learning-executive role heading up learning
services, Annie directed the overall learning strategy for Walgreens, led
multiple teams and units of learning professionals, and worked closely with
business management and operations. Annie has also led the Performance
Development and Training and Development units at Walgreens. She’s led
leadership development. She led the transition from classroom-only training to
an integrated e-learning system. She’s created customer-service initiatives, developed
videos, done executive coaching, managed instructional design, built 360-degree
instruments, and helped Walgreens as it merged with other corporations.

Annie has long been active in the International Society of
Performance Improvement, won an award, and earned her Certified Performance
Technology designation. Annie has presented at various industry conferences,
including a presentation she and I did together titled “Is Your Learning Organization Healthy?”

Annie’s colleagues remember her as a person of integrity,
wisdom, and a deep and practical knowledge of learning. They also remember her
as someone with a quirky sense of humor. “Often
she would tell you a story that was off-the-wall but she would do it with such
a straight face that you believe her. One time she mentioned that the new hire
missed her dog so much that we agreed she could take it to work.
”

For me, Annie represented the best kind of client. She
wanted to do the right thing and create learning interventions that really
worked. She was politically savvy enough know what would fly and what would be
derailed by management. She worked within the system to get the best results
possible. Annie was also just a joy to work with—friendly, open, direct, personable,
and caring. She also had great instincts about learning and performance. Her
recommendations and thinking were almost always aligned with research and best
practices. But perhaps the most remarkable thing about Annie was her
willingness to be open to new ideas and improvement in learning-and-performance
design. Her inquisitive non-defensive nature helped her and Walgreens Learning
to continue moving forward in continuing cycles of improvement. Annie just
seemed to want to do what was right for the learners of Walgreens. It never
appeared to be about her.

When a person has a passion to make things better, when they
are focused on the general good (and not themselves so much), when they are
skilled at getting things done, and when they are open—truly open—to learning,
the world will become a better place because of their actions. To me, that
seems the story of Annie Laures career at Walgreens.

I want to acknowledge Annie’s Walgreens work and also thank
Annie for hiring me when my research-and-consulting practice really needed the
work. I also want to wish Annie the best as she opens up her consulting
practice. If you’d like to inquire about Annie’s consulting services, contact
her at this email.

Thanks Annie!!