21st December 2020

Neon Elephant Award Announcement

Dr. Will Thalheimer, President of Work-Learning Research, Inc., announces the winner of the 2020 Neon Elephant Award, given to Mirjam Neelen and Paul Kirschner for writing the book, Evidence-Informed Learning Design: Use Evidence to Create Training Which Improves Performanceand for their many years publishing their blog 3-Star Learning Experiences.

Click here to learn more about the Neon Elephant Award…

2020 Award Winners – Mirjam Neelen and Paul Kirschner

Mirjam Neelen is one of the world’s most accomplished research-practitioners in the workplace learning field. On the practical side, Mirjam has played many roles. As of this writing, she is the Head of Global Learning Design and Learning Sciences at Novartis. She has been a Learning Experience Design Lead at Accenture and at the Learnovate Centre in Dublin, an Instructional Designer at Google, and Instructional Design Lead at Houghton Mifflin Harcourt. Mirjam utilizes evidence-informed wisdom in her work and also partners with Paul A. Kirschner in the 3-Star Learning Experience blog to bring research and evidence-informed insights to the workplace learning field. Mirjam is a member of the Executive Advisory Board of The Learning Development Accelerator.

Paul A. Kirschner is Professor Emeritus at the Open University of the Netherlands and owner of kirschner-ED, an educational consulting practice. Paul is an internationally recognized expert in learning and educational research, with many classic studies to his name. He has served as President of the International Society for the Learning Sciences, is an AERA (American Education Research Association) Research Fellow (the first European to receive this honor). He has published several very successful books: Ten Steps to Complex Learning, Urban Myths about Learning and Education. More Urban Myths about Learning and Education, and this year he published How Learning Happens: Seminal Works in Educational Psychology and What They Mean in Practice with Carl Hendrick — as well as the book he and Mirjam are honored for here. Kirschner previously won the Neon Elephant Award in 2016 for the book Urban Myths about Learning and Education written with Pedro De Bruyckere and Casper D. Hulshof. Also, Paul’s co-author on the Ten-Steps book, Jeroen van Merriënboer, won the Neon-Elephant award in 2011.

Relevant Websites

Mirjam’s and Paul’s book, Evidence-Informed Learning Design was published only ten months ago, but has already swept the world as a book critical to learning architects and learning executives in their efforts to build the most effective learning designs. In my book review earlier this year I wrote, “Mirjam Neelen and Paul Kirschner have written a truly beautiful book—one that everyone in the workplace learning field should read, study, and keep close at hand. It’s a book of transformational value because it teaches us how to think about our jobs as practitioners in utilizing research-informed ideas to build maximally effective learning architectures.”

Mirjam Neelen and Paul Kirschner are the kind of research translators we should honor and emulate in the workplace learning field. They are unafraid in seeking the truth, passionate in sharing research- and evidence-informed wisdom, dogged in compiling research from scientific journals, and thoughtful in making research ideas accessible to practitioners in our field. It is an honor to recognize Mirjam and Paul as this year’s winners of the Neon Elephant Award.

 

Click here to learn more about the Neon Elephant Award…

In 2016 I published a book on how to radically transform learner surveys into something useful. The book won an award from ISPI and helped thousands of companies update their smile sheets. Now, I’m updating the book with the knowledge I’ve gained in consulting with companies in the learning-evaluation efforts. The second edition will be titled: Performance-Focused Learner Surveys: A Radical Rethinking of a Dangerous Art Form (Second Edition).

In the first edition, I listed nine benefits of learner surveys, but I had only touched the surface. In the coming book, I will offer 20 benefits. Here’s the current list:

Supporting Learning Design Effectiveness

  1. Red-flagging training programs that are not sufficiently effective.
  2. Gathering ideas for ongoing updates and revisions of learning programs.
  3. Judging the strengths, weaknesses, and viability of program updates and pilots.
  4. Providing learning architects and trainers with feedback to aid their development.
  5. Judging the competence of learning architects and trainers.
  6. Judging the contributions to learning made by people outside of the learning team.
  7. Assessing the contributions of learning supports and organizational practices.

Supporting Learners in Learning and Application

  1. Helping learners reflect on and reinforce what they learned.
  2. Helping learners determine what (if anything) they plan to do with their learning.
  3. Nudging learners to greater learning and application efforts.

Nudging Action Through Stealth Messaging

  1. Guiding learning architects to create more effective learning by sharing survey questions before learning designs are finalized and sharing survey results after data is gathered.
  2. Guiding trainers to utilize more effective learning methods by sharing survey questions before learning designs are finalized and sharing survey results after data is gathered.
  3. Guiding organizational stakeholders to support learning efforts more effectively by sharing survey questions and survey results.
  4. Guiding organizational decision makers to better appreciate the complexity and depth of learning and development—helping the learning team to gain credibility and autonomy.

Supporting Relationships with Learners and Other Key Stakeholders

  1. Capturing learner satisfaction data to understand—and make decisions that relate to—the reputation of the learning intervention and/or the instructors.
  2. Upholding the spirit of common courtesy by giving learners a chance for feedback.
  3. Enabling learner frustrations to be vented—to limit damage from negative back-channel communications.

Maintaining Organizational Credibility

  1. Engaging in visibly credible efforts to assess learning effectiveness.
  2. Engaging in visibly credible efforts to utilize data to improve effectiveness.
  3. Reporting out data to demonstrate learning effectiveness.

If you want to learn when the new edition is available, sign up for my list. https://www.worklearning.com/sign-up/.

The second edition will include new and improved question wording, additional questions, additional chapters, etc.

Matt Richter and I, in our Truth-in-Learning Podcast, will be discussing learner surveys in our next episode. Matt doesn’t believe in smile sheets and I’m going to convince him of the amazing power of well-crafted learner surveys. This blog post is my first shot across the bow. To join us, subscribe to our podcast in your podcast app.

Design Thinking is all the rage! Even now in the Learning & Development field. A powerful methodology if done right, but a process that can go wrong when applied inappropriately to learning design. In their new book, Sharon Boller and Laura Fletcher show us how to use design thinking right—when we apply it to learning design.

 

Background

First, a little background on design thinking. The folks from IDEO, one of the exemplars of design thinking, say this about their work, “Human-centered designers are unlike other problem solvers—we tinker and test, we fail early and often, and we spend a surprising amount of time not knowing the answer to the challenge at hand. And yet, we forge ahead. We’re optimists and makers, experimenters and learners, we empathize and iterate, and we look for inspiration in unexpected places.”

Let me highlight two key things: Iteration and Empathy.

Too often in learning design and development, we design in a lockstep ADDIE fashion, from point A to point E. Instead of rapid prototyping and improvement, we are so insular in our overconfidence, that we build learning that just doesn’t work that well.

We also have a tendency to trust our learners too much with their learning design intuitions, even though tons of research shows us that learners have large misconceptions about learning. Some of us have taken the empathy notion from design thinking too far, blindly trusting learners. The history of this goes back long before design thinking to the harmful optimism of Malcolm Knowles and his theory of Andragogy.

The Fix

Sharon Boller and Laura Fletcher have long been believers in evidence-based practice. Just as importantly, they spent years utilizing and fine-tuning their learning development processes to include design-thinking principles and techniques. In Design Thinking for Training and Development: Creating Learning Journeys that Get Results, Sharon and Laura blend evidence and practice into workable and pragmatic guidelines for using design thinking. They integrate the power of design thinking and eliminate the wrong turns that happen when research and evidence is ignored.

Even in the subtitle, “Creating learning journeys that get results,” you can tell that Sharon and Laura are talking about creating meaningful learning designs. They seek out learner data, but put safeguards on this process, including special review iterations, routine prototyping, and meaningful evaluation.

The Upshot

Boller and Fletcher, in Design Thinking for Training and Development, build a powerful tool-set for learning professionals. By augmenting Design Thinking with research-based wisdom and practical insights about learning, the book provides a new learning-development methodology—a worthy replacement for learning-neutral processes like ADDIE.

The book is available tomorrow on Amazon. Click on these words to get the book.

 

Today is May 15, 2020.

This week, I asked my publisher to take all 254 remaining copies of Performance-Focused Smile Sheets and throw them in the recycling bin!

Yes! The book was one of the most important books written in the learning-and-development field over the last decade.

But I threw it out!

Why?

Because I’m writing a new, completely revised edition based on my experience developing learner surveys for organizations around the world. My learner-survey approach has been used at The Centers for Disease Control and Prevention, the World Bank, at Bloomberg, at Roche (Genentech), at Oxfam. It has revolutionized the moribund smile sheet!

But time doesn’t stand still, and I’ve gained a ton of insights building learner surveys over the past five years, hearing from clients and others, pilot testing learner surveys, and creating smile sheets for my own workshops.

The new book will have an all new set of recommended questions. It will have information on open-ended questions. It will describe how learner surveys fit into the new LTEM framework (the learning-evaluation model that is replacing the Kirkpatrick-Katzell Four-Level Model around the world). The new book will also share the thinking behind the new learning-evaluation approach, LEADS (Learning Evaluation As Decision Support), which I’m developing.

When will the new book be out? Within three months!

The book may still be available in some pipelines, but I’m working to get it off the shelves. My advice is don’t buy it!

Wait for the second edition, which I’ve re-titled, Performance-Focused Learner Surveys.

Stay tuned!

And sign up for my newsletter at https://www.worklearning.com/sign-up/ to be the first to know when the book is available.

Thanks for your patience,

= Will Thalheimer

Mirjam Neelen and Paul Kirschner have written a truly beautiful book—one that everyone in the workplace learning field should read, study, and keep close at hand. It’s a book of transformational value because it teaches us how to think about our jobs as practitioners in utilizing research-informed ideas to build maximally effective learning architectures.

Their book is titled, Evidence-Informed Learning Design: Use Evidence to Create Training Which Improves Performance. The book warns us of learning myths and misconceptions—but it goes deeper to bring us insights in how these myths arise and how we can disarm them in our work.

Here’s a picture of me and my copy! The book officially goes on sale today in the United States.

 

Click to get your copy of the book from Amazon (US).

The book covers the most powerful research-informed learning factors known by science. Those who follow my work will hear familiar terms like Feedback, Retrieval Practice, Spacing; but also, terms like double-barreled learning, direct instruction, nuanced design, and more. I will keep this book handy in my own work as a research-inspired consultant, author, provocateur—but this book is not designed for people like me. Evidence-Informed Learning Design is perfect for everyone with more than a year of experience in the workplace learning field.

The book so rightly laments that “the learning field is cracked at its foundation.” It implores us to open our eyes to what works and what doesn’t, and fundamentally to rethink how we as practitioners work in our teams to bring about effective learning.

The book intrigues as can be seen in sections like, “Why myths are like zombies,” and “No knowledge, no nothing,” and “Pigeonholing galore.”

One of my favorite parts of the book is the interviews of researchers that delve into the practical ramifications of their work. There are interviews with an AI expert, a neuroscientist, and an expert on complex learning, among others. These interviews will wake up more than a few of us.

What makes this book so powerful is that it combines the work of a practitioner and a researcher. Mirjam is one of our field’s most dedicated practitioners in bringing research inspirations to bear on learning practice. Paul is one of the great academic researchers in doing usable research and bringing that research to bear on educational practice. Together, for many years, they’ve published one of the most important blogs in the workplace learning field, the Three-Star Learning blog (https://3starlearningexperiences.wordpress.com/).

Here are some things you will learn in the book:

Big Picture Concepts:

  • What learning myths to avoid.
  • What learning factors to focus on in your learning designs.
  • How to evaluate research claims.

Specific Concepts:

  • Whether Google searches can supplant training.
  • What neuroscience says about learning, if anything.
  • How to train for complex skills.
  • How AI might help learning, now and in the future.
  • Types of research to be highly skeptical of.
  • Whether you need to read scientific research yourself.
  • Whether you should use learning objectives, or not, or when.
  • Whether learning should be fun.
  • The telltale signs of bad research.

This book is so good that it should be required reading for everyone graduating at the university level in learning-and-development.

 

 

Click on the book image to see it on Amazon (US).

 

 

 

12th December 2019

Neon Elephant Award Announcement

Dr. Will Thalheimer, President of Work-Learning Research, Inc., announces the winner of the 2019 Neon Elephant Award, given to David Epstein for writing the book Range: Why Generalists Triumph in a Specialized World, and for his many years as a journalist and science-inspired truth teller.

Click here to learn more about the Neon Elephant Award…

 

2019 Award Winner – David Epstein

David Epstein, is an award-winning writer and journalist, having won awards for his writing from such esteemed bodies as the National Academies of Sciences, Engineering, and Medicine, the Society of Professional Journalists, and the National Center on Disability and Journalism—and has been included in the Best American Science and Nature Writing anthology. David has been a science writer for ProPublica and a senior writer at Sports Illustrated where he helped break the story on baseball legend Alex Rodriguez’s steroid use. David speaks internationally on performance science and the uses (and misuses) of data and his TED talk on human athletic performance has been viewed over eight million times.

Mr. Epstein is the author of two books:

David is honored this year for his new book on human learning and development, Range: Why Generalists Triumph in a Specialized World. The book lays out a very strong case for why most people will become better performers if they focus broadly on their development rather than focusing tenaciously and exclusively on one domain. If we want to raise our children to be great soccer players (aka “football” in most places), we’d be better off having them play multiple sports rather than just soccer. If we want to develop the most innovative cancer researchers, we shouldn’t just train them in cancer-related biology and medicine, we should give them a wealth of information and experiences from a wide range of fields.

Range is a phenomenal piece of art and science. Epstein is truly brilliant in compiling and comprehending the science he reviews, while at the same time telling stories and organizing the book in ways that engage and make complex concepts understandable. In writing the book, David is debunking the common wisdom that performance is improved most rapidly and effectively by focusing practice and learning toward a narrow foci. Where others have only hinted at the power of a broad developmental pathway, Epstein’s Range builds up a towering landmark of evidence that will remain visible on the horizon of the learning field for decades if not millennium.

We in the workplace learning-and-development field should immerse ourselves in Range—not just in thinking about how to design learning and architect learning contexts, but also in thinking about how to evaluate prospects for recruitment and hiring. It’s likely that we currently undervalue people with broad backgrounds and artificially overvalue people with extreme and narrow talents.

Here is a nice article where Epstein wrestles with a question that elucidates an issue we have in our field—what happens when many people in a field are not following research-based guidelines. The article is set in the medical profession, but there are definite parallels to what we face everyday in the learning field.

Epstein is the kind of person we should honor and emulate in the workplace learning field. He is unafraid in seeking the truth, relentless and seemingly inexhaustible in his research efforts, and clear and engaging as a conveyor of information. It is an honor to recognize him as this year’s winner of the Neon Elephant Award.

 

Click here to learn more about the Neon Elephant Award…

Today, Ulrich Boser released an updated paperback version of his book, Learn Better: Mastering the Skills for Success in Life, Business, and School… It is available on Amazon (make sure you get the paperback version).

Ulrich does good work and his book has been hailed by Walter Isaacson as “Alternately humorous, surprising, and profound,” and by Amazon Editors as one of the Best Science Books of the Year.

You can learn more about Ulrich’s work at his website.

 

The Debunker Club — where I am an organizer — is sponsoring a members-only Book Group Discussion of The Knowledge Illusion — by Steven Sloman & Philip Fernbach.

This book is fascinating, laying out the argument that human cognition, because it is so resource intensive, is something that we humans tend to offload to others. That is, we have a tendency to avoid the hard work of learning when we can rely on simple heuristics or objects in our environment to inform our actions or other people who have more knowledge.

The book’s discussions are focused on knowledge, and have great relevance to those of us in the learning field.

If you’re a Debunker Club member, please join the community book discussion group starting tomorrow January 11th. You can join the discussion by clicking here.

Note: The Discussion will unfold over several months asynchronously and chapter by chapter so people from around the world can easily join. Don’t worry if you haven’t read the book yet. Grab it and join us.

If you’re not a member, it’s easy to join The Debunker Club. You can join by clicking here.

More information about The Debunker Club can be found by clicking here. We have over 800 members from around the world dedicated to eliminating learning myths and sharing evidence-based practices.

This is my preface to Clark Quinn’s book on debunking the myths in the learning field, Millennials, Goldfish & Other Training Misconceptions: Debunking Learning Myths and Superstitions. (available from Amazon here).

Clark Stanley worked as cowboy and later as a very successful entrepreneur, selling medicine in the United States that he made based on secrets he learned from an Arizona Hopi Indian medicine man. His elixir was made from rattlesnake oil, and was marketed in the 1890’s through public events in which Stanley killed live rattlesnakes and squeezed out their oil in front of admiring crowds. After his medicine gained a wide popularity, Stanley was able to set up production facilities in Massachusetts and Rhode Island with the help of a pharmacist. Stanley made himself a rich man.

You may not know his name, but you’ve certainly heard of his time and place. It was the era of patent medicines—false and sometimes dangerous elixirs sold to men and women of all stripes. Dr. Kilmer’s Swamp Root. Oxien. Kickapoo Indian Sagwa. Dr. Morse’s Indian Root Pills. Enzyte. Bonnore’s Electro Magnetic Bathing Fluid. Radithor. Liquozone. And of course, Clark Stanley’s Snake Oil Liniment.

These medicines were bought by the millions. Fortunes were made. Millions of people were bamboozled, made sick, killed or murdered depending on how you see it. It turns out that, upon being tested, Stanley’s elixir was found to be made mostly from mineral oil, a worthless potion sold by a charlatan. His story of the medicine man and the rattlesnake juice was a more potent concoction than his famous elixir, which when tested was found to have no snake oil anyway.

What causes men and women to miss the truth, to fail to see, to continue happily in harming themselves and those around them? This, unfortunately, is not a question just for the era of patent medicines. It is eternal. It goes back to the dawn of humanity and continues today as well. I have no answer except to assume that our credulity is part of our humanity—and should guide us to be on guard at all times.

What stopped the patent-medicine pandemic of poison, persuasion, and placebo? Did we the people rise up on our own and throw out the scoundrels, the money-grubbers, the snake-oil salesmen? Did we see that we were deceived, or too hopeful, or too blind? Did we as a community heed our senses and find a way to overcome the dangers hidden from us?

No! We did not!

It was not a mass movement back to rationality and truth that saved us. It was the work of a few intrepid agitators who made all the difference. Journalists began reporting on deaths, sicknesses, and addictions resulting from the use of patent medicines. In 1905, Collier’s Weekly published a cover story that exploded the industry. Written by Samuel Hopkins Adams a former crime reporter, with the title, “The Great American Fraud: The Patent Medicine Evil,” the long expose contained sections with headings like, “Medicine or Liquor?”, “The Men Who Back the Fake,” “Absolutely False Claims,” “Drugs that Deprave,” “Prescribing Without Authority,” and “Where the Money Goes.”

The article—or series of articles that today we would call investigative journalism—opened the floodgates and led directly to the Pure Food and Drug Act in 1906, followed later by additional regulations and requirements that continue to this day, with some success, protecting our health and safety.

The ugly truth is that we need help in seeing what we don’t see. This is true too in the learning industry and has been true since at least the early 1900’s when thought leaders in our industry floated bogus claims that people remember 10% of what they read, 20% of what they hear, 30% of what they see, et cetera. Indeed, it was partly the bogus claims floating around the learning industry in the late 1990’s that made me optimistic that starting a research-based consulting practice would find an audience, that perhaps the learning field could be protected from snake oil charlatans.

Bogus claims are not merely inert flotsam to be navigated around. At a minimum, they take attention away from learning practices that are more fundamental and effective, pushing us to waste time and resources. More insidious is that they proactively cause harm, hurting learners and weakening our learning outcomes.

I wish I could report that starting Work-Learning Research twenty years ago has had the influence that Samuel Hopkins Adams had in his journalism. Alas, I am a faint voice in the howling wind of our industry. Fortunately, there are many muckraking research-to-practice practitioners today, including folks like Paul Kirschner, Patti Shank, Guy Wallace, Pedro De Bruyckere, Julie Dirksen, Donald Clark, Ruth Clark, Mirjam Neelen, Jane Bozarth, and more. There are also legions of academic researchers who do the science necessary to enable research-to-practice wisdom to be compiled and conveyed to trainers, instructional designers, elearning developers and learning executives.

I am especially optimistic now that Clark Quinn has compiled, for the first time, the myths, misconceptions, and confusions that imbue the workplace learning field with faulty decision making and ineffective learning practices. As Clark rightly advises, don’t read the book in one sitting. You will find it too much—too many misconceptions and malingering falsehoods, and too much heartache to think that our field could tolerate so much snake oil.

Here’s what we don’t realize. Today’s workplace-learning snake oil is costing us billions of dollars in wasted effort, misspent resources, ill-advised decisions, and distraction from the science-of-learning fundamentals that have proven to be effective! Every time a trainer reads an article on learning styles and adjusts his or her training to make it suitable for visual, auditory, kinesthetic, and olfactory learners; time is wasted, money is spent, and learning is hurt. Every time an instructional designer goes to a conference and hears that neuroscience should guide learning design, he or she takes this faulty meme back to colleagues and infects them with false hope and ineffective learning strategies. Every time a Chief Learning Officer hears that learning events should be shrunk to 4-minute microlearning videos, that storytelling is everything, that all learning is social, that virtual reality is the future of learning—every time our learning executives jump on a bandwagon and send us to training or conferences or hire experts in these multitudinous fascinations—we are diverted from the veritable essence of learning. We waste our own developmental budgets with snake-oil rostrums. We waste time organizing ourselves around wrong-headed initiatives. We ignore what really works, all the while costing our organizations billions of dollars in waste and ineffective learning practices.

Let us start anew today. We can begin with Clark’s book. It is a veritable treasure chest of wisdom. But let’s keep going. Let’s stay skeptical. Let’s look to the scientific research for knowledge. Let’s become more demanding and knowledgeable ourselves, knowing that we all have more to learn. Let’s look to the research translators who know the work that we do as instructional designers, trainers, and developers. Let’s do our own testing. Let’s improve our evaluation systems so that we get better feedback day by day. Let’s pilot, rework, improve, and continue to learn!

As the history of patent medicine shows, we must be forever vigilant against our own blindness and against those who will sell us the miraculous hope of snake-oil cure-alls.

Too often, product training is based on the false assumption that sales people and customer service reps simply need to be fed information about product features and benefits. Fortunately, Dan Bixby has come along and written a book to help design product training that actually works.

Here’s the testimonial I wrote for the book:

“Dan Bixby’s book will help technical experts create training that actually works to improve performance. Backed by years of experience, Bixby connects with practical advice and empathy—and helps experts avoid the most common mistakes in product training.”

You can check out Bixby’s book on Amazon at: https://amzn.to/2wBN110