Posts

In 2016 I published a book on how to radically transform learner surveys into something useful. The book won an award from ISPI and helped thousands of companies update their smile sheets. Now, I’m updating the book with the knowledge I’ve gained in consulting with companies in the learning-evaluation efforts. The second edition will be titled: Performance-Focused Learner Surveys: A Radical Rethinking of a Dangerous Art Form (Second Edition).

In the first edition, I listed nine benefits of learner surveys, but I had only touched the surface. In the coming book, I will offer 20 benefits. Here’s the current list:

Supporting Learning Design Effectiveness

  1. Red-flagging training programs that are not sufficiently effective.
  2. Gathering ideas for ongoing updates and revisions of learning programs.
  3. Judging the strengths, weaknesses, and viability of program updates and pilots.
  4. Providing learning architects and trainers with feedback to aid their development.
  5. Judging the competence of learning architects and trainers.
  6. Judging the contributions to learning made by people outside of the learning team.
  7. Assessing the contributions of learning supports and organizational practices.

Supporting Learners in Learning and Application

  1. Helping learners reflect on and reinforce what they learned.
  2. Helping learners determine what (if anything) they plan to do with their learning.
  3. Nudging learners to greater learning and application efforts.

Nudging Action Through Stealth Messaging

  1. Guiding learning architects to create more effective learning by sharing survey questions before learning designs are finalized and sharing survey results after data is gathered.
  2. Guiding trainers to utilize more effective learning methods by sharing survey questions before learning designs are finalized and sharing survey results after data is gathered.
  3. Guiding organizational stakeholders to support learning efforts more effectively by sharing survey questions and survey results.
  4. Guiding organizational decision makers to better appreciate the complexity and depth of learning and development—helping the learning team to gain credibility and autonomy.

Supporting Relationships with Learners and Other Key Stakeholders

  1. Capturing learner satisfaction data to understand—and make decisions that relate to—the reputation of the learning intervention and/or the instructors.
  2. Upholding the spirit of common courtesy by giving learners a chance for feedback.
  3. Enabling learner frustrations to be vented—to limit damage from negative back-channel communications.

Maintaining Organizational Credibility

  1. Engaging in visibly credible efforts to assess learning effectiveness.
  2. Engaging in visibly credible efforts to utilize data to improve effectiveness.
  3. Reporting out data to demonstrate learning effectiveness.

If you want to learn when the new edition is available, sign up for my list. https://www.worklearning.com/sign-up/.

The second edition will include new and improved question wording, additional questions, additional chapters, etc.

Matt Richter and I, in our Truth-in-Learning Podcast, will be discussing learner surveys in our next episode. Matt doesn’t believe in smile sheets and I’m going to convince him of the amazing power of well-crafted learner surveys. This blog post is my first shot across the bow. To join us, subscribe to our podcast in your podcast app.

CrowdThinking Project on L&D Professionalization

At the L&D Conference 2020, which starts in a few days (seats still available), we are hosting the CrowdThinking Project, a two-pronged crowdsourced exploration designed to create a future-vision for the L&D field.

You and I, as learning professionals, are effective–but certainly, if there were more and better structural supports within the industry, within our organizations, within ourselves; you and I might be even more effective in our work.


First Part

The first part of the CrowdThinking Project is a survey of people like you to gather data along seven factors that influence our effectiveness and professionalization.

  1. The competencies, skills, and abilities we have as professionals
  2. The requirements and training/education needed to enter the L&D field
  3. The feedback we get on our effectiveness (learning evaluation)
  4. The support we get from our trade organizations
  5. The support/guidance we get from our graduate programs and universities
  6. The support and constraints from our business/organizational stakeholders
  7. The effort, direction, and perseverance we lend to our own development.

I developed this survey with the help from Fernando Senior.

This survey is open to all L&D professionals. I ask you to share it widely with your colleagues and friends in the L&D field.

 

Second Part

The second part of the CrowdThinking Project will take place within the L&D Conference (sorry, only if you’ve enrolled). Fernando Senior will take us through a modified world-cafe-style dialogue, focusing on four key questions.

  1. Consider your current circumstances in your L&D work situation—and more importantly, how those circumstances will change as a result of future trends in learning, technology, business, and society. Given the future you imagine, what will be the most important challenges to your work in L&D?
  2. What capabilities will L&D professionals like us need to acquire in anticipation of these upcoming challenges—to maximize our level of professionalization and our effectiveness?
  3. Whether today or in the future, how can we L&D professionals evidence and document our level of professionalization or maturity—in ways that will be understood and respected, and in ways that will add to our effectiveness.
  4. What other factors—besides our knowledge, skills, and attitudes—influence our ability to maximize our effectiveness? And, how will we be able to utilize these factors in the future to support our effectiveness?

 

Third Part

We will generate a report or reports on the findings of the survey and the discussions with recommendations for how the L&D field can continue to maintain and develop professionalization standards and practices.


How You Can Help

The most important thing I’d ask you to do right now, if you are in the workplace learning field is:

  1. Complete the survey (it’s not short. It takes 30 minutes)
  2. Ask others you know in L&D if they would consider it.


Joining the Conference

The L&D Conference 2020 runs over six weeks, it’s going to be truly amazing, and it starts in a few days (June 22 to July 31). Here’s the conference website: https://www.learningdevelopmentconference.com/

This is my conference. I’m the co-host along with my podcast partner, Matt Richter.

I know it’s last minute, so if you have trouble getting the funding figured out from your organization and want to get started, feel free to contact me to see if I can help.

Use this contact page to email me: https://www.worklearning.com/contact/

Today is May 15, 2020.

This week, I asked my publisher to take all 254 remaining copies of Performance-Focused Smile Sheets and throw them in the recycling bin!

Yes! The book was one of the most important books written in the learning-and-development field over the last decade.

But I threw it out!

Why?

Because I’m writing a new, completely revised edition based on my experience developing learner surveys for organizations around the world. My learner-survey approach has been used at The Centers for Disease Control and Prevention, the World Bank, at Bloomberg, at Roche (Genentech), at Oxfam. It has revolutionized the moribund smile sheet!

But time doesn’t stand still, and I’ve gained a ton of insights building learner surveys over the past five years, hearing from clients and others, pilot testing learner surveys, and creating smile sheets for my own workshops.

The new book will have an all new set of recommended questions. It will have information on open-ended questions. It will describe how learner surveys fit into the new LTEM framework (the learning-evaluation model that is replacing the Kirkpatrick-Katzell Four-Level Model around the world). The new book will also share the thinking behind the new learning-evaluation approach, LEADS (Learning Evaluation As Decision Support), which I’m developing.

When will the new book be out? Within three months!

The book may still be available in some pipelines, but I’m working to get it off the shelves. My advice is don’t buy it!

Wait for the second edition, which I’ve re-titled, Performance-Focused Learner Surveys.

Stay tuned!

And sign up for my newsletter at https://www.worklearning.com/sign-up/ to be the first to know when the book is available.

Thanks for your patience,

= Will Thalheimer

The L&D Conference—a quixotic attempt to reinvent conferencing. Aligning conference activities with how we humans actually learn. It’s an online conference, taking place over six weeks. It enables attendees to learn in the flow of their work. So they can use what they’re learning, share with their teammates back in the office, try things out, get advice, stay motivated and inspired to apply their new learning. The conference is the brainchild of Matt Richter and I, and we are now joined by marketing guru Ashley Sinclair. We are so excited to share this with you!

Unfortunately, when people think of online conferences, they think mediocre webinars strung together over one or two or a few days. THIS AIN’T THAT! We would NEVER design a conference that ignored the fundamentals of human learning!

We think you’ll like:

  • our global conference
  • our world-class speakers
  • our research-inspired sessions
  • our rollicking debates
  • our panels
  • our discount house
  • our sponsorship opportunities

BUT IF YOU NEED MORE CONVINCING—or you think YOUR BOSS might need convincing, we can help.

Download this gorgeous document. It makes the business case for joining us in the first L&D Conference.

And actually, the document does more than that. It persuades, in glorious detail, why this will be the best, most effective, and boldest L&D conference ever.

 

Click this Sentence to See the Detailed Document

 

Click this Sentence to See the Less Detailed Webpage

 

Matt Richter, my podcast brother, and presenter extraordinaire has agreed to share with me and my Presentation Science audience at short 35-minute session entitled:

ZOOM FOR PRESENTATION SCIENCE: PRACTICAL TIPS AND SURPRISES

 

 

I’m making this session available to everyone.

Day and Time

Monday April 6th (2020)

11AM New York Time

 

Zoom Coordinates

Join Zoom Meeting
https://zoom.us/j/197015730

Meeting ID: 197 015 730

One tap mobile
+13126266799,,197015730# US (Chicago)
+16465588656,,197015730# US (New York)

Dial by your location
+1 312 626 6799 US (Chicago)
+1 646 558 8656 US (New York)
+1 346 248 7799 US (Houston)
+1 669 900 6833 US (San Jose)
+1 253 215 8782 US
+1 301 715 8592 US
+54 112 040 0447 Argentina
+54 341 512 2188 Argentina
+54 343 414 5986 Argentina
+43 670 309 0165 Austria
+43 72 011 5988 Austria
+43 12 535 501 Austria
+43 12 535 502 Austria
+973 1619 7601 Bahrain
+973 1619 8488 Bahrain
+32 1579 5132 Belgium
+32 2 290 9360 Belgium
+32 2 588 4188 Belgium
+32 2 788 0172 Belgium
+32 2 788 0173 Belgium
+55 11 4680 6788 Brazil
+55 21 3958 7888 Brazil
+359 2 492 5688 Bulgaria
+359 3 257 1633 Bulgaria
+1 438 809 7799 Canada
+1 587 328 1099 Canada
+1 647 374 4685 Canada
+1 647 558 0588 Canada
+1 778 907 2071 Canada
+56 23 210 9066 Chile
+56 41 256 0288 Chile
+57 1 508 7702 Colombia
+57 2 620 7388 Colombia
+385 1300 0988 Croatia
+385 1777 6333 Croatia
+357 2 200 0888 Cyprus
+357 2 505 4777 Cyprus
+420 2 2888 2388 Czechia
+420 5 3889 0161 Czechia
+45 32 70 12 06 Denmark
+45 32 71 31 57 Denmark
+45 32 72 80 10 Denmark
+45 32 72 80 11 Denmark
+45 89 88 37 88 Denmark
+1 829 947 9220 Dominican Republic
+1 829 956 2188 Dominican Republic
+503 2113 9088 El Salvador
+503 2136 6444 El Salvador
+372 660 1699 Estonia
+372 880 1188 Estonia
+358 3 4109 2129 Finland
+358 9 4245 1488 Finland
+33 1 7037 2246 France
+33 1 7037 9729 France
+33 1 7095 0103 France
+33 1 7095 0350 France
+33 7 5678 4048 France
+995 3224 73988 Georgia
+995 7067 77954 Georgia
+49 30 5679 5800 Germany
+49 695 050 2596 Germany
+49 69 7104 9922 Germany
+30 211 198 4488 Greece
+30 231 118 0599 Greece
+36 1 408 8456 Hungary
+36 1 701 0488 Hungary
+353 1 536 9320 Ireland
+353 1 653 3895 Ireland
+353 1 653 3897 Ireland
+353 1 653 3898 Ireland
+353 6 163 9031 Ireland
+972 3 978 6688 Israel
+972 55 330 1762 Israel
+39 020 066 7245 Italy
+39 021 241 28 823 Italy
+39 069 480 6488 Italy
+371 6303 1808 Latvia
+371 6303 1888 Latvia
+370 3799 9260 Lithuania
+370 5214 1488 Lithuania
+352 2786 1188 Luxembourg
+352 2786 4277 Luxembourg
+356 2776 1777 Malta
+356 2778 1288 Malta
+52 229 910 0061 Mexico
+52 554 161 4288 Mexico
+31 20 241 0288 Netherlands
+31 20 794 0854 Netherlands
+31 20 794 6519 Netherlands
+31 20 794 6520 Netherlands
+31 20 794 7345 Netherlands
+47 2396 0588 Norway
+47 7349 4877 Norway
+507 833 9588 Panama
+51 1 707 5788 Peru
+51 1 730 6777 Peru
+48 22 307 3488 Poland
+48 22 398 7356 Poland
+351 308 804 188 Portugal
+351 308 810 988 Portugal
+351 211 202 618 Portugal
+351 265 120 012 Portugal
+1 787 945 1488 Puerto Rico
+1 787 966 7727 Puerto Rico
+40 31 630 1088 Romania
+40 37 170 0418 Romania
+7 495 283 9788 Russian Federation
+7 812 426 8988 Russian Federation
+421 233 056 888 Slovakia
+421 233 418 515 Slovakia
+386 1600 3102 Slovenia
+386 1888 8788 Slovenia
+27 87 550 3946 South Africa
+27 87 551 7702 South Africa
+34 84 368 5025 Spain
+34 91 787 0058 Spain
+34 917 873 431 Spain
+46 8 4468 2488 Sweden
+46 8 5050 0828 Sweden
+46 8 5050 0829 Sweden
+46 8 5052 0017 Sweden
+46 850 539 728 Sweden
+41 22 591 00 05 Switzerland
+41 22 591 01 56 Switzerland
+41 31 528 09 88 Switzerland
+41 43 210 70 42 Switzerland
+41 43 210 71 08 Switzerland
+90 216 900 1866 Turkey
+90 216 900 2606 Turkey
+44 131 460 1196 United Kingdom
+44 203 481 5237 United Kingdom
+44 203 481 5240 United Kingdom
+44 208 080 6591 United Kingdom
+44 208 080 6592 United Kingdom
+44 330 088 5830 United Kingdom

Meeting ID: 197 015 730
Find your local number: https://zoom.us/u/adcRKWndGM

 

The LEARNNOVATORS team (specifically Santhosh Kumar) asked if I would join them in their Crystal Balling with Learnnovators interview series, and I accepted! They have some really great people on the series, I recommend that you check it out!

The most impressive thing was that they must have studied my whole career history and read my publication list and watched my videos because they came up with a whole set of very pertinent and important questions. I was BLOWN AWAY—completely IMPRESSED! And, given their dedication, I spent a ton of time preparing and answering their questions.

It’s a two part series and here are the links:

Here are some of the quotes they pulled out and/or I’d like to highlight:

Learning is one of the most wondrous, complex, and important areas of human functioning.

The explosion of different learning technologies beyond authoring tools and LMSs is likely to create a wave of innovations in learning.

Data can be good, but also very very bad.

Learning Analytics is poised to cause problems as well. People are measuring all the wrong things. They are measuring what is easy to measure in learning, but not what is important.

We will be bamboozled by vendors who say they are using AI, but are not, or who are using just 1% AI and claiming that their product is AI-based.

Our senior managers don’t understand learning, they think it is easy, so they don’t support L&D like they should.

Because our L&D leaders live in a world where they are not understood, they do stupid stuff like pretending to align learning with business terminology and business-school vibes—forgetting to align first with learning.

We lie to our senior leaders when we show them our learning data—our smile sheets and our attendance data. We then manage toward these superstitious targets, causing a gross loss of effectiveness.

Learning is hard and learning that is focused on work is even harder because our learners have other priorities—so we shouldn’t beat ourselves up too much.

We know from the science of human cognition that when people encounter visual stimuli, their eyes move rapidly from one object to another and back again trying to comprehend what they see. I call this the “eye-path phenomenon.” So, because of this inherent human tendency, we as presenters—as learning designers too!—have to design our presentation slides to align with these eye-path movements.

Organizations now—and even more so in the near future—will use many tools in a Learning-Technology Stack. These will include (1) platforms that offer asynchronous cloud-based learning environments that enable and encourage better learning designs, (2) tools that enable realistic practice in decision-making, (3) tools that reinforce and remind learners, (4) spaced-learning tools, (5) habit-support tools, (6) insight-learning tools (those that enable creative ideation and innovation), et cetera

Learnnovators asked me what I hoped for the learning and development field. Here’s what I said:

Nobody is good at predicting the future, so I will share the vision I hope for. I hope we in learning and development continue to be passionate about helping other people learn and perform at their best. I hope we recognize that we have a responsibility not just to our organizations, but beyond business results to our learners, their coworkers/families/friends, to the community, society, and the environs. I hope we become brilliantly professionalized, having rigorous standards, a well-researched body of knowledge, higher salaries, and career paths beyond L&D. I hope we measure better, using our results to improve what we do. I hope we, more-and-more, take a small-S scientific approach to our practices, doing more A-B testing, compiling a database of meaningful results, building virtuous cycles of continuous improvement. I hope we develop better tools to make building better learning—and better performance—easier and more effective. And I hope we continue to feel good about our contributions to learning. Learning is at the heart of our humanity!

Given the challenges TEACHERS and PROFESSORS are facing with the Coronavirus Pandemic I’ve decided to make the Presentation Science Online Workshop available to Teachers and Professors for FREE (now through April 30th).

The workshop provides a strong science-of-learning foundation that will help educators make informed decisions as they move their courses online, create video recordings, or use any free time to update their classroom learning designs.

PLEASE share this with educators you know.

https://academy.worklearning.com/library/presentation-science/90041/about/

 

About the Presentation Science Workshop

Presentation Science is an online self-paced workshop designed specifically to help people who are trainers, teachers, professors, speakers, CEOs, Executive Directors, managers, military leaders, salespeople, team leads–anybody who uses presentation software–to help their audiences learn.

Inspired by learning science, this workshop will help speakers and educators to (1) keep their audiences’ attention, (2) support comprehension, (3) motivate audience members to take action, and (4) support them in remembering what’s been taught.

The workshop is also an excellent TRAIN-THE-TRAINER experience and organizations wanting to engage in a private cohort can make arrangements with Will Thalheimer (workshop creator and host) to do that. You can see the specific pricing options here: https://www.presentationscience.net/pricing.

And for more information about the workshop, see PresentationScience.Net.

Mirjam Neelen and Paul Kirschner have written a truly beautiful book—one that everyone in the workplace learning field should read, study, and keep close at hand. It’s a book of transformational value because it teaches us how to think about our jobs as practitioners in utilizing research-informed ideas to build maximally effective learning architectures.

Their book is titled, Evidence-Informed Learning Design: Use Evidence to Create Training Which Improves Performance. The book warns us of learning myths and misconceptions—but it goes deeper to bring us insights in how these myths arise and how we can disarm them in our work.

Here’s a picture of me and my copy! The book officially goes on sale today in the United States.

 

Click to get your copy of the book from Amazon (US).

The book covers the most powerful research-informed learning factors known by science. Those who follow my work will hear familiar terms like Feedback, Retrieval Practice, Spacing; but also, terms like double-barreled learning, direct instruction, nuanced design, and more. I will keep this book handy in my own work as a research-inspired consultant, author, provocateur—but this book is not designed for people like me. Evidence-Informed Learning Design is perfect for everyone with more than a year of experience in the workplace learning field.

The book so rightly laments that “the learning field is cracked at its foundation.” It implores us to open our eyes to what works and what doesn’t, and fundamentally to rethink how we as practitioners work in our teams to bring about effective learning.

The book intrigues as can be seen in sections like, “Why myths are like zombies,” and “No knowledge, no nothing,” and “Pigeonholing galore.”

One of my favorite parts of the book is the interviews of researchers that delve into the practical ramifications of their work. There are interviews with an AI expert, a neuroscientist, and an expert on complex learning, among others. These interviews will wake up more than a few of us.

What makes this book so powerful is that it combines the work of a practitioner and a researcher. Mirjam is one of our field’s most dedicated practitioners in bringing research inspirations to bear on learning practice. Paul is one of the great academic researchers in doing usable research and bringing that research to bear on educational practice. Together, for many years, they’ve published one of the most important blogs in the workplace learning field, the Three-Star Learning blog (https://3starlearningexperiences.wordpress.com/).

Here are some things you will learn in the book:

Big Picture Concepts:

  • What learning myths to avoid.
  • What learning factors to focus on in your learning designs.
  • How to evaluate research claims.

Specific Concepts:

  • Whether Google searches can supplant training.
  • What neuroscience says about learning, if anything.
  • How to train for complex skills.
  • How AI might help learning, now and in the future.
  • Types of research to be highly skeptical of.
  • Whether you need to read scientific research yourself.
  • Whether you should use learning objectives, or not, or when.
  • Whether learning should be fun.
  • The telltale signs of bad research.

This book is so good that it should be required reading for everyone graduating at the university level in learning-and-development.

 

 

Click on the book image to see it on Amazon (US).

 

I’m thrilled and delighted to share the news that Jane Bozarth, research-to-practice advocate, author of Show Your Work, and Director of Research for the eLearning Guild, is pledging $1,000 to the Learning Styles Challenge!!

 

 

Jane has been a vigorous debunker of the Learning-Styles Myth for many, many years! For those of you who don’t know, the Learning-Styles Notion is the idea that different people have different styles of learning and that by designing our learning programs to meet each style—that is, to actually provide different learning content or activities to different learners—learning will be improved. Sounds great, but unfortunately, dozens and dozens of research studies and many major research reviews have found the Learning-Styles Notion to be untrue!

 

“Decades of research suggest that learning styles, or the belief that people learn better when they receive instruction in their dominant way of learning, may be one of the most pervasive myths about cognition.”

Nancekivell, S. E., Shah, P., & Gelman, S. A. (2020).
Maybe they’re born with it, or maybe it’s experience:
Toward a deeper understanding of the learning style myth.
Journal of Educational Psychology, 112(2), 221–235.

 

 

“Several reviews that span decades have evaluated the literature on learning styles (e.g., Arter & Jenkins, 1979; Kampwirth & Bates, 1980; Kavale & Forness, 1987; Kavale, Hirshoren, & Forness, 1998; Pashler et al., 2009; Snider, 1992; Stahl, 1999; Tarver & Dawson, 1978), and each has drawn the conclusion that there is no viable evidence to support the theory.”

Willingham, D. T., Hughes, E. M., & Dobolyi, D. G. (2015).
The scientific status of learning styles theories.
Teaching of Psychology, 42(3), 266-271.

 

With Jane’s contribution, the Learning Styles Challenge is up to $6,000! That is, if someone can demonstrate a beneficial effect from using learning styles to design learning, the underwriters will pay that person or group $6,000.

The Learning Styles Challenge began on August 4th 2006 when I offered $1,000 for the first challenge. In 2014, it expanded to $5,000 when additional pledges were made by Guy Wallace, Sivasailam “Thiagi” Thiagarajan, Bob Carleton, and Bob’s company, Vector Group.

Thank you to Jane Bozarth for her generous contribution to the cause! And check out her excellent research review of the learning-styles literature. Jane’s report is filled with tons of research, but also many very practical recommendations for learning professionals.

In my online-anytime workshop, Presentation Science, I make over one hundred recommendations for giving more effective presentations, based on the science of learning. You can learn more about the workshop by clicking here.

Below is Tip 4 in my workshop marketing effort. Please share with others if you think they’ll find it useful. This Tip 4 video is a bit longer than Tips 1, 2, and 3, because it takes a bit more time to explain. It’s still only four and half minutes, but this content is really critical.

Bullet points bore and cause pain for our audiences. We need to get rid of them. In the video I share one of the most powerful ways to do that!


Tip 4 — Disguising Our Bullet Points