Subscription Learning is the idea of providing short–usually less than 10 minute–nuggets of learning content or learning interaction. Often, we think of this as an information-delivery platform. In other words, we think of it as training that is spread over time in tiny packets of training content.

But we really ought to think beyond this limited “training” perspective! Subscription Learning can be so much more.

Just this week, I spoke with Marty Rosenheck of Cognitive Advisors, a cognitive-science inspired consultancy and learning-development shop. Cognitive Advisors has recently released TREK, a “Learning Experience Manager” that allows organizations to go beyond training and support people as they learn in the workplace.

From the Cognitive Advisors website, here’s what TREK does:

  • Manages, tracks and reports on the full range of learning
  • Is built from the ground-up with the Experience API (Tin Can) to capture learning outside the LMS
  • Offers cloud-based software designed for today’s mobile workforce
  • Aligns all learning experiences to competencies
  • Supports personalized learning paths to tailor learning for each learner

This technology wasn’t possible just a few years ago, and the future of this product category, which Marty has coined the “Learning Experience Manager,” seems exceedingly bright. The future looks promising because it fulfills a real need, providing support for on-the-job learning.

Indeed, one of the problems with the last decade’s flirtation with informal learning (and on-the-job learning) is that it happens willy nilly without adequate support. People can learn the wrong things, they can learn too slowly, they can get frustrated and give up, they can fail to learn. Just as learning researchers have found that discovery learning is not generally effective–unless it is “guided,” informal learning is often not as effective and efficient as it ought to be.


Real-world Example

Here’s a real-world example of how TREK has been used:

Professionals in the water-quality-assessment field have to get up to speed quickly to do their jobs. And, they have to do their jobs right. There’s no tolerance for poor water-quality testing. The problem is compounded because there’s so much to learn. New folks often feel they’re gazing through muddy waters until they’ve got lots of experience under their belts–and they have clarity about what to do when. They rely on experienced people to answer questions, provide guidance, and monitor their progress.

The down side of this is that it’s very expensive to keep sending experienced people out with the new folks, and its very inefficient and frustrating when inexperienced people have to keep calling their supervisors and mentors from the field.

Enter TREK. The Water Quality Association, working with Cognitive Advisors, piloted a program to provide their members with a structured on-the-job learning path, enabling learners to learn on the job–while being coached by their immediate supervisors.

TREK worked using employee’s smartphones’ sensors (camera, audio and video recorder, and GPS). Employees captured evidence of their critical actions at each step in their learning path. This evidence was submitted through TREK to each person’s designated manager-coach. As each step was completed, managers were notified and were prompted to review their direct reports’ submissions.

Managers provided brief feedback–either written or in a recorded audio nugget–and this feedback was presented to the learners. Managers weren’t left on their own to flounder in their coaching activities. They were provided with coaching guides, checklists, and success criteria within the TREK interface on their smartphones.

Interestingly, what TREK did in this case was to provide support to both learners and the learners’ managers. For both groups, this improved the effectiveness and efficiency of the learning/coaching experience.


Not Just a Technology

As Marty explained to me in discussing this new product category–the reason Learning Experience Management technology is possible now is that several forces have come together. First and foremost, a new database specification has been developed (Experience API, also known as Tin Can) that enables experiences to be collected and categorized–where once we could only capture training-related information. Second, mobile technology is now ubiquitous in cell phones. Third, cloud computing has become a norm, enabling continuous connectivity between learners and others. Finally, learning analytics, social media, and badging have added to the user experience.

But even given these technological advances, the thing that makes TREK and its performance-support coaching possible is that it aligns with human cognition. The key to providing a great learning experience is to ensure that the learning content is captured and presented in ways that can maximize learning.

Marty illustrated the key link in the process, as he described how he leads customers through a cognitive task analysis process he’s perfected over the years. He calls it knowledge harvesting. It reminded me very much of my days building simulations and working with SME’s to extract knowledge. It’s incredibly valuable, but you don’t want to skimp on the process.


Why is This Subscription Learning

TREK provides a subscription learning experience because by focusing on individual workplace tasks, it is providing small chunks of learning that is spaced over time.

The diagram Cognitive Advisors uses to explain their cognitive apprenticeship model highlights a number of small nuggets on the learning-path trajectory.

 

Folks to Watch

Marty Rosenheck and his business partner Colleen Enghauser are folks to watch–as is their company, Cognitive Advisors. Their dedication to creating learning technology aligned with the learning research is inspiring!

 

 

The ISPI Conference-Session Feedback Form was designed with the following ideas in mind:

Why do the questions ask what they do?

  1. ISPI is devoted to performance-improvement so the form should focus on factors known to support workplace performance improvement, including such factors as:
    • Comprehension of concepts taught.
    • Likelihood of achieving expected work results.
    • Reinforcement through practice–going beyond lecture.
    • Likelihood of sharing concepts with others.
    • Validity of the concepts taught as supported by research and evidence.
  2. Conference attendees have a reasonable expectation to receive sessions that they feel are valuable, as evidenced by the following:
    • Level of engagement they experienced.
    • Importance of the concepts taught.
  3. Open-ended questions often provide the best feedback.

 

Why are the questions designed the way they are?

  1. The form was designed to fit on one page, to make it useable in a conference context.
  2. The answer options attempt to provide — as much a possible in a small space — learners with options that provide enough granularity so they can make clear distinctions when deciding between options.
  3. Such answer-option specificity has the added advantage of providing clearer feedback to presenters and conference organizers about the success of the session.
  4. Answer options are specifically designed NOT to use numbers, nor scaled adjectives (e.g., “strongly agree,” “agree,” etc.) — as both approaches make it difficult for learners to make precise decisions and for stakeholders to understand the meaning of the results.

 

How was the form designed?

  1. Will Thalheimer was asked by ISPI leadership to design the form, based on his work on performance-focused smile sheets and his upcoming book. This web page sits on his blog.
  2. After Will’s initial input, feedback was solicited and improvements were made to the form. Special thanks to Bonnie Beresford for providing important feedback and partnering in piloting this form and another form.
  3. Both forms will be piloted in 2015 and additional improvements will be considered.

 

We’d love your feedback!

The form has been modified to enable feedback on the form directly. However, if you’re interested in making additional comments, please use the form below to provide feedback. Thank you!!!

 

The "Learning Benefits of Questions" is a research-to-practice report on how to use questions to boost learning results. First published back in 2003, and partially funded by Questionmark (to whom I am still grateful), the Learning Benefits of Questions was inspired by fundamental learning research, provided a practical perspective, and even provided a diagnostic to help readers determine how well they understood questions for learning.

Still getting requests and seeing people refer to this research-to-practice report, I've decided to reissue the report, with a few minor improvements. You can download the report using the following link:

Download Learning Benefits of Questions 2014 v2.0

You can also see our other papers, articles, and job aids at the Work-Learning Research catalog.

I just read a vendor blog post that lists the pros and cons of gamification.

PLEASE, let us be smarter than this!

Gamification is NOT a THING !!!!!!!!!!!!!!!!!!!!!!!

There can be NO pros and cons to gamification…

Gamification is a label for dozens of specific factors, each of which can be used or not used, or used alone or in concert with other gamification learning factors.

Here is a small list of gamification factors (just off the top of my head):

  1. competing against a standard
  2. competing against others
  3. being given some sort of non-tangible "award" for perseverance
  4. being given some sort of non-tangible "award" for some level of success
  5. being given some sort of non-tangible "award" randomly as you "play"
  6. working on a team
  7. escaping a threat
  8. working toward a specific goal
  9. performing with a time constraint
  10. et cetera (ad infinitum?)

[Hey, if anybody has published a list of gamification factors, let me know and I'll post it.]

Seriously, when we oversimplify, we not only show our ignorance of the magical complexity of human learning and cognition, we also hurt our own thinking and problem solving and those of every person with whom we are communicating.

Sure, some vendor wants to sell gamification. I get that. But what is really being said is: some vendor wants to sell gamification to the most vulnerable within our profession (newbies, etc.) and even to the less vulnerable and even to the best-and-brightest who may have a temporary brain freeze from such miscommunication.

No, no, no! Sorry! That's too cynical, right! Probably the vendor honestly thinks that their list of pros and cons is being helpful. Probably the vendor doesn't really understand that all of us must look more deeply than our industry's surface ripples.

How to end this blog post? Hmmm. This is difficult. I'm not sure. Okay, I got it.

Final advice:

  1. Evaluate the labels used in the learning industry.
  2. Seek the constituent factors.
  3. Get data on their causal effects.
  4. Hire learning experts from time to time to reality check your learning designs.
  5. Give yourself a gold star for reading this blog post to the end.
  6. You have reached WAWL Level 2, performing better than 92.4% of your colleagues!
  7. To get to WAWL Level 3, do a Google Search of Gamification, find a list of Gamification Factors, and send me the link.
  8. To get to WAWL Level 4, create your own list, reflect on what you discover, post it somewhere, and send me the link.
  9. May the forces of the Neo Elepha keep you safe on your journey. WHO-LA!

 

New research shows that while people tend to avoid saying hello to strangers, when they do say hello, they are happier for it. Read this nice article in the New York Times which describes the research.

I add this article to this blog on learning, because it reminded me that we in the learning field often put too much trust on our learners. The bottom line is that learners often don't know how best to learn.

This means that we shouldn't willy-nilly design our learning to enable our learners to do anything they want to. Sometimes this can be beneficial because it gives learners a sense of autonomy and it can help them get information they need–but more often than not, it enables learners to make bad decisions about their own learning.

The article also reminded me about research on creativity which shows that domain spanners–people who spend time interacting with others outside their main area of interest are often more creative because they draw from other spheres of thought. It's interesting that we are shy talking to strangers, even though doing so will not only make us happier, but it may enrich us ways we can't imagine.

To put it simply, subscription learning is a radical new paradigm in elearning. It enables us to think about elearning in new ways–in ways that challenge our old mental models of elearning.

Elearning is still a relatively young field, having its start in the 1960’s during the advent of the computer age and gradually gaining a critical mass after the internet became a mass phenomenon. Because it’s a young field, we are still learning how to think about elearning. With each new paradigm, we think more deeply, more fully about what elearning is–and can be. Below is my categorization of the most important elearning paradigms as of 2014.

eLearning Paradigms 2014

  • Content Presenter (enables content to be presented to learners)
  • Comprehension Tester (enables learners’ knowledge to be tested–and feedback provided)
  • Practice Provider (enables learners’ decision-making to be tested–and feedback provided)
  • Performance Supporter (enables performers to be prompted toward action)
  • Reminder (enables learners or performers to be reminded to learn and/or take action)
  • Social Augmentation Provider (enables learners to learn from and with each other)
  • Gamification Provider (provides motivational incentives and behavioral prompts to action)
  • Mobile Learning Provider (provides learning and/or performance support through mobile technology)
  • Data Utilizer (enables data collection and data-based interventions)
  • Video Provider (enables video to be utilized in various ways)
  • Learning Organizer (provides organizational structure around learning opportunities)
  • Personalizer (enables content or prompting to be individualized or tailored)
  • Learning-Delivery Augmenter (enables easy delivery of content or prompting)
  • Context-Based Triggerer (enables content or prompting to be delivered depending on context)
  • Cost Saver (enables learning to be delivered at a lower cost)

I’m sure that I’m missing some elearning paradigms. You might have noticed that I’m only listing elearning memes that have a positive connotation. I am not mentioning such things as boring, trivial, poorly-designed. Also, some of the list may not be true, or may not always be true. For example, I’ve recently read research that shows that elearning is not often a cost saver. The bottom line, however, is that the list above represents a good number of the ways in which we tend to think about elearning.

Here’s the thing: The paradigms listed above represent the dominant mental models we use when we think about elearning. As Thomas Kuhn wrote many years ago, paradigms are a double-edge sword. On the one hand, they help us think. On the other hand, they put boundaries on what we think. For us in the learning field, we get both benefits and costs from our elearning paradigms. They help us consider ways that we might design or utilize elearning. On the darker side, they constrict our thinking. One of the reasons we created the eLearningManifesto was to get the field to think beyond some of its weaker paradigms.

Our Thinking on eLearning is Still Evolving 500px

The Subscription-Learning Meme

I am pushing the idea of subscription learning not just because it is aligned with the learning research on the spacing effect, but also because it gives us a completely new way of thinking about elearning. It opens elearning to new possibilities.

Where we often think of elearning content delivery as requiring relatively long events of 30 minutes or more, subscription learning lets us think of much shorter events spaced over time. Where we often think of performance support as being delivered through a single-focus system at a time of known need, subscription learning can prompt a series of thoughts or actions even when learners don’t know they need to know. Where we think that learners have to seek their own learning nuggets, subscription learning can push learning to learners to better support learning as a process.

For folks going to the ISPI Conference in Indianapolis next week, here are my session titles.

For details, specific timing, and slides, click here.

 

SUNDAY
Speed Mentoring

TUESDAY
Special Session
The Serious eLearning Manifesto:
An Effort to Help eLearning Live Up to its Promise

WEDNESDAY MID MORNING
General Session (RTP)
The Decisive Dozen–Research-Supported Learning Factors

WEDNESDAY LATE MORNING THRU LUNCH
Pit Stop & Refuel: Roundtable & Lunch Discussion 
Insight Learning: Helping Employees Have Creative Insights

In the Decisive Dozen–the list of the twelve most important learning factors–Content Validity is Rule #1. If you're training your learners on bad content, you're doing more harm than good.

The TSA has apparently broken this rule to the tune of One Billion Dollars, spending money to train TSA agents to read body language–when there is no scientific evidence that people can actually detect liars.

This has to be one of the most costly training errors in the history of training and development! Congratulations TSA officials…

If you want to test your own skill at detecting liars, see this nice NY Times interactive.

I'm teaching the following half-day workshop at the upcoming ISPI conference.

This workshop goes for $150, which is an absolutely incredible bargain…

Click here to learn more about the workshop or the conference.

 

Radically New Research-Inspired Smile Sheet Design
Will Thalheimer, Work-Learning Research, Inc.
contact: will.thalheimer@work-learning.com
Workshop Code: WUQ

Research shows that traditional smile sheets are not related to learning results. Given this, we could just ignore smile sheets. However, because most organizations want to use smile sheets anyway, shouldn’t we try to make them better? In this workshop, Thalheimer will show how to create a “performance-focused smile sheet,” a radical improvement in smile sheet design–one that has been inspired by his years of research-based practice. The new design involves two meta-improvements. First, it is a smile sheet design that seeks information about key performance factors; and second, it is a smile sheet design that creates outputs that are more meaningful than data from scale responses. Performance-focused smile sheets will never be a one-source learning-feedback mechanism, but they provide a significant improvement over the traditional smile sheet.

Hello ISPI Attendees!!

Thanks for your interest!!

My public sessions are detailed below, and the slides are available as a download for my session on the Decisive Dozen. Other handouts will be conveniently delivered during the sessions.

If you’d like to set up a private meeting to discuss your organization’s issues while we’re at the conference, feel free to email me at: will.thalheimer@work-learning (dot) com or just grab me at the conference…

 

——————————————————-

SUNDAY
Speed Mentoring

Sunday April 13, 2 to 5PM

I will be one of many mentors available for quick mentoring sessions.
Looks like a great group of folks. Should be fun!!

 

 

——————————————————-

TUESDAY
Special Session
The Serious eLearning Manifesto:
An Effort to Help eLearning Live Up to its Promise

Tuesday April 15 – 10:00-11:15 am

Handouts Handed Out at Session

The eLearning Manifesto was recently released on March 13, 2014. An effort by a small group of elearning thought-leaders, the Manifesto aims to help elearning live up to its promise by challenging members of the elearning industry to take personal responsibility for building more effective elearning. The Manifesto is free and available at eLearningManifesto.org. It offers 22 principles as guidance for elearning developers, elearning buyers, and elearning advocates. Many of its principles come right out of the ISPI playbook. The Manifesto was designed to get a conversation going–and that is the aim of this session. Facilitated by one of the Manifesto authors, we will review the actual Manifesto, discuss its origins, debate its strengths and limitations, and consider how to make it practically useful.

Participants will be able to:

• Describe the rationale behind the need for improved elearning.
• Articulate the values inherent in the Serious eLearning Manifesto.
• Consider the Manifesto’s strengths and limitations.
• Plan for using the Manifesto to drive improved elearning

 

——————————————————-

WEDNESDAY
General Session (RTP)
The Decisive Dozen–Research-Supported Learning Factors

Wednesday April 16 – 10:00-11:15 am

Download Slides:  Download Thalheimer — Decisive Dozen — ISPI 2014

For decades, the workplace learning and performance field has been swamped with fads and misconceptions that harm learners and depress learning results. Research can provide salvation, but only if it is made accessible and easy to use. After 15 years of research, Dr. Will Thalheimer has developed a dozen learning factors that–if implemented–can improve learning results dramatically. These “decisive dozen” will be detailed in a forthcoming book and have been published in a research review.

Participants will be able to:

• Radically improve their learning designs and learning results.
• Conceptualize learning in terms of comprehension, remembering, and application.
• Access a research review of 12 key learning factors.

 

——————————————————-

Pit Stop & Refuel: Roundtable & Lunch Discussion 
Insight Learning: Helping Employees Have Creative Insights

Wednesday April 16 – 11:30 am-1:30 pm

Handouts Handed Out at Session

Most learning interventions target “transfer learning,” learning that is transferred from a person (or instructional program) to one or more learners. But people learn when they have insights as well.“Insight learning” can be targeted by workplace learning professionals to enable employees to have more and higher-quality creative insights.
 

——————————————————-

ALL Half-Day Workshops CANCELLED, Including Mine:

Preconference Workshop
Radically New Research-Inspired Smile Sheet Design
Workshop Code: WUQ

Sorry it was cancelled! If interested in this workshop delivered to your company or available to you online, contact me at: will.thalheimer@work-learning (dot) com