NPR's Morning Edition produced a five-minute radio piece on the U.S. Airforce Academy's attempt at improving learning results by modifying the ability-grouping of their cadets.

Shankar Vedantam

According to the piece, reported by Shankar Vedantam, based on research by Dartmouth researcher Bruce Sacerdote and colleagues:

  • Weaker students did better when in squadrons with stronger students (but note caveats below).
  • However, when researchers intentionally created squadrons with only the strongest and weakest students (that is, the middle students were removed), the weaker students did worse than they otherwise would have. The researchers argue that this was caused by the splintering of the squadron into groups of strong students and groups of weak students.
  • Middle students did better when they didn't have weaker and stronger students in their squadrons.
  • It appears that the middle students acted as a glue in the mixed-ability squadrons–and specifically, they helped the squadron to avoid splitting into groups.

Of course, one study should not be taken without some skepticism. Indeed, there is a long history of research on academic ability grouping. For example see the review article:

Schofield, J. W. (2010). International evidence on ability grouping with curriculum differentiation and the achievement gap in secondary schools. Teachers College Record, 112(5), 1492-1528.

As Schofield reports:

International research supports the conclusion that having high-ability/high-achieving schoolmates/classmates is associated with increased achievement. It also suggests that ability grouping with curriculum differentiation increases the achievement gap. For example, attending a high-tier school in a tiered system is linked with increased achievement, whereas attending a low-tier school is linked with decreased achievement, controlling for initial achievement. Furthermore, there is a stronger link between students’ social backgrounds and their achievement in educational systems with more curriculum differentiation and in those with earlier placement in differentiated educational programs as compared with others.

But she also warns:

However, numerous methodological issues remain in this research, which suggests both the need for caution in interpreting such relationships and the value of additional research on mechanisms that may account for such relationships.

In addition, social effects are probably not the only effects in play. For example, the research tells us that learners do better when they are presented with information and given instructional supports targeted specifically to their cognitive needs. So for example, this could be why the middle-ability students did better when they were grouped together.

Also interesting is that neither the NPR piece or Shofield's abstract reports specifically on how the mixed groupings affect the stronger learners.

Indeed, other researchers have advocated that gifted students should not be so ignored. See for example the following review article:

Subotnik, R. F., Olszewski-Kubilius, P., & Worrell, F. C. (2012). A proposed direction forward for gifted education based on psychological science. Gifted Child Quarterly, 56(4), 176-188.

Here's what these authors recommend:

In spite of concerns for the future of innovation in the United States, the education research and policy communities have been generally resistant to addressing academic giftedness in research, policy, and practice. The resistance is derived from the assumption that academically gifted children will be successful no matter what educational environment they are placed in, and because their families are believed to be more highly educated and hold above-average access to human capital wealth. These arguments run counter to psychological science indicating the need for all students to be challenged in their schoolwork and that effort and appropriate educational programing, training and support are required to develop a student’s talents and abilities.

In the Decisive Dozen–the list of the twelve most important learning factors–Content Validity is Rule #1. If you're training your learners on bad content, you're doing more harm than good.

The TSA has apparently broken this rule to the tune of One Billion Dollars, spending money to train TSA agents to read body language–when there is no scientific evidence that people can actually detect liars.

This has to be one of the most costly training errors in the history of training and development! Congratulations TSA officials…

If you want to test your own skill at detecting liars, see this nice NY Times interactive.

One of the worries expressed by people new to the idea of subscription learning is whether learners will actually stay subscribed to subscription-learning threads.

As evidenced in other blog posts here, getting people to pay attention doesn't have to be a problem, though it can be. Strategies that work include requiring learning engagement, creating relevant interactions, building interesting learning events, tracking progress, encouraging interpersonal competition, providing rewards, et cetera.

We know that people aren't hard-wired to avoid subscriptions. People have been subscribing to magazines and newspapers for more than a century. Charles Dickens had people subscribing to his books back in the 1800's. Steven King has done the same more recently.

Now we have more evidence. As reported in The Boston Globe, the modern serial novel is hot, hot, hot! Indeed, it says, "Wattpad is a leader in this new storytelling environment, with more than 2 million writers producing 100,000 pieces of material a day for 20 million readers on an intricate international social network."

If you're interested in subscription learning, Wattpad might be worth a look–not as a delivery device, but as a way to generate ideas for your subscription-learning efforts. Or, if you're lazy, you could just decide that storytelling is the key and incorporate elements of storytelling into your subscription-learning threads.

Here's an example of an author whose 110-part series has been read over 1.7 million times!



If you've seen anybody using story-telling for subscription learning, please let me know!


Micro Lectures, 1 to 3 minute recorded lectures, are offered as a way to support learning. See today's blog post from Jana Jan, who is a provider of a tool to enable micro lectures connected through "learning maps."

Unsupported with other instructional scaffolding, micro lectures will not be as effective as they could be.

I'm teaching the following half-day workshop at the upcoming ISPI conference.

This workshop goes for $150, which is an absolutely incredible bargain…

Click here to learn more about the workshop or the conference.


Radically New Research-Inspired Smile Sheet Design
Will Thalheimer, Work-Learning Research, Inc.
Workshop Code: WUQ

Research shows that traditional smile sheets are not related to learning results. Given this, we could just ignore smile sheets. However, because most organizations want to use smile sheets anyway, shouldn’t we try to make them better? In this workshop, Thalheimer will show how to create a “performance-focused smile sheet,” a radical improvement in smile sheet design–one that has been inspired by his years of research-based practice. The new design involves two meta-improvements. First, it is a smile sheet design that seeks information about key performance factors; and second, it is a smile sheet design that creates outputs that are more meaningful than data from scale responses. Performance-focused smile sheets will never be a one-source learning-feedback mechanism, but they provide a significant improvement over the traditional smile sheet.

Hello ISPI Attendees!!

Thanks for your interest!!

My public sessions are detailed below, and the slides are available as a download for my session on the Decisive Dozen. Other handouts will be conveniently delivered during the sessions.

If you’d like to set up a private meeting to discuss your organization’s issues while we’re at the conference, feel free to email me at: will.thalheimer@work-learning (dot) com or just grab me at the conference…



Speed Mentoring

Sunday April 13, 2 to 5PM

I will be one of many mentors available for quick mentoring sessions.
Looks like a great group of folks. Should be fun!!




Special Session
The Serious eLearning Manifesto:
An Effort to Help eLearning Live Up to its Promise

Tuesday April 15 – 10:00-11:15 am

Handouts Handed Out at Session

The eLearning Manifesto was recently released on March 13, 2014. An effort by a small group of elearning thought-leaders, the Manifesto aims to help elearning live up to its promise by challenging members of the elearning industry to take personal responsibility for building more effective elearning. The Manifesto is free and available at It offers 22 principles as guidance for elearning developers, elearning buyers, and elearning advocates. Many of its principles come right out of the ISPI playbook. The Manifesto was designed to get a conversation going–and that is the aim of this session. Facilitated by one of the Manifesto authors, we will review the actual Manifesto, discuss its origins, debate its strengths and limitations, and consider how to make it practically useful.

Participants will be able to:

• Describe the rationale behind the need for improved elearning.
• Articulate the values inherent in the Serious eLearning Manifesto.
• Consider the Manifesto’s strengths and limitations.
• Plan for using the Manifesto to drive improved elearning



General Session (RTP)
The Decisive Dozen–Research-Supported Learning Factors

Wednesday April 16 – 10:00-11:15 am

Download Slides:  Download Thalheimer — Decisive Dozen — ISPI 2014

For decades, the workplace learning and performance field has been swamped with fads and misconceptions that harm learners and depress learning results. Research can provide salvation, but only if it is made accessible and easy to use. After 15 years of research, Dr. Will Thalheimer has developed a dozen learning factors that–if implemented–can improve learning results dramatically. These “decisive dozen” will be detailed in a forthcoming book and have been published in a research review.

Participants will be able to:

• Radically improve their learning designs and learning results.
• Conceptualize learning in terms of comprehension, remembering, and application.
• Access a research review of 12 key learning factors.



Pit Stop & Refuel: Roundtable & Lunch Discussion 
Insight Learning: Helping Employees Have Creative Insights

Wednesday April 16 – 11:30 am-1:30 pm

Handouts Handed Out at Session

Most learning interventions target “transfer learning,” learning that is transferred from a person (or instructional program) to one or more learners. But people learn when they have insights as well.“Insight learning” can be targeted by workplace learning professionals to enable employees to have more and higher-quality creative insights.


ALL Half-Day Workshops CANCELLED, Including Mine:

Preconference Workshop
Radically New Research-Inspired Smile Sheet Design
Workshop Code: WUQ

Sorry it was cancelled! If interested in this workshop delivered to your company or available to you online, contact me at: will.thalheimer@work-learning (dot) com



If you're going to the eLearning Guild's Learning Solutions Conference this coming week in Orlando, come join me–and say hello!

I'll be speaking in three sessions:

Featured Session (F2)
Subscription Learning: A Fundamentally Different Form of eLearning

Time: Wednesday March 19, 10:45AM

Details on the session

Slides for the session

Over 300 people are expected to attend. Get there early for a good seat!

Concurrent Session (105)
Serious eLearning Manifesto (Also with Clark Quinn and Michael Allen)

Time: Wednesday March 19, 1:00PM

Details on the session

We will hand out paper version of the Manifesto at the session (there are no slides)

Morning Buzz (MB31)

Time: Thursday March 20, 7:15AM

A casual conversation about the eLearning Manifesto and Instructional Design

Note: Look for Clark Quinn, or Michael Allen's name (as mine is not listed), but I'll be there!


This week, on March 19th, Wednesday at 10:45 AM, Will Thalheimer (that's me) will be speaking on Subscription Learning at the eLearning Guild's Learning Solutions Conference.

300 people are expected to attend, so come early for your seat!!

Details: Learning Solutions Featured Session F2

Slides for the session

My friend, Jonathan Kaye, elearning-guy-extraordinaire, has posted the first parody of the Serious eLearning Manifesto–and I am proud to share it with you. Read his dLearning Manifesto principles (in this blog post) to get a LOL experience.



Yesterday March 13, 2014 (25 years and 1 day after the internet was born) the Serious eLearning Manifesto was released. As one of the "authors" of the Manifesto, I am overwhelmed by the outpouring of passion around the issue of elearning's unrequited promise. Below I will share some of the endorsement statements so that you can see first-hand the dedication of folks in the elearning field.

Manifesto Overview Graphic

The Manifesto

The Manifesto is an attempt to provide people and organizations in the elearning industry a lever to radically raise the effectiveness of elearning.

  • Helping elearning buyers (CLO's, training managers, CEO's, Deans, School Superintendents) demand better elearning, by pointing to the Manifesto's 22 principles as ideals to be achieved or worked towards.
  • Helping elearning designers and developers by providing design and deployment guidance.
  • Helping elearning vendors guide clients to better elearning designs.
  • Helping elearning shops find the leverage to get resourcing and support for truly effective elearning.
  • Helping graduate schools provide guidance on curriculum decisions.
  • Helping trade organizations develop credentials, provide workshops, and make programming decisions.


To read the manifesto or to become a signatory and endorse it:


To see the Manifesto release video:

Manifesto Release Video


First-Day Endorsements by Signatories


You can read these and more–and see who wrote them–by clicking here.

"I wholeheartedly agree with, and endorse this manifesto.  This is what is needed to turn the eLearning industry around and on it’s head, so that eLearning can be meaningful and appropriate to meet the needs of users and their educational requirements."

"This Manifesto finally puts what the eLearning professional strives for into a concise format that can be used as a daily commitment to quality."

"It is important that each of us, in our own organizations as well as our own personal commitment, strive to improve our industry and set higher standards. The Manifesto is a step towards this goal."

"What really speaks to me are the principles that have been outlined. There are many ideals here that I have tried to live up to, then there are those I want to live up to, and a few that I haven’t yet thought of to explore. In a time when budgets often are the first thing that matter in organizations, it’s more and more important to prove and show your worth and value to an organization.  Implementing these principles into each of the solutions that we create for our organizations will and can only solidify our true value."

"The work we do in helping people learn is ‘sacred’ work. If we just create content heavy, learning poor courses we fail in our responsibility."

"The manifesto is based on solid, empirical evidence that supports what we should be doing when we build our e-learning."

"It’s not about training – it’s about performance. I agree with the manifesto."

"People know what lawyers and accountants do and how to buy from them as providers of professional services. e-learning is a younger and far more misunderstood industry. The manifesto codifies what good practice looks like and I hope that we can build from this to giving learners great experiences that shift organisational performance."

"I commit to developing eLearning that falls in line with the Serious eLearning Manifesto. I also commit to encouraging others on my immediate team and in my organization to commit to these principles."

"I endorse the principles of the eLearning Manifesto and believe the importance of using these principles to ensure eLearning meets the amazing potential available for real learning to solve real problems."

"Bravo!  Finally an ‘agile manifesto’ for the eLearning community.  These principles will help all of us focus on performance rather than design ephemera…and stay relevant."

"These principles and guidelines are helpful to not only remind eLearning designers and developers of the important aspects of creating effective eLearning. It also helps us by providing support when we need to explain our new designs to clients."

"The eLearning Manifesto represents a step forward for the field of eLearning.  It accurately reflects the vision we, as eLearning developers,   need to adopt in order to move away from ineffective practice and towards the realization of eLearning’s full potential."

"Exquisitely concise and pragmatic! Implementing even a fraction of the ideas in the Manifesto will make a dramatic difference in the kind of eLearning coming into the world."

"I heartily endorse this manifesto. It supports the research that is available and the 22 guiding principles will definitely lead us as practitioners to develop higher quality learning events."

"I’ve been frustrated with bad eLearning for years.  If clients, stakeholders and subject matter experts agree to follow our lead and accept these principles we can finally improve performance.  Furthermore, my hope is that the authoring tools we use adopt these principles in the development of their software."

"I’m a TV producer new to the eLearning industry and every single point you’ve made is exactly what we’ve been saying to our clients.  It’s fantastic to see these values backed up by industry heavyweights who have done the background research to prove their points. The eLearning industry SO needs to be disrupted because so many content suppliers seem to have an attitude that says, "Hey, I know the courses we’re building are based on tech levels from the 70s but that’s what the buyers want.  And we’re making pretty good dough, so ix-nay."  Really? Thank you so much for calling out the complacency in the eLearning industry. Long live the Manifesto."

"The eLearning Manifesto provides a solid foundation on which all eLearning content should be created. If you’re wondering why your eLearning is not producing positive results – odds are you are not following the guiding principles in the Manifesto."

"I am a television and online video producer and new to the elearning world. My first task was to investigate current standards and what I found transported me twenty years into the past to the dawn of Power Point. The manifesto is clearly needed for learners, the end users, the audience, those that are here to be engaged by the content we design and create for them to improve their lives. Thank you for your dedication to a positive  elearning experience and creating a manifesto that puts elearning on a new path to a bright and exciting future."

"I vow to hold the Serious eLearning Manifesto as the new standard for all eLearning modules produced by our eLearning team. We are committed to interactive, real life scenarios and simulations for improved performance on the job.  Now, the Manifesto will help us make our design and development more powerful in our workplace!"

"It is tremendous that four great practitioners took time to formulate these principles and best practices! In addition, the list of Trustees is a "Who’s Who" of eLearning. This initiative will be a great boon to set standards of quality in our field!"

"The Serious eLearning Manifesto says in writing what we’ve all been whispering to our industry peers for years: eLearning is broken! We’ve had the awareness, knowledge, and skills to fix it for some time, but this coordinated effort and the Manifesto’s principles will boost our desire to act through shared accountability and best practice."

"I have longed to see some recognition of the sorry  state of current eLearning. Certainly there are shining examples of what it can be, but so many examples exist that illustrate how utterly woeful the norm has become. We can do better in meeting the promise that online technology presents us with. The Principles associated with the eLearning Manifesto reflect what we need to start implementing if eLearning is to reach it’s potential. Performance, not content; context, not generic; and consequences, not canned feedback are some of the principles we need to start incorporating as best we can within the constraints imposed in work and education. I wholeheartedly endorse this effort for the sake of this profession."