This is my preface to Clark Quinn’s book on debunking the myths in the learning field, Millennials, Goldfish & Other Training Misconceptions: Debunking Learning Myths and Superstitions. (available from Amazon here).

Clark Stanley worked as cowboy and later as a very successful entrepreneur, selling medicine in the United States that he made based on secrets he learned from an Arizona Hopi Indian medicine man. His elixir was made from rattlesnake oil, and was marketed in the 1890’s through public events in which Stanley killed live rattlesnakes and squeezed out their oil in front of admiring crowds. After his medicine gained a wide popularity, Stanley was able to set up production facilities in Massachusetts and Rhode Island with the help of a pharmacist. Stanley made himself a rich man.

You may not know his name, but you’ve certainly heard of his time and place. It was the era of patent medicines—false and sometimes dangerous elixirs sold to men and women of all stripes. Dr. Kilmer’s Swamp Root. Oxien. Kickapoo Indian Sagwa. Dr. Morse’s Indian Root Pills. Enzyte. Bonnore’s Electro Magnetic Bathing Fluid. Radithor. Liquozone. And of course, Clark Stanley’s Snake Oil Liniment.

These medicines were bought by the millions. Fortunes were made. Millions of people were bamboozled, made sick, killed or murdered depending on how you see it. It turns out that, upon being tested, Stanley’s elixir was found to be made mostly from mineral oil, a worthless potion sold by a charlatan. His story of the medicine man and the rattlesnake juice was a more potent concoction than his famous elixir, which when tested was found to have no snake oil anyway.

What causes men and women to miss the truth, to fail to see, to continue happily in harming themselves and those around them? This, unfortunately, is not a question just for the era of patent medicines. It is eternal. It goes back to the dawn of humanity and continues today as well. I have no answer except to assume that our credulity is part of our humanity—and should guide us to be on guard at all times.

What stopped the patent-medicine pandemic of poison, persuasion, and placebo? Did we the people rise up on our own and throw out the scoundrels, the money-grubbers, the snake-oil salesmen? Did we see that we were deceived, or too hopeful, or too blind? Did we as a community heed our senses and find a way to overcome the dangers hidden from us?

No! We did not!

It was not a mass movement back to rationality and truth that saved us. It was the work of a few intrepid agitators who made all the difference. Journalists began reporting on deaths, sicknesses, and addictions resulting from the use of patent medicines. In 1905, Collier’s Weekly published a cover story that exploded the industry. Written by Samuel Hopkins Adams a former crime reporter, with the title, “The Great American Fraud: The Patent Medicine Evil,” the long expose contained sections with headings like, “Medicine or Liquor?”, “The Men Who Back the Fake,” “Absolutely False Claims,” “Drugs that Deprave,” “Prescribing Without Authority,” and “Where the Money Goes.”

The article—or series of articles that today we would call investigative journalism—opened the floodgates and led directly to the Pure Food and Drug Act in 1906, followed later by additional regulations and requirements that continue to this day, with some success, protecting our health and safety.

The ugly truth is that we need help in seeing what we don’t see. This is true too in the learning industry and has been true since at least the early 1900’s when thought leaders in our industry floated bogus claims that people remember 10% of what they read, 20% of what they hear, 30% of what they see, et cetera. Indeed, it was partly the bogus claims floating around the learning industry in the late 1990’s that made me optimistic that starting a research-based consulting practice would find an audience, that perhaps the learning field could be protected from snake oil charlatans.

Bogus claims are not merely inert flotsam to be navigated around. At a minimum, they take attention away from learning practices that are more fundamental and effective, pushing us to waste time and resources. More insidious is that they proactively cause harm, hurting learners and weakening our learning outcomes.

I wish I could report that starting Work-Learning Research twenty years ago has had the influence that Samuel Hopkins Adams had in his journalism. Alas, I am a faint voice in the howling wind of our industry. Fortunately, there are many muckraking research-to-practice practitioners today, including folks like Paul Kirschner, Patti Shank, Guy Wallace, Pedro De Bruyckere, Julie Dirksen, Donald Clark, Ruth Clark, Mirjam Neelen, Jane Bozarth, and more. There are also legions of academic researchers who do the science necessary to enable research-to-practice wisdom to be compiled and conveyed to trainers, instructional designers, elearning developers and learning executives.

I am especially optimistic now that Clark Quinn has compiled, for the first time, the myths, misconceptions, and confusions that imbue the workplace learning field with faulty decision making and ineffective learning practices. As Clark rightly advises, don’t read the book in one sitting. You will find it too much—too many misconceptions and malingering falsehoods, and too much heartache to think that our field could tolerate so much snake oil.

Here’s what we don’t realize. Today’s workplace-learning snake oil is costing us billions of dollars in wasted effort, misspent resources, ill-advised decisions, and distraction from the science-of-learning fundamentals that have proven to be effective! Every time a trainer reads an article on learning styles and adjusts his or her training to make it suitable for visual, auditory, kinesthetic, and olfactory learners; time is wasted, money is spent, and learning is hurt. Every time an instructional designer goes to a conference and hears that neuroscience should guide learning design, he or she takes this faulty meme back to colleagues and infects them with false hope and ineffective learning strategies. Every time a Chief Learning Officer hears that learning events should be shrunk to 4-minute microlearning videos, that storytelling is everything, that all learning is social, that virtual reality is the future of learning—every time our learning executives jump on a bandwagon and send us to training or conferences or hire experts in these multitudinous fascinations—we are diverted from the veritable essence of learning. We waste our own developmental budgets with snake-oil rostrums. We waste time organizing ourselves around wrong-headed initiatives. We ignore what really works, all the while costing our organizations billions of dollars in waste and ineffective learning practices.

Let us start anew today. We can begin with Clark’s book. It is a veritable treasure chest of wisdom. But let’s keep going. Let’s stay skeptical. Let’s look to the scientific research for knowledge. Let’s become more demanding and knowledgeable ourselves, knowing that we all have more to learn. Let’s look to the research translators who know the work that we do as instructional designers, trainers, and developers. Let’s do our own testing. Let’s improve our evaluation systems so that we get better feedback day by day. Let’s pilot, rework, improve, and continue to learn!

As the history of patent medicine shows, we must be forever vigilant against our own blindness and against those who will sell us the miraculous hope of snake-oil cure-alls.

Too often, product training is based on the false assumption that sales people and customer service reps simply need to be fed information about product features and benefits. Fortunately, Dan Bixby has come along and written a book to help design product training that actually works.

Here’s the testimonial I wrote for the book:

“Dan Bixby’s book will help technical experts create training that actually works to improve performance. Backed by years of experience, Bixby connects with practical advice and empathy—and helps experts avoid the most common mistakes in product training.”

You can check out Bixby’s book on Amazon at:

Connie Malamed is The eLearning Coach, an intriguing podcaster, and the author of two fantastic books on visual design. Here I interview her in regards to her most recent book, Visual Design Solutions.


Here is the book:

Here is Connie:


Connie, in your book, Visual Design Solutions: Principles and Creative Inspiration for Learning Professionals, your goal seems to be to help learning professionals utilize effective visuals to improve their learning outcomes. Indeed, you dedicate the book to “the hard-working creative learning professionals who want to make a difference.”

Tell me about your hope for the book and the importance visual design has for learning professionals?


My three goals in writing this book were to: 1) prove that it is possible to improve one’s visual design skills without being an artist, 2) demonstrate the benefits of using visuals to enhance and amplify learning, and 3) raise awareness about the importance of aesthetics in a learner’s experience.

For those with normal vision, the brain processes more sensory information from the eyes than from any other sense. So learning professionals should expect that the visual aspect of instructional materials would be of great importance to comprehension, retention and the user experience. The good news is that anyone can become more competent in visual design by learning, applying and practicing the foundation principles.



Many learning professionals enter the field with little or no background or experience in graphic arts, visualization principles, or aesthetics—and yet you declare in your book that “you do not need drawing talent to work as a visual designer.” First, let me ask you, “Why not?” Second, let me ask you what key skills people do need to be effective at the visual aspects of learning design?


Early in my career, I met an excellent designer who didn’t know how to draw. He told me he wasn’t an illustrator. I was shocked. Since then, I’ve met and read about many designers who do not illustrate. Visual design involves the arrangement of images and text in graphic space. To be able to do this, one doesn’t need to render with a pen or pencil. Of course, it’s always nice to brainstorm ideas with a pencil and sketchpad, but visual concepts can be communicated using geometric shapes and stick figures.

The skills that I think people need for visual design competence can be learned. Here is my list:

  • An understanding of how to think about and solve visual problems.
  • Foundation principles of design, such as use of white space, establishing a visual hierarchy and appreciating typography.
  • Awareness of design in the world around you to see how others have solved visual problems.



I noticed in your book that you begin with lots of research supporting the benefits of using visuals. But certainly visuals can also be used in such a way that causes harm. What are some of the problems involved in using visuals? What are some of the most common mistakes learning professionals make?


Right. Like anything else, it takes thoughtfulness to come up with an effective visual design. I think common problems are: cluttering a layout with too many flourishes, using irrelevant graphics that are distracting, and splitting attention so that the visuals and text or activity are not well integrated. One way to avoid common mistakes is to get your work critiqued by peers, sponsors and potential users.



In the book, you emphasize white space, and yet I’ll bet this is one of the hardest things for us instructional designers to get. Just as we tend to cram content into our curriculums and don’t leave enough time for learning, I’ve often seen designs that cram content into our visuals without thinking about white space. What is white space and what are the top three things learning designers should realize about it?


White space is also known as negative space. It’s the area in a visual that does not contain any images or text. It even includes the area in between letters. Here are three tips about working with white space.

  1. Think of white space as another element, just like text and images. These three building blocks of design have to work together to create a clear communication. Without white space, you can’t have form and without form, you can’t have white space.
  2. As you design, if you begin to focus on the shape of the white space, you begin to bring it into the foreground perceptually. Become conscious of the white space and make sure the shape is pleasing and that it’s not broken up into tiny little pieces.
  3. White space gives a viewer’s eyes a place to rest and allows a design to breathe. So don’t be stingy with your white space. Let your designs have some spaciousness.





I put a ton of time into creating PPT slides for my workshops and presentations—and I’ve developed some beliefs over the years that may or may not be true. Would you comment and critique my visual-design prejudices?

  1. Never use clip art; photos are cheap, easily available, and convey more credibility.
  2. Using a transparency fade (for example when you take a photo but gradually fade one side of it into the background) is good looking, adds credibility, and enables room for pertinent text.
  3. It is better to have one major learning point per slide with a nice supporting visual, than to offer many ideas on the same slide.
  4. Using objects with some gradient is almost always preferred to no gradient.


  1. It depends on what you mean by clip art. Although you want to avoid the silly smiling characters, there are wonderful collections of illustrated and simplified vector drawing that you can use to represent concepts and objects. You can make an entire presentation or eLearning course using this minimalist and distilled style.
  2. Adding a transparency fade is a good way to be able to add text. But it’s not the only way and if it looks very feathered, it could look dated. Other ways to add text are to use a large 1024×768 photo (the size of the entire slide) and then overlay a slightly transparent rectangle on part of the photo where you want the text. Place the text within that rectangle and ensure there is enough contrast to read it.
  3. I think you’re probably right about one point per slide. Don’t tell anyone but sometimes I might put three related points on a slide. Maybe I’m being lazy.
  4. A gradient can give an object a 3D appearance, but it’s not always necessary. The flat design trend has moved away from full gradients and you will now probably see more designs with flat or solid-looking objects. I don’t think there’s a wrong or a right way to fill in objects, but as trends change, a viewer’s idea of what is aesthetically pleasing will change.



You mention perspective in your book. First, tell us what it is. Second, because you recommend it “to add realism to a story or scenario,” could you tell us if there is a secret to searching for photos with perspective in photo databases (so we don’t have to search through an endless array of photos)?


Perspective is a way to trick the eyes into perceiving three dimensions on what is really a flat surface. I don’t know of any great way to find images with perspective other than to type that word into your photo database. Such as “street perspective,” for example.


You mentioned distilled graphics. What are they and when should we use them?

Distilled graphics are simplified, schematic or iconic visuals that represent objects or concepts. We perceive and understand them quickly, similar to the images on road signs. I think it’s a good idea to use these when you want to get an idea across quickly. Also, using a distilled graphic like a silhouette works like a visual suggestion of what it represents without getting into the detail. Another suggestion is to use distilled graphics instead of bullet points, placed near the text to represent the concept or fact. It’s tough to explain without a visual accompaniment!



Connie, maybe you can help me. When I look for photos, sometimes I find myself spending half an hour or more just to find an image I deem acceptable. Am I nuts? Please help me! How long should I spend looking for an image?

Will, I’m going to guess that you are a little nuts, but not because you take so long to find photos. But yes, searching for photos is one of the most time-consuming aspects of this career. Most stock photo sites that weren’t specifically made for eLearning seem to have an advertising/marketing focus. The photographers still have that mindset, where rather than showing people in realistic situations, they show people smiling at the camera or cheering. There’s not enough diversity in the image choices either. I got so frustrated one day, I sat down and wrote an article about this problem: 21 Reasons Why Stock Photo Sites Make Me Cry.



I have a new favorite font, but while it’s in Microsoft Office, when some of the online meeting tools convert it, they replace my beautiful artsy font with a boring font often of the wrong size. Are there any ways to do a work around? Can I search-and-replace fonts for example?

For those situations, you could make a second version of the presentation for online meeting tools and change the font to a similar but more common one in your template. Even though it might be boring, it everything will line up the way you want it to. If the online meeting tool is hosted by a professional association or a company, you can see if they would be willing to install the font on the hosting computer.

As to replacing fonts, I’ve always replaced the font in the Slide Master and that usually works. You may have to choose the Master Layout again in your slide though.



Connie, I love the section in your book on creating a visual hierarchy. As you describe it, visual hierarchies send unconscious signals to our brains that prompt us to look at certain parts of the visual before other parts. I didn’t really know this until I read your book. Thank you! I’m a big believer in PowerPoint (or KeyNote, etc.) of revealing aspects of our visuals one at a time, which is probably cheating to get a similar effect to hierarchies—but sometimes there would be a huge benefit in having a visual hierarchy. Please educate us on how to create a visual hierarchy, and tell us why it’s so important.


A visual hierarchy indicates where the viewer should look first, second and perhaps third. Make your most important element the first thing that people will look at. You can do this through contrast.  Make it larger, place it in the upper left or top of the screen, make the element more colorful or brighter or add movement (if appropriate to the learning). There are other ways, but that’s a good place to start.



I notice you added humor into your book. I laughed out loud when you told me that to transform myself into an “expert designer” that I’d need to wear all black. LOL. Your book is extremely helpful—and one doesn’t even have to change their wardrobe.

What’s your most important message for us learning professionals? Besides reading your book, what else can we do to be more effective? And, are there any methods you’ve seen for getting evaluation feedback on our visual designs?

There are a lot of things learning professionals can do in addition to reading my book, dressing in all black and getting piercings in weird places:

  • Analyze the visual design in your environment and see what works and what doesn’t. Think about what the designer was trying to achieve. Notice the layout, typography, color palette and focal point. This means studying the design of websites, apps, magazines, brochures, posters, books, catalogs, packaging, billboards, subway ads, store interiors, videos, icons and junk mail. You get the idea.
  • Start an online collection (via Pinterest, bookmark sites, etc.) of designs that you like. Then use these for inspiration the next time you are stuck.
  • Read design books. Although most design books focus on advertisements and branding, they still offer a lot of sound principles and inspiration.



Finally, what’s the best way for people to get your book?

Thanks for asking. My book is available on Amazon, Barnes and Noble, and the ATD online bookstore, where it is discounted for members. I hope your readers understand that there are around 130 color graphics, which makes it a little more on the expensive side due to printing costs.


Will’s Note:
You can view the book on Amazon
by clicking the image below:

I was honored to be invited to write an afterword to 3rd edition of the classic text, Six Disciplines of Breakthrough Learning.

In my afterword, I noted the following:

"Roy Pollock, Andy Jefferson, and Cal Wick have provided a proven conceptual structure—the 6Ds—as a foundation. We’ve been shown how the 6Ds approach works in real organizations. We’ve been given practical tools that have been refined and updated. We’ve been privy to one of the best compilations of industry wisdom ever assembled in one book. We’ve been read the riot act, heard the gospel truth, and made to wonder why so many of us are failing on the fundamentals. This book lays it out for us, if only we have the guts and perseverance to do the right thing."

I've been following the work of Wick, Jefferson, and Pollock since 2006 when I awarded the very first Neon Elephant Award to Cal Wick. I love these guys! Their work has had a major impact on the field.

The book publisher, Wiley, has been nice enough to let me publish my afterword separately, so I'm going to provide you the link just below. In my provocative piece, I show how the workplace learning-and-performance field is poised for a transformation based on four vectors. It's good stuff!

And of course, I highly recommend their book, which you can buy by clicking on the image below.



There’s too much crap floating around the learning and education fields; too many myths, misconceptions, and maladaptive learning designs. I started The Debunker Club and have been working to debunk myths for many years, so I’m passionate about the need for more debunking. The need is great and the danger to learning and learners is dire.

Fortunately, entering the world is a great new book by three researchers, Pedro De Bruyckere, Paul A. Kirschner, and Casper D. Hulshof. Their book is titled, Urban Myths about Learning and Education, and it’s jam packed with a list of 35 myths that plague our field.

You can buy it through Amazon by clicking the image below.

The book is sure to have a major impact in the education and training fields.

Partial List of Topics

Here is a list of the SOME of the myths they debunk*:

  • Learning styles
  • The bastardized version of Dale’s Cone
  • 70, 20, 10
  • No need for knowledge if we can look everything up
  • Discovery learning is best
  • Problem-based learning
  • School kills creativity
  • 93% of communication is non-verbal
  • We use only 10% of our brains
  • You can train your brain with brain games
  • We think most clearly when we’re under pressure
  • Neuroscience provides helpful recommendations
  • New technology is causing a revolution in education
  • The internet belongs on the classroom
  • The internet makes us dumber
  • Class size doesn’t matter

*Note that “debunk” doesn’t necessarily mean to rule out completely! Often the authors find supporting evidence for some of the claims, or partial evidence, or they highlight boundary conditions.

Quotes from the Book

To give you a sense of the book, here are some quotes:

On Learning Styles:

“Though appealing, no solid evidence exists showing that there is any benefit in adapting and designing education and instruction to these so-called styles. It may even  e the case that in doing so, administrators, teachers, parents and even learners are negatively influencing the learning process and the products of education and instruction.”


On the Too-Ready Belief in Neuroscience:

“In practice, at the moment it is only the insights of cognitive psychology [not neuropsychology] that can be effectively used in education, but even here care needs to be taken. Neurology has the potential to add value to education, but in general there are only two real conclusions we can make at present:

– For the time being, we do not really understand all that much about the brain.
– More importantly, it is difficult to generalize what we do know into a set of concrete precepts of behavior, never mind devise methods for influencing that behavior.”


On the 70-20-10 Rule

“Informal learning is certainly very important, but we could find no evidence in the scientific literature to support the ratio of 70% information learning, 20% learning from others, and 10% formal learning.”


On Problem-Based Learning

“The use of problem-based learning to learn new content does not have a positive learning effect. But there is a positive learning effect if you use problem-based learning to further explore and remember something that the learner already knows.”


On Class Size

“Some studies show that smaller classes are not necessarily better, but that is just a part of the story. The quality of the teachers seems to be more important than class size, but other studies do suggest that smaller classes also seem to have performed better.”

Strengths of the Book

  • After each of the 35 myths, the authors write a short conclusion that very clearly and succinctly sums up their findings. This is very helpful.
  • The 35 myths are almost all very well known and important issues that need a research-based commentary.
  • The authors appear to have done their homework in researching the topics in the book. Certainly in the areas of research that I know best, their findings are consistent with my reading of the research.
  • The authors weigh complicated evidence in a manner that is fair and thoughtful.

Weaknesses of the Book

  • While the authors have designed the book specifically to reach practitioners (teachers, trainers, instructional designers, professors, and other learning professionals), they too often fall into the trap of using research jargon, which will make it difficult for some of their intended audience to fully comprehend some of the finer points of the book.

To Buy the Book, or Not?

Absolutely! Buy the book now! Occasionally, you might have some trouble with the jargon, but the most important messages will come through loud and clear.

This is a great book to peruse in short bursts. Each myth has its own chapter, which can be quickly read and deciphered. A great book to keep on your desk, in the bathroom, or on your cell phone. I’m loving it on my phone’s Kindle reader.

You can buy it on Amazon right now.


One of my favorite people in the workplace-learning space is Clark Quinn. Smart, passionate, and research-based, Clark has been consulting on learning and learning technology for decades. We worked on the Serious eLearning Manifesto together. We attempted—and failed—to build a community of workplace learning thought leaders. We commiserate frequently. I feel that I know some of what is in Clark’s heart. Now you can too!

With publication of his new book, Revolutionize Learning and Development, Clark shares his unbridled passion for our field. His book is a cross between a furious Charles Bukowski poem and a winding Jack Kerouac road trip. His words stream like a molten-steel lava flow of wisdom, love, and thunder.

Clark loves what we do. He just wishes with hope that we did it better. He puts the focus on on-the-job performance, saying that our learning solutions should be aimed at creating results in the workplace. Indeed, Clark nimbly changes our name from the Learning and Development team to the Performance and Development team! Learning is just a means to performance.

Clark comes to his wisdom based on years of immersion in the learning research and working on practical learning-design issues. Early in the book he explains how the brain works and how its architecture should impact our learning designs. This is very helpful because too many of us in the learning field think that conscious information transfer is all that’s needed, when in fact most of our work is done subconsciously.

In the book, Clark highlights the need for us to redouble our focus on two areas, performance consulting and development facilitation. Performance consulting looks at how to optimize execution. It requires that we analyze tasks and root causes before we develop appropriate solutions—and, incidentally, it demands that we not always look to training as the solution. Development facilitation is focused on how people develop their knowledge and skills over time. Where performance consulting works directly through our outside efforts, development facilitation helps “people improve while working on their own and together.”


Nuggets From the Book:

  • “Our species has in many ways survived because we learned how to physically augment our resources.” We learned how to overcome our tendency to forget by providing ourselves with informational sources, including such things as reference materials, job aids, and performance support.
  • We in the workplace learning-and-performance field used to have a responsibility to help people execute their job tasks better. Now we also have a responsibility to help them be innovative.
  • We, as workplace learning-and-performance professionals, need to work backwards. “We need to identify the performance we desire and then decide how to distribute information among the individual, network, and resources—digital and real.”
  • “As much as possible, we should resist trying to change the individual, as this is difficult…Our focus must be on how we want people to perform, and we must figure out what can be ‘in the world’… and then what has to be ‘in the head.’”
  • “The principle here is to recognize that, when we want people to perform in a resourced environment, we should develop the formal learning to incorporate the performance resources in the [learning] experience. If we can avoid formal learning, we can and should, be when we can’t, we should develop the resources before we develop the training.”
  • “The very first thing to do is to stop doing what we are doing. That, arguably, is impossible, yet there needs to be fundamental change. We have to stop being order-takers and start being performance consultants and improvement facilitators.”
  • On-the-job learning requires an environment where people feel safe to contribute and make mistakes, diversity is valued, people and groups are open to new ideas, and time is made for reflection. We, as learning professionals, can help enable these tendencies.
  • “A performance support focus is a better starting point for organizations than courses!”
  • “A real revolution in social tools has taken place…these capabilities need to be leveraged for learning as well.”
  • “The room is smarter than the smartest person in the room if you manage the process right. If not, the room might be only as smart as the most dominant person in the room or the one with the most authority.”
  • Workplace learning professionals should consider the “Least Assistance Principle.” It’s counterintuitive for many of us, but it basically suggests that we answer the question, ‘What’s the least I can do to guide performance?’” So instead of jumping in with a training solution, we should consider alternatives first.
  • Formal learning methods are fine for novices but for experts more informal learning methods are needed. See graph below.

Critiques of Our Current Practices:

  • Our tendency to have a course/event mindset keeps us from achieving real change.
  • Our tendency to focus on providing knowledge keeps us from focusing on decision-making and task competency.
  • Instead of designing instruction to make the learning intrinsically interesting, we use all manner of attention-getting gimmicks like throwing rubber squish-balls around the room.
  • One of the key things we do wrong is provide insufficient practice. “And we practice until someone gets it right, instead of practicing until they can’t get it wrong.”
  • As an industry we seem incapable of making job aids, even when they are often much more effective than training or training alone.


There’s one thing that I regret about the book. Clark needed, and deserved, a better editor/publisher—one who would have wrestled his lightning-bolt wisdom into a tighter package. For example, the book uses the term, “To Hand,” but even after going through the book twice, I still don’t get what that means.


Banging the drum for revolution, Clark Quinn has done our field a great favor! His research-based focus is on target. His call for a performance-focus captures the high ground.

If you’re new to the idea of a performance focus, the book will help you see through the smoke of current practices.

If you’re a performance true believer, you’ll deepen your passion and restock your armaments with fresh insights and imperatives.

You can buy the book by clicking below.


Nice review of Diane Ravitch's new book on schools, schooling, and the importance of good teachers.

Seems like it shows just how hard it is to find easy answers.

One of the biggest gaps in the learning-and-performance
field occurs after the training is done. Learners fail to apply what they’ve
learned and their managers fail to support training implementation.
Fortunately, the Fort Hill gang writes again. Where their blockbuster book, The Six Disciplines of Breakthrough Learning,
laid out a comprehensive process for getting training results, their new book (Getting Your Money’s Worth from Training and
) provides a call-to-action for training’s most important
players. Using the brilliantly diabolical approach of dividing the book in
half—one half for learners, the other for managers—Jefferson, Pollock, and Wick
provide an energizing action-plan to help organizations maximize training’s
impact on job performance.

I’m so impressed with the Fort Hill guys. It seems that they
(1) have looked deeply at the training-and-development trade, (2) found an area
where time and time again we fail to do what’s right, and (3) written the
perfect book to ensure that training maximizes business results. Too often in
today’s organizations, training is seen as magic pill that works without alignment
and support. In this double-dose of a book, Jefferson, Pollock, and Wick
explode that myth, helping both learners and their managers bring potency to
the training effort.

The design of the book tells the story itself. Managers read
from one cover while learners read from the other cover. The book’s title stays
the same—Getting Your Money’s Worth from
Training and Development
—but the subtitles change for the two audiences (i.
e., A Guide to Breakthrough Learning for Managers; A Guide to Breakthrough
Learning for Participants.
). Only
in partnership is training truly effective. The symbolism speaks loudly, but so
too does the content, showing how both learners and their managers can work
together to ensure that training transfers to on-the-job performance

The book is written in a conversational style. It speaks
directly to the audiences in terms that will resonate. No motherhood and apple
pie in the Fort Hill world. It’s all about results, wiifm’s, and tools. The
example worksheets in the back of each book (remember it’s two books in one)
are worth the cover price.

I recommend this book with the greatest enthusiasm. Companies
ought to buy two copies for every training participant. One for the participant
and one for his/her manager.

You can click the link below to learn more about the book (and go directly to Amazon to decide whether to purchase it).

Sharon Shrock and Bill Coscarelli recently completed the third edition of their important book, Criterion-Referenced Test Development: Technical and Legal Guidelines for Corporate Training. If this book isn’t in your collection already, I’ll give you a link below to buy from

In this third edition, Sharon and Bill have updated the book from the second edition (published in 2000) in some critical ways. One of those ways is truly transformational for the workplace learning and performance field. I’ll get to that in a minute.

Also updated is the excellent chapter at the end of the book by Patricia S. Eyres, a lawyer with employment-law credentials. Her chapter covers the legal ramifications and guidelines in dealing with employee testing, especially as that testing affects employee selection, advancement, and retention. She has updated her chapter with new case law and legal precedent from that in the second edition. Most people in the training field have very little knowledge of the legal ramifications of testing, and I’d recommend the book for this chapter alone—it’s a great wakeup call that will spur a new appreciation of the legal aspects of testing.

In the second edition, Shrock and Coscarelli put forward what they call the “Certification Suite.” In criterion-referenced testing, the goal is to decide whether a test taker has met a criterion or not. When they have met the criterion, they are said to be “certified” as competent in the area on which they were tested. The Certification Suite has six levels, some which offer full Certification and some which offer Quasi-Certification:


  1. Real World
  2. High-Fidelity Simulation
  3. Scenarios


  1. Memorization
  2. Attendance
  3. Affiliation

As the authors say in the book (p. 111), “Level C represents the last level of certification that can be considered to assess an ability to perform on the job.”

The truly transformational thing offered by Shrock and Coscarelli is that Level D Memorization, in the second edition of the book, was considered to offer Certification. NO MORE!! That’s right. Two of our leading thinkers on testing say that memorization questions are no longer good enough!!!!!!!!!!!!!! Disclosure: In speaking with Bill Coscarelli in 2006, I gently encouraged this change. This is mentioned in the book, so it’s not like I’m bragging. SMILE.

I love this, of course, because it follows what we know about human learning. For tests to be predictive of real-world performance, they have to offer similar cues to those that learners will face in the real world. If they offer different cues—like almost all memorization questions do—they are just not relevant. And, from a learning standpoint (as opposed to a certification standpoint) memorization questions won’t spur spontaneous remembering through the triggering mechanism of the real-world cues.

This literal and figurative raising of the bar—to move it beyond memorization—should shake us to our core (especially since this is one of the few books on assessment that covers legal stuff—so it may have some evidentiary heft in court). If the compliance tests at the end of your e-learning programs are based on memorization questions, you are so in trouble. If your credentialing is based on completion (and 85% of our respondents in the eLearning Guild research report said they utilized completion as a learning measure), you are in even worse trouble. And, of course, if you ever thought your memorization-level questions supported learning, well, sorry. They don’t! At least not as strongly as they might.

Have you bought the book yet? You should. You ought to at least have it around to show management (or your clients) why it’s important (absolutely freakin’ critical) to use high-value assessment items.

I’ve got some quibbles with the book as well. They list 6 reasons for testing. I’ve recently come up with 18, so it appears they’re missing some, or I’m drinking too much. I also don’t like the use of Bloom’s Taxonomy to index some of the recommendations. In short, Bloom’s has issues. I don’t like the way they talk about learning objectives. They use the methodology of relying on a single objective to guide the process of both instructional design and evaluation. I am now advocating to free instructional-design objectives from the crazy constraint of being super-glued to the evaluation objectives. They need to be linked of course, but not hog-tied. I wish they emphasized more strongly the distinction between testing to assess and testing to support learning. They are different animals and most of us are confused about this.

Overall, it’s a great and thoughtful book. I bought it. You should too.

Here’s a link that will let you click here to buy.

The Learning Measurement Series will continue in January…

(But watch to see who wins this year’s Neon Elephant Award, which I’ll announce on Saturday (December 22nd 2007). The winner(s) is/are all about learning measurement.)

Questioning Strategies for
Audience Response Systems:
How to Use Questions to Maximize
Learning, Engagement, and Satisfaction

by Dr. Will Thalheimer


The buzz in the learning industry is focused on e-learning, m-learning, wikis, and blogs; but one of the most powerful learning technologies is being overlooked, probably because it’s an in-the-classroom technology—audience response systems. In this research-to-practice white paper I offer a blueprint for how to use audience response systems to maximize higher-order learning in the classroom and beyond.

What One Reader Wrote to Me

Dr. Thalheimer,

Just wanted to drop you a little note this morning to express my gratitude for your paper "Questioning Strategies for Audience Response Systems: How to Use Questions to Maximize Learning, Engagement, and Satisfaction."

A friend recommended that I read it to prepare for a Higher Order Questioning staff development class that she and I are teaching together (in conjunction with some CPS [audience response] training we’re offering). To tell you the truth, I really wasn’t looking forward to reading it because I expected it to be dry and full of boring I’m-trying-to-sound-snobbily-intellectual writing, but I LOVED it. 🙂

I enjoyed your approachable style and dry sense of humor so much I read all the way through (including the endnotes!) and had many a good laugh along the way. In addition to being a blast to read, the paper challenged and inspired me to find new ways to push my questioning skills to a higher level for the next school year.

Thanks again, for the inspiration and for the great read. I’ll be checking out your website later today and hope to find that equally enjoyable.


Liz Walhof

Spanish Teacher

From the Paper’s Introduction

"Audience response systems have enormous potential for transforming lectures from dry recitals into rich jam sessions of deeply resonant learning. The technology is widely available, but the key to success is not in the technology; it’s in the instruction. To maximize meaningful learning, instructors must become adept in using questioning and discussion techniques. Unfortunately, some of us may come to believe that we can simply sprinkle our lectures with a few multiple-choice questions. This approach is emphatically inadequate, and is simply not worthy of our profession.

This report provides a near-exhaustive list of questioning strategies, and a comprehensive guide on using questions to facilitate classroom learning. No other resource exists that is research-based and comprehensive, while also being practical and useful. It has been designed specifically to provide practical guidance for trainers, teachers, and professors so that their learners—whether they are eight, forty-eight, or eighty years old—can experience deep and meaningful learning."

Special thanks to eInstruction for agreeing to license the paper for distribution to their clients. Such underwriting helps move the audience-response field forward and demonstrates an enlightened commitment to effective learning in classrooms of all types throughout the world. Other underwriting opportunities are available for research on audience-response learning. Contact Dr. Thalheimer with inquiries.

Additional Information

  • Number of Pages: 124
  • Number of Research Citations: 54
  • Publication Date: March 2007
  • Available to you Immediately as downloadable Electronic file (PDF).
  • Purchasing utilizes industry-leading security.
  • 100% Satisfaction Guaranteed.
  • Cost: $40.00 (US)

Click here to purchase…