I’ve had the distinct honor of being invited to speak at the Learning Technologies conference in London for three years in a row. This year, I talked about two learning innovations:
- Performance-Focused Learner Surveys
- LTEM (The Learning-Transfer Evaluation Model)
It was a hand-raising experience!
Most importantly, they have done a great job capturing my talk on YouTube.
Indeed, although I’ve made some recent improvements in the way I talk about these two learning innovations, the video does an excellent job of capturing some of the main points I’ve been making about the state of learning evaluation and two innovations that are tearing down some of the obstacles that have held us back from doing good evaluation.
Thanks to Stella Collins at Stellar Learning for organizing and facilitating my session!
Special thanks to the brilliant conference organizer and learning-industry influencer Robert Taylor for inviting and supporting me and my work.
This is NOT a post about Bob Mager. It is something else entirely.
In probably the best video I will ever create, I made the case that learning professionals and learners should NOT receive the same set of learning objectives.
The rationale is this: Because objectives are designed to guide behavior, how could one statement possibly guide the behaviors of two separate audiences? Sometimes maybe! But not always!
Arguments for the Infallibility of an Instructional-Design Hero
Recently, I’ve heard it argued that Bob Mager, in his classic text, “Preparing Instructional Objectives,” urged us to create instructional objectives only for us as learning professionals, that he never intended that instructional objectives be presented to learners. This is a testable assertion, which is great! We can agree that Mager gave us some good advice on how to craft objectives for ourselves as learning professionals. But did Mager also, perhaps, suggest that objectives could be presented to learners?
Here are several word-for-word quotes from Mager’s book:
Page 16: Heading: “Goal Posts for Students”
Page 16: “Clearly defined objectives also can be used to provide students with the means to organize their own time and efforts toward accomplishment of those objectives.”
Page 17: “With clear objectives, it is possible to organize the instruction itself so that instructors and students alike can focus their efforts on bridging the gap…”
Page 19: Chapter Summary. “Objectives are useful for providing: … Tools for guiding student efforts…”
Page 43: “Objectives in the hands of students prevent the students from having to guess at how they might best organize their time and effort.”
So Mager clearly started the confusion! But Mager wrote at a time before research on cognition enabled greater insight.
Forget Mager’s contribution. The big problem is that the most common practice seems to still be efforts to create a set of learning objectives to use for both learners and learning practitioners.
I was even scolded for not knowing the difference between an instructional objective (for learning professionals) and a learning objective (for learners). Of course, these revisionist definitions are not true and are not helpful. They are fake news, concocted perhaps by a person who thinks or was taught that our instructional-design heroes are perfect and their work is sacrosanct. The truth is that these terms have been used interchangeably. For example, in a research study by my mentor and academic advisor, Ernie Rothkopf, he and his research partner used the term instructional objectives to refer to objectives presented to learners.
Rothkopf, E. Z., & Kaplan, R. (1972). An exploration of the effect of density and specificity of instructional objectives on learning from text. Journal of Educational Psychology, 6, 295-302.
My Main Points
- We need at least two types of objectives (although I’ve argued for more)—one to guide the design, development, and evaluation of learning; one to guide learners as they are learning. I’ve called these “focusing objectives,” because the research shows that they guide attention toward objective-relevant content.
- When we make arguments, we ought to at least skim the sources to see if we know what we’re talking about.
- We ought to stop with hero worship. All of us do some good things and some bad things. Even the best of us.
- Hero worship in the learning field is particularly problematic because learning is so complex and we all still have so much to learn. All of us attempting to make recommendations are likely to be wrong some of the time.
- It is ironic that our schools of instructional design teach graduate students to memorize facts and hold up heroes as infallible immortals—when instead they ought to be educating these future citizens how progress gets made over long periods of time by a large collective of people. They also ought to be teaching students to understand at a deeper level, not just a knowledge level. But truly, we can’t blame the schools of instructional design. After all, they started with canonically-correct instructional objectives (focused on low-level knowledge because they are easier to create).
Finally, let me say that in the video I praise Bob Mager’s work on learning objectives for us learning professionals. This post is not about Mager.
Guy Wallace has been an exemplar of the highest quality in the performance-improvement field for decades. His 31-page bio is a testament to his incredible work experience. He has worked with other industry luminaries including Dick Hanshaw, Geary Rummler, Dick Clark, Dale Brethower. He not only has been at the center of the move from training to performance—represented in the long arc of ISPI—he’s been capturing that history for years.
I highly recommend his video series.
The only blemish in that series is the video interview he released this week, featuring me. Legacy schmegacy! Seriously though, I am honored. Thank you Guy for all you do and have done!
And Guy’s still going strong in his work, offering optimal methodologies in performance analysis/assessment and curriculum architecture.
Research shows that one-on-one tutoring is generally highly effective — with a good tutor, of course. Similarly, John Anderson — the cognitive psychologist — along with others — developed Intelligent tutoring systems (ITS) that tailor content to learners based on cognitive models of what they know and don't know.
These learning approaches are beneficial because they personalize learning based on a diagnosis of learner knowledge. This is not a new concept. Indeed, the Socratic Method also took learners on a journey based on their responses. B. F. Skinner's operant conditioning provided reinforcement based on learner actions. Skinner's Programmed Learning and Fred Keller's Personalized System of Instruction are practical applications based on operant conditioning principles.
We've known for millennium that personalized learning is good — and we've even dabbled in scalable implementations like programmed learning — but for the most part we are still awaiting the promise of such personalization.
Now may be the time. New technologies are beginning to show promise. For example, subscription-learning threads based on personalized spacing schedules personalize learning nuggets based on learner responses. Still, one area where personalization hasn't been much in evidence is video. That may be changing.
Consensus allows its users to create videos tailored to each learner. Their focus is on sales and marketing — and particularly on creating demos — but the technology could be used for other needs as well. Consensus tailors content to users based on their interests, background, etc. When folks choose a topic as “Very Important,” users get the full video segment and get it first. When folks choose a topic as “Somewhat Important,” users get a summary video. When they choose “Not Important,” users don’t get any segments associated with those topics.
Consensus can also segment users based on their role in an organization, based on the size of their company, or based on other demographics as well.
They’ve got a great video that you can tailor to your needs. Worth watching! Click to check out Consensus.
I created a video to help organizations fully understand the meaning of their smile sheets.
You can also view this directly on YouTube: https://youtu.be/QucqCxM2qW4
For years I've been telling clients they ought to use more video in their learning programs, especially their elearning offerings. My arguments have been as follows:
- People react to well-crafted videos and audio with increased attention.
- The storytelling often inherent in video is powerfully seductive.
- Video is now fairly cheap; you don't necessarily need high production values, expensive equipment, or professional help.
- Nothing persuades our learners better than seeing real people who are like them — giving testimony, telling their stories, giving their lessons learned.
- Video can utilize scenario-based decision making, which we know from the learning research is a powerful tool to support comprehension and remembering.
- More and more of our learners are everyday video watchers; their expectations for media consumption are more visual, less textual.
Video is the New Text
Today's headlines hint that video on the internet is the number one draw. Certainly, some of these are silly cat videos, but now serious sources are turning to videos. Take for example the TED videos, the New York Times, The Economist. Even National Public Radio (whose life blood flows through a non-visual medium) has a YouTube channel!
Video is here to stay. As of this day in 2015, more and more elearning is utilizing good video; but still more can be done. Still too many instructional designers don't have video skills or even knowledge. Still too many opportunities are lost for getting employees on video telling their stories and lessons learned. Still too few scenario-based decisions are wrapped in a video context.
But Isn't Video Hard to Do?
It's NOT easier than writing text, especially since most of us have more text-writing experience than video-creation experience. But it's not that hard and it doesn't have to be expensive.
My confession is that even while I was imploring my clients to use video, I rarely used it. So, six years ago I began to learn about video. I put in some time to learn it. I bought myself video equipment. I produced a few videos. Here's a sampling:
- My latest one on learning objectives (good overall, with some minor issues)
- One where I interviewed myself (with audio problems–it was too quiet)
- One that was really just a narrated PowerPoint
- Another narrated PowerPoint (that went on too long)
- An interview I did with Eric Shepherd of Questionmark
- My second video ever (I'm stiff as a board, but my daughter and Allison Rossett shine)
I still don't do very many videos. As a consultant I can't really afford to take the time, but I create them occasionally because of the value they provide.
If you think you can't do video, check out the second video I ever produced (the last one on the list). As an on-screen presence, I was terrible, but overall the video is pretty good.
My List of Equipment:
So you can learn from my consumer research, here is the list of video equipment I use in my videos.
- Consumer HD video camcorder. NOT a professional video camera. About 7 years old, so not the latest technology.
- Inexpensive wireless microphone. (something like this)
- Inexpensive tripod (something like this)
- Inexpensive lighting (something like this, and this)
- Inexpensive video-editing software (like this)
- Inexpensive audio-editing software (like that included in most video-editing software).
I also endeavor to use some aesthetic sensibilities; developed over the years regarding audio, music, visuals, cinematography, etc. based on who knows what (going to art museums, listening to music, playing music as a kid, observing the craft while watching videos, movies, TV, etc.). I really don't know what I know or what I lack, but I do know that some aesthetic sensibility is important. I also know that real video pros have more of this than I do…so there are definite advantages to getting help from the pros.
One of my mentors in video production is Jason Fararooei of Yellow Cape Communications. I first met him at an ISPI conference where he talked about how he and his team created a video for a client learning engagement. I liked his thoughtful approach to the aesthetics and potency of video. I'm pretty sure the following video is from the talk I saw him give:
Jason's in the business of creating videos, so my videos certainly don't come up to his standards, but he does seem mindful that shooting on a budget can still produce good results. Here he talks about recording a conference session, but the idea can be used for recording short snippets from training as well.
Video IS the New Text…Sometimes
I don't really think that video will replace text, but it will replace some text. And remember this. In evaluating video vs. text, you can't just look at the costs. It's the cost/benefit that matters. In some sense you may have to do a comparison like the following (note the numbers are pulled from thin air as examples):
- TEXT PASSAGE that only 20 out of 100 people read.
- VIDEO SHORT that 70 out of 100 people view.
Your numbers will vary…but the point should be clear. Where video gets more eyeballs and more mathemagenic processing (learning-generating processing), it may be worth the extra investment.
Many times, you and your team will be able to create the video. Sometimes you'll have to get help from a professional.
Large organizations, or vendors who produce lots of instruction, should consider developing video capability in-house.
For all of us who call ourselves instructional designers, we ought to dive in and learn some video skills. If I can do it, you can too!
Dr. T, What Are You Thinking?
And, just to avoid 100 snippy comments, let me anticipate your next question…Hey, Dr. T, why did you use TEXT here, NOT VIDEO?
Roll the credits…SMILE
It has been my pleasure and privilege to co-teach several learning measurement workshops with Dr. Roy Pollock, and to follow the important work that he and his colleagues have done at The Fort Hill Company over the years. I acknowledged their work by awarding Cal Wick, Fort Hill's Founding Father, the Neon Elephant Award back in 2006. I've also reviewed their ground-breaking book, The Six Disciplines of Breakthrough Learning, and have recently reviewed their new book, Getting Your Money's Worth from Training and Development.
Now, I have captured Roy in a video interview, that I think you'll enjoy and learn from.
Again, I highly recommend the book. Read my book review to see how much.
Learning professionals (like me) can often gain insights about our industry from people in the field who have different vantage points than our own. I recently talked with Eric Shepherd, CEO of Questionmark, to get a sense of our industry and how it has been affected by the bad economy. Eric has been a good friend and long-time supporter of my research over the years and I’ve come to value his counsel.
Questionmark is the leading provider of assessment software according to a recent eLearning Guild study. I thought from his perch overseeing all-things-assessment, Eric might be able to give us some unique insight into the learning-and-performance field in general.
Check out my interview with him at the recent Guild conference. I divided it into two parts to make viewing easier.
Part 1: What trends do you see that we may be missing?
The following video provides an entertaining and, I hope, enlightening look at the humble job aid.
- This is only the second video that I shot and edited. See how I did.
- Allison Rossett, co-author of the book, Job Aids and Performance Support (with Lisa Schafer) is interviewed.
- Worldwide public introduction to incredible new talent, the incomparable Alena.
- Brewer the dog has cameo role.
Because of YouTube size restrictions, it is divided into 2 parts.
Enjoy in HD (if your computer can handle it) by:
- Starting the Video
- Clicking on HD at Lower Right, AND
- Clicking on the full-screen display (the box in a box) at Lower Right
- IF the audio doesn't track, your computer can't handle HD.
Purchasing (or learning more about) Allison's Book: