Posts

CEO’s are calling for their companies to be more innovative in the ever-accelerating competitive landscape! Creativity is the key leverage point for innovation. Research I’ve compiled (from the science on creativity) shows that unique and valuable ideas are generated when people and teams look beyond their inner circle to those in their peripheral networks. GIVEN THIS, a smart company will seed themselves with outside influencers who are working with new ideas.

But what are a vast majority of big companies doing that kills their own creativity? They are making it difficult or virtually impossible for their front-line departments to hire small businesses and consultants. It’s allowed, but massive walls are being built! And these walls have exploded over the last five to ten years:

  1. Only fully vetted companies can be hired, requiring small lean companies to waste time in compliance—or turn away in frustration. Also causing large-company managers to favor the vetted companies, even if a small business or consultant would provide better value or more-pertinent products or services.
  2. Master Service Agreements are required (pushing small companies away due to time and legal fees).
  3. Astronomical amounts of insurance are required. Why the hell do consultants need $2 million in insurance, even when they are consulting on non-safety-related issues? Why do they need any insurance at all if they are not impacting critical safety factors?
  4. Companies can’t be hired unless they’ve been in business for 5 or 10 or 15 years, completely eliminating the most unique and innovative small businesses or consultants—those who recently set up shop.
  5. Minimum company revenues are required, often in the millions of dollars.

These barriers, of course, aren’t the only ones pushing large organizations away from small businesses or consultants. Small companies often can’t afford sales forces or marketing budgets so they are less likely to gain large companies’ share of attention. Small companies aren’t seen as safe bets because they don’t have a name, or their website is not as beautiful, or they haven’t yet worked with other big-name companies, or the don’t speak the corporate language. Given these surface characteristics, only the bravest, most visionary frontline managers will take the risk to make the creative hire. And even then, their companies are making it increasingly hard for them to follow through.

Don’t be fooled by the high-visibility anecdotes that show a CEO hiring a book author or someone featured in Wired, HBR, or on some podcast. Yes, CEO’s and senior managers can easily find ways to hire innovators, and the resulting top-down creativity infusion can be helpful. But it can be harmful as well!!!! Too many times senior managers are too far away from knowing what works and what’s needed on the front lines. They push things innocently not knowing that they are distracting the troops from what’s most important, or worse, pushing the frontline teams to do stupid stuff against their best judgment.

Even more troublesome with these anecdotes of top-down innovation is that they are too few and far between. There may be ten senior managers who can hire innovation seeds, but there are dozens or hundreds or thousands of folks who might be doing so but can’t.

A little digression: It’s the frontline managers who know what’s needed—or perhaps more importantly the “leveraging managers” if I can coin a term. These are the managers who are deeply experienced and wise in the work that is getting done, but high enough in the organization to see the business-case big picture. I will specifically exclude “bottle-cap managers” who have little or no experience in a work area, but were placed there because they have business experience. Research shows these kind of hires are particularly counterproductive in innovation.

Let me summarize.

I’m not selling anything here. I’m in the training, talent development, learning evaluation business as a consultant—I’m not an innovation consultant! I’m just sharing this out of my own frustration with these stupid counter-productive barriers that I and my friends in small businesses and consultancies have experienced. I also am venting here to provide a call to action for large organizations to wake the hell up to the harm you are inflicting on yourselves and on the economy in general. By not supporting the most innovative small companies and consultants, you are dumbing-down the workforce for years to come!

Alright! I suppose I should offer to help instead of just gripe! I have done extensive research on creativity. But I don’t have a workshop developed, the research is not yet in publishable form, and it’s not really what I’m focused on right now. I’m focused on innovating in learning evaluation (see my new learning-evaluation model and my new method for capturing valid and meaningful data from learners). These are two of the most important innovations in learning evaluation in the past few years!

However, a good friend of mine did, just last month, suggest that the world should see the research on creativity that I’ve compiled (thanks Mirjam!). Given the right organization, situation, and requirements—and the right amount of money—I might be willing to take a break from my learning-evaluation work and bring this research to your organization. Contact me to try and twist my arm!

I’m serious, I really don’t want to do this right now, but if I can capture funds to reinvest in my learning-evaluation innovations, I just might be persuaded. On the contact-me link, you can set up an appointment with me. I’d love to talk with you if you want to talk innovation or learning evaluation.

A huge fiery debate rages in the learning field.

 

What do we call ourselves? Are we instructional designers, learning designers, learning experience designers, learning engineers, etc.? This is an important question, of course, because words matter. But it is also a big freakin’ waste of time, so today, I’m going to end the debate! From now on we will call ourselves by one name. We will never debate this again. We will spend our valuable time on more important matters. You will thank me later! Probably after I am dead.

How do I know the name I propose is the best name? I just know. And you will know it too when you hear the simple brilliance of it.

How do I know the name I propose is the best name? Because Jim Kirkpatrick and I are in almost complete agreement on this, and, well, we have a rocky history.

How do I know the name I propose is the best name? Because it’s NOT the new stylish name everybody’s now printing on their business cards and sharing on LinkedIn. That name is a disaster, as I will explain.

The Most Popular Contenders

I will now list each of the major contenders for what we should call ourselves and then thoroughly eviscerate each one.

Instructional Designer

This is the traditional moniker—used for decades. I have called myself an instructional designer and felt good about it. The term has the benefit of being widely known in our field but it has severe deficiencies. First, if you’re at a party and you tell people you’re an instructional designer, they’re likely to hear “structural designer” or “something-something designer” and think you’re an engineer or a new-age guru who has inhaled too much incense. Second, our job is NOT to create instruction, but to help people learn. Third, our job is NOT ONLY to create instruction to help people learn, but to also create, nurture, or enable contexts that help people learn. Instructional designer is traditional, but not precise. It sends the wrong message. We should discard it.

Learning Designer

This is not bad. It’s my second choice. But it suffers from being too vanilla, too plain, too much lacking in energy. More problematic is that it conveys the notion that we can control learning. We cannot design learning! We can only create or influence situations and materials and messages that enable learning and mathemagenic processes—that is, cognitive processes that give rise to learning. We must discard this label too.

Learning Engineer

This seems reasonable at first glance. We might think our job is to engineer learning—to take the science and technology of learning and use it to blueprint learning interventions. But this is NOT our job. Again, we don’t control learning. We can’t control learning. We can just enable it. Yes! The same argument against “designing learning” can be used against “engineering learning.” We must also reject the learning engineering label because there are a bunch of crazed technology evangelists running around advocating for learning engineering who think that big data and artificial intelligence is going to solve all the problems of the learning profession. While it is true that data will help support learning efforts, we are more likely to make a mess of this by focusing on what is easy to measure and not on what is important and difficult to measure. We must reject this label too!

Learning Experience Designer

This new label is the HOT new label in our field, but it’s a disastrous turn backward! Is that who we are—designers of experiences? Look, I get it. It seems good on the surface. It overcomes the problem of control. If we design experiences, we rightly admit that we are not able to control learning but can only enable it through learning experiences. That’s good as far as it goes. But is that all there is? NO DAMMIT! It’s a freakin’ cop-out, probably generated and supported by learning-technology platform vendors to help sell their wares! What the hell are we thinking? Isn’t it our responsibility to do more than design experiences? We’re supposed to do everything we can to use learning as a tool to create benefits. We want to help people perform better! We want to help organizations get better results! We want to create benefits that ripple through our learners’ lives and through networks of humanity. Is it okay to just create experiences and be happy with that? If you think so, I wish to hell you’d get out of the learning profession and cast your lack of passion and your incompetence into a field that doesn’t matter as much as learning! Yes! This is that serious!

As learning professionals we need to create experiences, but we also need to influence or create the conditions where our learners are motivated and resourced and supported in applying their learning. We need to utilize learning factors that enable remembering. We need to create knowledge repositories and prompting mechanisms like job aids and performance support. We need to work to create organizational cultures and habits of work that enable learning. We need to support creative thinking so people have insights that they otherwise wouldn’t have. We also must create learning-evaluation systems that give us feedback so we can create cycles of continuous improvement. If we’re just creating experiences, we are in the darkest and most dangerous depths of denial. We must reject this label and immediately erase the term “Learning Experience Designer” from our email signatures, business cards, and LinkedIn profiles!

The Best Moniker for us as Learning Professionals

First, let me say that there are many roles for us learning professionals. I’ve been talking about the overarching design/development role, but there are also trainers, instructors, teachers, professors, lecturers, facilitators, graphic designers, elearning developers, evaluators, database managers, technologists, programmers, LMS technicians, supervisors, team leaders, et cetera, et cetera, et cetera. Acknowledged!!! Now let me continue. Thanks!

A month ago, Mirjam Neelen reached out to me because she is writing a book on how to use the science of learning in our role as learning professionals. She’s doing this with another brilliant research-to-practice advocate, the learning researcher Paul Kirschner, following from their blog, 3-Star Learning. Anyway, Mirjam asked me what recommendation I might have for what we call ourselves. It was a good question, and I gave her my answer.

I gave her THE answer. I’m not sure she agreed and she and Paul and their publisher probably have to negotiate a bit, but regardless, I came away from my discussions with Mirjam convinced that the learning god had spoken to me and asked me to share the good word with you. I will now end this debate. The label we should use instead of the others is Learning Architect. This is who we are! This is who we should be!

Let’s think about what architects do—architects in the traditional sense. They study human nature and human needs, as well as the science and technology of construction, and use that knowledge/wisdom to create buildings that enable us human beings to live well. Architects blueprint the plans—practical plans—for how to build the building and then they support the people who actually construct the buildings to ensure that the building’s features will work as well as possible. After the building is finished, the people in the buildings lead their lives under the influence of the building’s design features. The best architects then assess the outcomes of those design features and suggest modifications and improvements to meet the goals and needs of the inhabitants.

We aspire to be like architects. We don’t control learning, but we’d like to influence it. We’d like to motivate our learners to engage in learning and to apply what they’ve learned. We’d like to support our learners in remembering. We’d like to help them overcome obstacles. We’d like to put structures in place to enable a culture of learning, to give learners support and resources, to keep learners focused on applying what they’ve learned. We’d like to support teams and supervisors in their roles of enabling learning. We’d like to measure learning to get feedback on learning so that we can improve learning and troubleshoot if our learners are having problems using what we’ve created or applying what they’ve learned.

We are learning architects so let’s start calling ourselves by that name!

But Isn’t “Architect” a Protected Name?

Christy Tucker (thanks Christy!) raised an important concern in the comments below, and her concern was echoed by Sean Rea and Brett Christensen. The term “architect” is a protected term, which you can read about on Wikipedia. Architects rightly want to protect their professional reputation and keep their fees high, protected from competition from people with less education, experience, and competence.

But, to my non-legal mind, this is completely irrelevant to our discussion. When we add an adjective, the name is a different name. It’s not legal to call yourself a doctor if you’re not a doctor, but it’s okay to call yourself the computer doctor, the window doctor, the cakemix doctor, the toilet doctor, or the LMS doctor.

While the term “architect” is protected, putting an adjective in front of the name changes everything. A search of LinkedIn for “data architects” lists 57,624 of them. A search of “software architect” finds 172,998. There are 3,110 “performance architects,” 24 “justice architects,” and 178 “sustainability architects.”

Already on LinkedIn, 2,396 people call themselves “learning architects.”

Searching DuckDuckGo, some of the top results were consultants calling themselves learning architects from the UK, New Zealand, Australia. LinkedIn says there are almost 10,000 learning architecture jobs in the United States.

This is a non-issue. First, adding the adjective changes the name legally. Second, even if it didn’t, there is no way that architect credentialing bodies are going to take legal action against the hundreds of thousands of people using the word “architect” with an adjective. I say this, of course, not as a lawyer—and you should not rely on my advice as legal advice.

But still, this has every appearance of being a non-issue and we learning professionals should not be so meek as to shy away from using the term learning architect.

I was listening to a podcast last week that interviewed Jim Kirkpatrick. I like to listen to what Jim and Wendy have to say because many people I speak with in my work doing learning evaluation are influenced by what they say and write. As you probably know, I think the Kirkpatrick-Katzell Four-Level Model causes more harm then good, but I like to listen and learn things from the Kirkpatrick’s even though I never hear them sharing ideas that are critical of their models and teachings. Yes! I’m offering constructive criticism! Anyway, I was listening to the podcast and agreeing with most of what Jim was saying when he mentioned that what we ought to call ourselves is, wait for it, wait for it, wait for it: “Learning-and-Performance Architects!” Did I mention that I just love Jim Kirkpatrick! Jim and I are in complete agreement on this. I’ll quibble in that the name Learning-and-Performance Architect is too long, but I agree with the sentiment that we ought to see performance as part of our responsibility.

So I did some internet searching this week for the term “Learning Architect.” I found a job at IBM with that title, estimated by Glassdoor to pay between $104,000 and $146,000, and I think I’m going to apply for that job as this consulting thing is kind of difficult these days, especially having to write incisive witty profound historic blog posts for no money and no fame.

I also found a podcast by the eLearning Coach Connie Malamed on her excellent podcast where she reviews a book by the brilliant and provocative Clive Shepherd with the title, The New Learning Architect. It was published in 2011 and now has an updated 2016 edition. Interestingly, in a post from just this year in 2019, Clive is much less demonstrative about advocating for the term Learning Architect, and casually mentions that Learning Solutions Designer is a possibility before rejecting it because of the acronym LSD. I will reject it because designing solutions may give some the idea that we are designing things, when we need to design more than tangible objects.

In searching the internet, I also found three consultants or group of consultants calling themselves learning architects. I also searched LinkedIn and found that the amazing Tom Kuhlmann has been Vice President of Community at Articulate for 12 years but added the title of Chief Learning Architect four years and eight months ago. I know Tom’s great because of our personal conversations in London and because he’s always sharing news of my good works to the Articulate community (you are, right? Tom?), but most importantly because on Tom’s LinkedIn page one of the world’s top entrepreneurs offered a testimonial that Tom improved his visual presentations by 12.9472%. You can’t make this stuff up, not even if you’re a learning experience designer high on LSD!

Clearly, this Learning Architect idea is not a new thing! But I have it on good authority that now here today, May 24, 2019, we are all learning architects!

Here are two visual representations I sent to Mirjam to help convey the breadth and depth of what a Learning Architect should do:

 

I offer these to encourage reflection and discussion. They were admittedly a rather quick creation, so certainly, they must have blind spots.

Feel free to discuss below or elsewhere the ideas discussed in this article.

And go out and be the best learning architect you can be!

I have it on good authority that you will be…

 

 

 

I’m trying to develop a taxonomy for types of learning. I’ve been working on this for several years, but I want to get one more round of feedback to see if I’m missing anything. Please provide your feedback below or contact me directly.

Types of Learning (Proposed Taxonomy)

SHORT LEARNING

  • READ AND ACKNOWLEDGE (rules, regulations, or policies)
  • WEBINAR (90 minutes or less)
  • DISCUSSION-BASED LEARNING (not training, but more of a discussion to enable learning)

TRADITIONAL GUIDED LEARNING

  • CLASSROOM LEARNING (where an instructor/facilitator leads classroom activities)
  • LIVE-FACILITATED ELEARNING (eLearning facilitated and/or presented by a live person; more involved than a basic webinar)
  • SEMI-FACILITATED ELEARNING (eLearning periodically facilitated by an instructor or learning leader as learning takes place over time)
  • NON-FACILITATED ELEARNING (where materials are presented/available, but no person is actively guiding the learning)

LEARNING OVER TIME

  • SELF-STUDY LEARNING (learners provided materials that they largely learn from on their own)
  • SUBSCRIPTION LEARNING (short nuggets delivered over a week or more)

PRACTICE-BASED LEARNING

  • SKILL-PRACTICE (where focus is on improving based on practicing, not on learning lots of new information)
  • ACTION LEARNING (involving both training and on-the-job experiences designed to support learning)
  • APPRENTICESHIP (where people learn by working under the close guidance of more-experienced others)
  • MENTORSHIP, INTERNSHIP, COACHING, SUPERVISION (where a person gets periodic feedback and guidance to elicit learning)

MISCELLANEOUS LEARNING

  • ONBOARDING (where people are introduced to a new organization, unit, or job role)
  • TEAM LEARNING (where groups of people plan and organize themselves to intentionally learn from each other)

I’m thrilled to announce that my Gold-Certification Workshop on Performance-Focused Smile Sheets is now open for registration, with access available in about a week on Tuesday May 14 (2019).

This certification workshop is the culmination of years of work and practice. First there was my work with clients on evaluation. Then there was the book. Then I gained extensive experience building and piloting smile sheets with a variety of organizations. I taught classroom and webinar workshops. I spoke at conferences and gave keynotes. And of course, I developed and launched LTEM (The Learning-Transfer Evaluation Model), which is revolutionizing the practice of workplace learning—and providing the first serious alternative to the Kirkpatrick-Katzell Four-Level Model.

Over the last year, I’ve been building an online, asynchronous workshop that was rigorous, comprehensive, and challenging enough to offer a certification. It’s now ready to go!

I’d love if you would enroll and join me and others in learning!

You can learn more about this Gold-Certification Workshop by clicking here.

 

The 70-20-10 Framework has been all the rage for the last five or ten years in the workplace learning field. Indeed, I organized a great debate about 70-20-10 through The Debunker Club (you can see the tweet stream here). I have gone on record saying that the numbers don’t have a sound research backing, but that the concept is a good one—particularly the idea that we as learning professionals ought to leverage on-the-job learning where we can.

What is 70-20-10?

The 70-20-10 framework is built on the belief that 10% of workplace learning is, or should be, propelled by formal training; that 20% is, or should be, enabled by learning directly from others; and that 70% of workplace learning is, or should, come from employee’s learning through workplace experiences.

Supported by Research?

Given all the energy around 70-20-10, you might think that lots of rigorous scientific research has been done on the framework. Well, you would be wrong!

In fact, up until today (April 19, 2019), only one study has been published in a scientific journal (my search of PsycINFO only reveals one study). In this post, I will review that one study, published last year:

Johnson, S. J., Blackman, D. A., & Buick, F. (2018). The 70:20:10 framework and the transfer of learning. Human Resource Development Quarterly. Advance online publication.

Caveats

All research has strengths, weaknesses, and limitations—and it’s helpful to acknowledge these so we can think clearly. First, one study cannot be definitive, and this is just one study. Also, this study is qualitative and relies on subjective inputs to draw its conclusions. Ideally, we’d like to have more objective measures utilized. It is also gathering data from a small sample of public sector workers, where ideally we want a wider range of diverse participants.

Methodology

The researchers found a group of organizations who had been bombarded with messages and training to encourage the use of the 70-20-10 model. Specifically, the APSC (The Australian Public Sector Commission), starting in 2011, encouraged the Australian public sector to embrace 70-20-10.

The specific study “draws from the experiences of two groups of Australian public sector managers: senior managers responsible for implementing the 70:20:10 framework within their organization; and middle managers who have undergone management capability development aligned to the 70:20:10 framework. All managers were drawn from the Commonwealth, Victorian, Queensland, and Northern Territory governments.”

A qualitative approach was chosen according to the researchers “given the atheoretical nature of the 70:20:10 framework and the lack of theory or evidence to provide a research framework.”

The qualitative approaches used by the researchers were individual structured interviews and group structured interviews.

The researchers chose people to interview based on their experience using the 70-20-10 framework to develop middle managers. “A purposive sampling technique was adopted, selecting participants who had specific knowledge of, and experience with, middle management capability development in line with the 70:20:10 framework.”

The researchers used a text-processing program (NVivo) to help them organize and make sense of the qualitative data (the words collected in the interviews). According to Wikipedia, “NVivo is intended to help users organize and analyze non-numerical or unstructured data. The software allows users to classify, sort and arrange information; examine relationships in the data; and combine analysis with linking, shaping, searching and modeling.”

Overall Results

The authors conclude the following:

“In terms of implications for practice, the 70:20:10 framework has the potential to better guide the achievement of capability development through improved learning transfer in the public sector. However, this will only occur if future implementation guidelines focus on both the types of learning required and how to integrate them in a meaningful way. Actively addressing the impact that senior managers and peers have in how learning is integrated into the workplace through both social modeling and organizational support… will also need to become a core part of any effective implementation.”

“Using a large qualitative data set that enabled the exploration of participant perspectives and experiences of using the 70:20:10 framework in situ, we found that, despite many Australian public sector organizations implementing the framework, to date it is failing to deliver desired learning transfer results. This failure can be attributed to four misconceptions in the framework’s implementation: (a) an overconfident assumption that unstructured experiential learning will automatically result in capability development; (b) a narrow interpretation of social learning and a failure to recognize the role social learning has in integrating experiential, social and formal learning; (c) the expectation that managerial behavior would automatically change following formal training and development activities without the need to actively support the process; and (d) a lack of recognition of the requirement of a planned and integrated relationship between the elements of the 70:20:10 framework.”

Specific Difficulties

With Experiential Learning

“Senior managers indicated that one reason for adopting the 70:20:10 framework was that the dominant element of 70% development achieved through experiential learning reflected their expectation that employees should learn on the job. However, when talking to the middle managers themselves, it was not clear how such learning was being supported. Participants suggested that one problem was a leadership perception across senior managers that middle managers could automatically transition into middle management roles without a great deal of support or development.”

“The most common concern, however, was that experiential learning efficacy was challenged because managers were acquiring inappropriate behaviors on the job based on what they saw around them every day.”

“We found that experiential learning, as it is currently being implemented, is predominantly unstructured and unmanaged, that is, systems are not put in place in the work environment to support learning. It was anticipated that managers would learn on the job, without adequate preparation, additional support, or resourcing to facilitate effective learning.”

With Social Learning

“Overall, participants welcomed the potential of social learning, which could help them make sense of their con-text, enabling both sense making of new knowledge acquired and reinforcing what was appropriate both in, and for, their organization. However, they made it clear that, despite apparent organizational awareness of the value of social learning, it was predominantly dependent upon the preferences and working styles of individual managers, rather than being supported systematically through organizationally designed learning programs. Consequently, it was apparent that social learning was not being utilized in the way intended in the 70:20:10 framework in that it was not usually integrated with formal or experiential learning.”

Mentoring

“Mentoring was consistently highlighted by middle and senior managers as being important for both supporting a middle manager’s current job and for building future capacity.”

“Despite mentoring being consistently raised as the most favored form of development, it was not always formally supported by the organization, meaning that, in many instances, mentoring was lacking for middle managers.”

“A lack of systemic approaches to mentoring meant it was fragile and often temporary.”

Peer Support

“Peer support and networking encouraged middle managers to adopt a broader perspective and engage in a community of practice to develop ideas regarding implementing new skills.”

“However, despite managers agreeing that networks and peer support would assist them to build capability and transfer learning to the workplace, there appeared to be few organizationally supported peer learning opportunities. It was largely up to individuals to actively seek out and join their own networks.”

With Formal Learning

“Formal learning programs were recognized by middle and senior managers as important forms of capability development. Attendance was often encouraged for new middle managers.”

“However, not all experiences with formal training programs were positive, with both senior and middle managers reflecting on their ineffectiveness.”

“For the most part, participants reported finishing formal development programs with little to no follow up.”

“There was a lack of both social and experiential support for embedding this learning. The lack of social learning support partly revolved around the high workloads of managers and the lack of time devoted to development activities.”

“The lack of experiential support and senior management feedback meant that many middle managers did not have the opportunity to practice and further develop their new skills, despite their initial enthusiasm.”

“A key issue with this was the lack of direct and clear guidance provided by their line managers.”

“A further issue with formal learning was that it was often designed generically for groups of participants…  The need for specificity also related to the lack of explicit, individualized feedback provided by their line manager to reinforce and embed learning.”

What Should We Make of This Preliminary Research?

Again, with only one study—and a qualitative one conducted on a narrow type of participant—we should be very careful in drawing conclusions.

Still, the study can be helpful in helping us develop hypotheses for further testing—both by researchers and by us as learning professionals.

We also ought to be careful in casting doubt on the 70-20-10 framework itself. Indeed, the research seems to suggest that the framework was not always implemented as intended. On the other hand, when it is demonstrated that a model tends to be used poorly in its routine use, then we should become skeptical that it will produce reliable benefits.

Here are a list of reflections generated in me by the research:

  1. Why so much excitement for 70-20-10 with so little research backing?
  2. Formal training was found to have all the problems normally associated with it, especially the lack of follow-through and after-training support—so we still need to work to improve it!
  3. Who will provide continuous support for experiential and social learning? In the research case, the responsibility for implementing on-the-job learning experiences was not clear, and so the implementation was not done or was poorly done.
  4. What does it take in terms of resources, responsibility, and tasking to make experiential and social learning useful? Or, is this just a bridge too far?
  5. The most likely leverage point for on-the-job learning still seems, to me, to be managers. If this is a correct assumption—and really it should be tested—how can we in Learning & Development encourage, support, and resource managers for this role?

Sign Up For Will’s News by Clicking Here

 

 

 

Links of Interest:

 

 

I’ve had the distinct honor of being invited to speak at the Learning Technologies conference in London for three years in a row. This year, I talked about two learning innovations:

  • Performance-Focused Learner Surveys
  • LTEM (The Learning-Transfer Evaluation Model)

It was a hand-raising experience!

Most importantly, they have done a great job capturing my talk on YouTube.

Indeed, although I’ve made some recent improvements in the way I talk about these two learning innovations, the video does an excellent job of capturing some of the main points I’ve been making about the state of learning evaluation and two innovations that are tearing down some of the obstacles that have held us back from doing good evaluation.

Thanks to Stella Collins at Stellar Learning for organizing and facilitating my session!

Special thanks to the brilliant conference organizer and learning-industry influencer Robert Taylor for inviting and supporting me and my work.

Again, click here to see the video of my presentation at Learning Technologies London 2019.

While I was in London a few months ago, where I talked about learning evaluation, I was interviewed by the LearningNews about learning evaluation.

Some of what I said:

  • “Most of us have been doing the same damn thing we’ve always done [in learning evaluation]. On the other hand, there is a breaking of the logjam.”
  • “A lot of us are defaulting to happy sheets, and happy sheets that aren’t effective.”
  • “Do we in L&D have the skills to be able to do evaluation in the first place?…. My short answer is NO WAY!”
  • “We can’t upskill ourselves fast enough [in terms of learning evaluation].

It was a fun interview and LearningNews did a nice job in editing it. Special thanks to Rob Clarke for the interview, organizing, and video work (along with his great team)!!

Click here to see the interview.

I want to thank David Kelly and the eLearning Guild for awarding me the prestigious title of Guild Master.

Guild Masters including an amazing list of folks, including lots of research-to-practice legends like Ruth Clark, Julie Dirksen, Clark Quinn, Jane Bozarth, Karl Kapp, and others who utilize research-based recommendations in their work.

Delighted to be included!

 

 

LTEM, the Learning-Transfer Evaluation Model, was designed as an alternative to the Kirkpatrick-Katzell Four-Level Model of learning evaluation. It was designed specifically to better align learning evaluation with the science of human learning. One way in which LTEM is superior to the Four-Level Model is in the way it highlights gradations of learning outcomes. Where the Four-Level model crammed all “Learning” outcomes into one box (that is, “Level 2”), LTEM separates learning outcomes into Tier-4 Knowledge, Tier-5 Decision-Making Competence, and Tier-6 Task Competence. This simple, yet incredibly powerful categorization, changes everything in terms of learning evaluation. First and foremost, it pushes us to go beyond inconsequential knowledge checks in our learning evaluations (and in our learning designs as well). To learn more about how LTEM creates additional benefits, you can click on this link, where you can access the model and a 34-page report for free, compliments of  me, Will Thalheimer, and Work-Learning Research, Inc.

Using LTEM in Credentialing

LTEM can also be used in credentialing—or less formally in specifying the rigorousness of our learning experiences. So for example, if our training course only asks questions about terminology or facts in its assessments, than we can say that the course provides a Tier-4 credential. If our course asks learners to successfully complete a series of scenario-based decisions, we can say that the course provides a Tier-5 credential.

Wow! Think of the power of naming the credential level of our learning experiences. Not only will it give us—and our business stakeholders—a clear sense of the strength of our learning initiatives, but it will drive our instructional designs to meet high standards of effectiveness. It will also begin to set the bar higher. Let’s admit a dirty truth. Too many of our training programs are just warmed-over presentations that do very little to help our learners make critical decisions or improve their actual skills. By focusing on credentialing, we focus on effectiveness!

 

Using LTEM Credentialing at Work-Learning Research

For the last several months, I’ve been developing an online course to teach learning professionals how to transform their learner surveys into Performance-Focused Smile Sheets. As part of this development process, I realized that I needed more than one learning experience—at least one to introduce the topic and one to give people extensive practice. I also wanted to provide people with a credential each time they successfully completed a learning experience. Finally, I wanted to make the credential meaningful. As the LTEM model suggests, attendance is NOT a meaningful benchmark. Neither is learner satisfaction. Nor is knowledge regurgitation.

Suddenly, it struck me. LTEM already provided a perfect delineation for meaningful credentialing. Tier-5 Decision-Making Competence would provide credentialing for the first learning experience. For people to earn their credential they would have to perform successfully in responding to realistic decision-making scenarios. Tier-6 Task Competence would provide credentialing for the second, application-focused learning experience. Additional credentials would only be earned if people could show results at Tier-7 and/or Tier-8 (Transfer to Work Performance and associated Transfer Effects).

 

 

The Gold-Certification Workshop is now ready for enrollment. The Master-Certification Workshop is coming soon! You can keep up to date or enroll now by going to the Work-Learning Academy page.

 

How You Can Use LTEM Credentialing to Assess Learning Experiences that Don’t Use LTEM

LTEM is practically brand new, having only been released to the public a year ago. So, while many organizations are gaining a competitive advantage by exploring its use, most of our learning infrastructure has yet to be transformed. In this transitional period, each of us has to use our wisdom to assess what’s already out there. How about you give it a try?

Two-Day Classroom Workshop — What Tier Credential?

What about a two-day workshop that gives people credit for completing the experience? Where would that be on the LTEM framework?

Here’s a graphic to help. Or you can access the full model by clicking here.

The two-day workshop would be credentialed at a Tier-1 level, signifying that the experience credentials learners by measuring their attendance or completion.

Two-Day Classroom Workshop with Posttest — What Tier Credential?

What if the same two-day workshop also added a test focused on whether the learners understood the content—and provided the test a week after the program. Note that in the LTEM model, credentialing is encouraged at Tiers 4, 5, and 6 to include assessments that show learners are able to remember, not just comprehend in the short term.

If the workshop added this posttest, we’d credential it at Tier-4, Knowledge Retention.

Half-Day Online Program with Performance-Focused Smile Sheet — What Tier Credential?

What if there was a half day workshop that used one of my Performance-Focused Smile Sheets to evaluate success. At what Tier would this be credentialed?

It would be credentialed at Tier-3, or Tier-3A if we wanted to delineate between learner surveys that assess learning effectiveness and those that don’t.

Three-Session Online Program with Traditional Smile Sheet — What Tier Credential?

This format—using three 90-minute sessions with a traditional smile sheet—is the most common form of credentialing in the workplace learning industry right now. Go look around at those that are providing credentials. They are providing credentials using relatively short presentations and a smile sheet at the end. If this is what they provide, what credentialing Tier do they deserve? Tier-3 or Tier-3B! That’s right! That’s it. They only tell us that learners are satisfied with the learning experience. They don’t tell us whether they can make important decisions or whether they can utilize new skills.

What is this credential really worth?

You can decide for yourself, but I think it could be worth more, if only those making the money provided credentialing at Tier-5, Tier-6, and beyond.

With LTEM we can begin to demand more!

 

Work-Learning Research and Will Thalheimer can Help!

People tell me I need to stop giving stuff away for free, or at least I ought to be more proactive in seeking customers. So, this is a reminder that I am available to help you improve your learning and learning evaluation strategies and tactics. Please reach out to me at my nifty contact form by clicking here.