I don't usually delve into K-12 classroom-based research to any great extent. However, as my daughter's elementary school is starting a "redesign" process—and I heard arguments on both sides of the multigrade-classroom issue—I thought perhaps I'd give a quick look at the research available.

I had hoped that this would be a quick review, where I would find one or two definitive research reviews in scientific refereed journals, but unfortunately, the research base is rather frail and unclear.

Still, I think the following review does provide some wisdom about how to think about multiage classrooms.

————————————————-

Extent of Review             

I did a moderately quick review—not an exhaustive review—of a couple dozen recent research articles on multigrade classrooms.

This review was conducted in September and October of 2010.

————————————————-

Major Conclusions (See More Specific Recommendations Further Below):

The research, although being too scant and too difficult to interpret to make definitive recommendations, generally suggests that multigrade classroom approaches are not likely to produce results that differ substantially from single-grade classrooms.

Specifically, it is likely that the quality of the learning methods utilized and the teacher’s performance in the classroom makes more of difference than whether a multigrade or single-grade approach is utilized.

Multigrade teaching is generally considered more difficult and onerous than single-grade teaching.

If a multigrade approach is utilized, then it should be utilized with due diligence—providing teacher support and development, utilizing team teaching, encouraging many diagnostic opportunities (so that learning can be tailored to learner’s current levels), and so forth.

————————————————- 

Context:             

Researchers and practitioners use many labels for multigrade education, including the following: multigrade, multiage, mixed-age, vertical grouping, combination, composite, double-grade, split-grade, dual-age, hyphenated, nongraded, etc. These terms are often confused, redundant, etc., making researching and thinking about related issues difficult. In this report, I will use the following terms:

  • Single-grade classrooms are comprised of one grade.
  • Multigrade classrooms are comprised of two (or more) grades.
    • Combination Multigrade classrooms are multigrade classrooms that are utilized for logistical reasons.
    • Pedagogic Multigrade classrooms are multigrade classrooms that are utilized for pedagogic (learning-related) reasons.

Sometimes multigrade classrooms are created because of logistical reasons such as declining or uneven enrollments (Veenman, 1996; Burns & Mason, 2002; Mulryan-Kyne, 2007).

Sometimes multigrade classrooms are created for philosophical and pedagogical rationale. There are strong advocacy groups for pedagogic multigrade classrooms. Some of this advocacy is at odds with the research—in the sense that they claim overwhelming benefits for multigrade classrooms when the research is more balanced and uncertain (I noticed this myself in reviewing the research, but also see Burns & Mason, 1997).

It can be helpful to view classes on a continuum from single-grade classes, to combination multigrade classes, to pedagogic multigrade classes (See Mason & Burns, 1997). While many different arrangements are possible, we can talk in generalities as follows:

  • Single-grade classes have students only from one grade.
  • Combination multigrade classes have students from two (or more) grades and one teacher who teaches them.
  • Pedagogic multigrade classes have students from two (or more) grades but are taught by a team of teachers. Pedagogic multigrade classes also tend to be more focused on providing learners individually-tailored learning content than either single-grade or combination multigrade classes.

Teachers in combination multigrade classes are less likely to receive the support and structure they need than teachers in pedagogic multigrade classes (Mason & Burns, 1997).

Multigrade classrooms are not aberrations, but are used throughout the world, and are likely to continue and grow in use in the future (Mulryan-Kyne, 2007).

————————————————-

Quality of Available Research:     

Unfortunately, it seems that much of the available research is tainted by methodological weaknesses. Specifically, it appears that more experienced teachers tend to teach multi-age classrooms—so that when advantages appear in the educational results, those advantages may be due to teacher experience as opposed to the multi-age classroom itself. In addition, more affluent students are the ones who tend to be taught in multigrade classrooms—so that when research results show advantages, those could be due to socio-economic and educational advantages of parents as opposed to the multigrade classrooms.

Quotes from the Research:

“Effective research in the area of multiage education is still in its infancy.” From Kinsey (2001).

“The literature on multigrade teaching is relatively sparse, some of it anecdotal in nature and/or of poor quality.” From Mulryan-Kyne (2007).

“There is considerable evidence that principals, in an effort to reduce the burden on multigrade teachers, place more able, more independent, and more cooperative students in multigrade classes.” From Mason and Burns (1996). Note: This is relevant in that good results in comparison to single-age classrooms may be due to having better students in the class.

No studies in which students were randomly assigned to experimental and control groups were found.” From Veenman (1995). Note: While such random assignment is the gold standard in research, it is difficult to implement in the classroom.

“Because of the lack of distinction between combination and multiage/nongraded classes and the omission of important studies and methodological considerations, it appears that researchers have drawn overly optimistic and erroneous conclusions about the effects of combination classes.” From Mason & Burns (1997).

Conclusion based on Quality of Available Research:

These methodological weaknesses make firm conclusions difficult. Tentative conclusions are still possible.

————————————————-

Teaching is Often More Difficult in Multigrade Classrooms:            

It appears that teaching in a multigrade classroom is more difficult than teaching in a single-age classroom. This conclusion comes from those on all sides of the debate, so it is a fairly strong conclusion.

Quotes from the Research:

“It is commonly stated in the literature that multigrade teaching is more difficult than single grade teaching” From Mulryan-Kyne (2007).

“Multigrade classes hold instructional potential for some, but they are potentially onerous for most. Indeed, we argue that multigrade classes lead to a negative instructional effect and that they increase teachers' stress and may jeopardize teachers' motivation and commitment to teaching.” From Mason and Burns (1996).

“Most teachers, when asked about their feelings toward and organizational and teaching strategies for combination classes, responded negatively and preferred not to teach them.” From Mason and Burns (1995).

“The professional knowledge and skills that are relevant and necessary to teaching effectively in single-grade contexts are also relevant and necessary for effective multigrade teaching… However, many of these skills need heightened emphasis in the context of the preparation of teachers for multigrade teaching.” From Mulryan-Kyne (2007).

“[Teachers] report that these classes require more planning, are more difficult to teach, and diminish instruction and curriculum coverage.” From Mason & Burns (1997).

“We concluded that the difficulties teachers face in multigrade classes are centered around five problem areas: (a) the efficient use of instructional time, (b) the design of effective instruction, (c) classroom management, (d) the organization of independent practice or learning, and (e) the formulation of clear and collectively agreed-upon goals for making the multigrade school work.” From Veenman (1996).

“…multigrade teachers pressed for instructional time and the mastery of basic skills (e.g., reading, writing, and mathematics) might neglect science, social studies, and other subjects, which would lead to negative achievement effects in these areas.” From Mason & Burns (1996).

————————————————-

Academic Achievement Results from Multigrade Classrooms:

In the two most credible recent reviews of the research, the findings have shown no differences in academic achievement between multigrade classrooms and single-grade classrooms (Veenman, 1995; Mason & Burns, 1997). However, because of the likelihood that this research is tainted in having better students and teachers in multi-grade classrooms, it is possible that multigrade classrooms produce “at least small negative effects” (Mason & Burns, 1997).

Separating the results for combination multigrade classrooms and pedagogical multigrade classrooms is difficult because researchers haven’t always noted this difference. In Veenman’s (1995) review, both combination and pedagogic multigrade classrooms produced non-significant results—in other words, they were found to produce the same academic results as single-grade classrooms. However, as Mason and Burns (1996) pointed out, Veenman didn’t take into account some important potential biases. Mason and Burns write: “We conclude that multigrade classes have at least a small negative effect on achievement as well as potentially negative effects on teacher motivation.”

Unfortunately, in Mason and Burns (1997) follow-up research review, they only focused on combination multigrade classrooms—NOT pedagogic multigrade classrooms. Their results on combination multigrade classrooms were consistent with Veenman’s in finding no statistical differences, but they warned that “all things being equal, combination classes have at least small negative effects.” Such a negative outcome was found in a 2008 study of combination multigrade classrooms in California (Sims, 2008).

Conclusions on Academic Achievement:

The research is not definitive because there are factors that have not really been teased out as of yet.

In terms of academic results, the tentative conclusion is that multigrade classes on average are probably no better and no worse than single-grade classes, but if they are different from single-grade classes, they may be slightly worse.

————————————————-

Social/Emotional Results from Multi-age Classrooms:

Veenman’s (1995) research found that a majority of combination multigrade classes performed no better than single-grade classes in terms of attitudes towards school, self-concept, and personal and social adjustment.  Mason and Burns (1997) examination of combination multigrade classes found similar non-significant affective benefits.

For pedagogic multigrade classes, however, Veenman (1996) found a “very small” effect, showing slight benefits for pedagogic multigrade classes in terms of attitudes towards school, self-concept, and personal and social adjustment. Again, because of the potential biasing effects in terms of student and teacher selection, these results could be due to bias instead of any benefits for pedagogic multigrade classes.

Conclusions on Social/Emotional Results:

The research is not definitive because there are factors that have not really been teased out as of yet.

In terms of social/emotional results, the tentative conclusion is that multigrade classes on average are probably no better and no worse than single-grade classes. If there are very small benefits, they might be obtained in well-designed pedagogic multigrade classrooms—as opposed to logistically-driven combination multigrade classrooms.

————————————————-

Do some Students Benefit More than Others from Multigrade Classrooms?             

Although this was beyond the scope of my review—I did come across one recent study that dealt specifically with the question of how different types of students might be affected by a multigrade approach (Ong, Allison, & Haladyna, 2000).

First, it should be noted that in this one study, multigrade classrooms tended to outperform single-grade classrooms for all students, regardless of their background. Given that as background, the study found the following:

  • It found no differences between boys and girls.
  • It found that regular students seemed to benefit more from multigrade classrooms than disadvantaged students (that is, Title 1 students).
  • It found that overall non-Hispanic students seemed to benefit more from multigrade classrooms than Hispanic students.

Caveat: This was just one study and should be evaluated with caution.

————————————————-

Overall Recommendations Regarding Multigrade, Multiage, Combination Classrooms:         

The scarcity, frailty, and equivocation in the research make strong recommendations impossible. Instead, I offer these tentative ideas for consideration:

1. It is likely that the quality of the learning methods utilized and the teacher’s performance in the classroom makes more of difference than whether a multigrade or single-grade approach is utilized. Given this, it appears that choosing a multigrade approach would be acceptable, though the following points should also be kept in mind.

2. If a multigrade approach is utilized, then it should be utilized with due diligence—providing teacher support and development, utilizing team teaching, encouraging many diagnostic opportunities (so that learning can be tailored to learner’s current levels*), and so forth.

* Recent research indicates that teachers, tutors, and other learning professionals tend not to be very good at providing instructional explanations at learner’s appropriate levels—but that they can improve on this by specifically being informed of their learners’ level of understanding (Wittwer, Nückles, & Renkl, 2010).

3. Multigrade teaching seems to offer the possibility for alternative methods of learning, including group work targeted to different developmental levels. On the other hand, multigrade teaching by itself is no guarantee of good teaching methodology.

4. It should be recognized that multigrade teaching probably has costs associated with its use. Specifically, teachers may have to invest more effort and care in the process and may have a tendency to tradeoff other desirable educational goals.

5. In some forms of multigrade teaching, especially those that utilize team teaching, teachers have to learn the skill levels (in each discipline) of many more students. Because one of the most important aspects of teaching is providing learners with just the right level of instruction, this may cause students to be underserved at first as the teacher learns their students' skill levels. One way to ameliorate this problem is for students entering a multigrade cycle to be diagnosed through testing, problem solving, and other performance metrics early on. This “slow-start” issue has a flip side—because multigrade classes stretch into two (or more) years, students in subsequent years will probably experience accelerated learning due to deeper understanding of each student by teachers and hence better instructional scaffolding.

6. Teachers with less experience may be especially unprepared for multigrade teaching. With experience, teachers learn how to automate aspects of their classroom performance so that they can utilize their limited working memory capacity to focus on supporting their learners in learning (for example, differentiating their instruction, etc.). Newer teachers will be unlikely to provide the necessary instructional scaffolding to perform at a high level.

7. Students who have behavioral problems or who are less competent in learning may make the task of multigrade teaching more difficult. This may suggest that extra effort and guidance in the early grades is warranted or that a multigrade approach should be delayed until a time when most students are ready to engage fully in the process.

————————————————-

Research References:

Burns, R. & Mason, D. (2002). Class composition and student achievement in elementary schools. American Educational Research Journal, 39 (1), 207-233.

Burns, R. B., & Mason, D. A. (1998). Class formation and composition in elementary schools. American Educational Research Journal, 35(4), 739-772.

Kinsey, S. (2001). Multiage Grouping and Academic Achievement. ERIC Digest, January 2001.

Mason, D. A., & Burns, R. B. (1995). Teachers' views of combination classes. Journal of Educational Research, 89(1), 36-45.

Mason, D. A., & Burns, R. B. (1996). "Simply no worse and simply no better" may simply be wrong: A critique of Veenman's conclusion about multigrade classes. Review of Educational Research, 66(3), 307-322.

Mason, D., & Burns, R. (1997). Reassessing the effects of combination classes. Educational Research and Evaluation, 3(1), 1–53.

Mulryan-Kyne, C. (2007). The preparation of teachers for multigrade teaching. Teaching and Teacher Education, 23(4), 501-514.

Ong, W., Allison, J., & Haladyna, T. M. (2000). Student achievement of 3rd-graders in comparable single-age and multiage classrooms. Journal of Research in Childhood Education, 14(2), 205-215.

Sims, D. (2008). A strategic response to class size reduction: Combination classes and student achievement in California. Journal of Policy Analysis and Management, 27(3), 457-478.

Veenman, S. (1995). Cognitive and noncognitive effects of multigrade and multi-age classes: A best-evidence synthesis. Review of Educational Research, 65, 319-381.

Veenman, S. (1996). Effects of multigrade and multi-age classes reconsidered. Review of Educational Research, 66(3), 323-340.

Wittwer, J., Nückles, M., & Renkl, A. (2010). Using a diagnosis-based approach to individualize instructional explanations in computer-mediated communication. Educational Psychology Review, 22(1), 9-23.

————————————————-

Some of the articles reviewed but not cited:

Song, R.  Spradlin, T. E., & and Plucker, J. A. (2009). The Advantages and Disadvantages of Multiage Classrooms in the Era of NCLB Accountability. Education Policy Brief, 7, 1-7. Published by the Center for Evaluation and Education Policy of Indiana University.

Why not cited:
This article is not from refereed journal so it may not have been fully vetted. Also, there are hints of bias in the article, for example, (a) providing invited proponents of multiage classrooms to describe its value, without providing a similar counterpoint, and (b) talking about the lack of good research but then plowing ahead with a list of the “benefits (perceived and real)” of multiage classrooms—so in a real sense the article ignores the research by plowing ahead to benefits. Still, I wouldn’t discount this article completely. Its research review seems good when it is limited to reviewing the actual research. Also, it makes recommendations that are mostly consistent with the findings contained in this review, so I can’t dismiss their conclusions. The bottom line is that this review does not add anything to a review of the research, and, because it is not vetted in a refereed journal and appears slightly biased, I think it safer not to cite it as separate evidence.

Hoffman, J. (2003). Multiage teachers beliefs and practices. Journal of Research in Childhood Education, 18, 5-17.

Why not cited:
This article looked at only 4 teachers, all who had chosen to be teaching in multiage classrooms. It was really a descriptive research project and did not look at actual learning outcomes. No single-grade teachers were examined so we don’t really know how different results with single-grade teaching might be.

Linley, L. (1999). Multi-Age Classes and High Ability Students. Review of Educational Research, 69, 187-212.

Why not cited:
This article was a review of other reviews, not a review of research studies itself. Also, it focused only on high-ability students.

Gerard, M. (2005). Bridging the gap: Understanding young children’s thinking in multiage groups. Journal of Research in Childhood Education, 19, 243-250.

Why not cited:
This article utilized an incredibly biased research design. It took one classroom at one school that used multiage grouping and compared it on standardized exams with the national average.

Mariano, S., & Kirby N. (2009). Achievement of Students in Multigrade Classrooms Evidence from the Los Angeles Unified School District. From the RAND Education Working Paper Series (WR-685-IES).

Why not cited:
This article is not from refereed journal so it may not have been fully vetted. It also uses somewhat opaque statistical methods to estimate findings, not looking directly at actual comparisons between multigrade and single-grade classrooms. I must admit that I don’t fully understand all the statistical employments utilized (for example, “doubly robust regression,” “non-parametric generalized boosting,” “the Kolmogorov-Smirnov (K-S) statistic”). Thus, I may be missing the full implications of the research employed.

I don’t usually delve into K-12 classroom-based research to any great extent. However, as my daughter’s elementary school is starting a “redesign” process—and I heard arguments on both sides of the multigrade-classroom issue—I thought perhaps I’d give a quick look at the research available.

I had hoped that this would be a quick review, where I would find one or two definitive research reviews in scientific refereed journals, but unfortunately, the research base is rather frail and unclear.

Still, I think the following review does provide some wisdom about how to think about multiage classrooms.

————————————————-

Extent of Review 

I did a moderately quick review—not an exhaustive review—of a couple dozen recent research articles on multigrade classrooms.

This review was conducted in September and October of 2010.

 

————————————————-

Major Conclusions (See More Specific Recommendations Further Below):

The research, although being too scant and too difficult to interpret to make definitive recommendations, generally suggests that multigrade classroom approaches are not likely to produce results that differ substantially from single-grade classrooms.

Specifically, it is likely that the quality of the learning methods utilized and the teacher’s performance in the classroom makes more of difference than whether a multigrade or single-grade approach is utilized.

Multigrade teaching is generally considered more difficult and onerous than single-grade teaching.

If a multigrade approach is utilized, then it should be utilized with due diligence—providing teacher support and development, utilizing team teaching, encouraging many diagnostic opportunities (so that learning can be tailored to learner’s current levels), and so forth.

————————————————- 

Context:    

Researchers and practitioners use many labels for multigrade education, including the following: multigrade, multiage, mixed-age, vertical grouping, combination, composite, double-grade, split-grade, dual-age, hyphenated, nongraded, etc. These terms are often confused, redundant, etc., making researching and thinking about related issues difficult. In this report, I will use the following terms:

  • Single-grade classrooms are comprised of one grade.
  • Multigrade classrooms are comprised of two (or more) grades.
    • Combination Multigrade classrooms are multigrade classrooms that are utilized for logistical reasons.
    • Pedagogic Multigrade classrooms are multigrade classrooms that are utilized for pedagogic (learning-related) reasons.

Sometimes multigrade classrooms are created because of logistical reasons such as declining or uneven enrollments (Veenman, 1996; Burns & Mason, 2002; Mulryan-Kyne, 2007).

Sometimes multigrade classrooms are created for philosophical and pedagogical rationale. There are strong advocacy groups for pedagogic multigrade classrooms. Some of this advocacy is at odds with the research—in the sense that they claim overwhelming benefits for multigrade classrooms when the research is more balanced and uncertain (I noticed this myself in reviewing the research, but also see Burns & Mason, 1997).

It can be helpful to view classes on a continuum from single-grade classes, to combination multigrade classes, to pedagogic multigrade classes (See Mason & Burns, 1997). While many different arrangements are possible, we can talk in generalities as follows:

  • Single-grade classes have students only from one grade.
  • Combination multigrade classes have students from two (or more) grades and one teacher who teaches them.
  • Pedagogic multigrade classes have students from two (or more) grades but are taught by a team of teachers. Pedagogic multigrade classes also tend to be more focused on providing learners individually-tailored learning content than either single-grade or combination multigrade classes.

Teachers in combination multigrade classes are less likely to receive the support and structure they need than teachers in pedagogic multigrade classes (Mason & Burns, 1997).

Multigrade classrooms are not aberrations, but are used throughout the world, and are likely to continue and grow in use in the future (Mulryan-Kyne, 2007).

————————————————-

Quality of Available Research:

Unfortunately, it seems that much of the available research is tainted by methodological weaknesses. Specifically, it appears that more experienced teachers tend to teach multi-age classrooms—so that when advantages appear in the educational results, those advantages may be due to teacher experience as opposed to the multi-age classroom itself. In addition, more affluent students are the ones who tend to be taught in multigrade classrooms—so that when research results show advantages, those could be due to socio-economic and educational advantages of parents as opposed to the multigrade classrooms.

Quotes from the Research:

“Effective research in the area of multiage education is still in its infancy.” From Kinsey (2001).

“The literature on multigrade teaching is relatively sparse, some of it anecdotal in nature and/or of poor quality.” From Mulryan-Kyne (2007).

“There is considerable evidence that principals, in an effort to reduce the burden on multigrade teachers, place more able, more independent, and more cooperative students in multigrade classes.” From Mason and Burns (1996). Note: This is relevant in that good results in comparison to single-age classrooms may be due to having better students in the class.

No studies in which students were randomly assigned to experimental and control groups were found.” From Veenman (1995). Note: While such random assignment is the gold standard in research, it is difficult to implement in the classroom.

“Because of the lack of distinction between combination and multiage/nongraded classes and the omission of important studies and methodological considerations, it appears that researchers have drawn overly optimistic and erroneous conclusions about the effects of combination classes.” From Mason & Burns (1997).

Conclusion based on Quality of Available Research:

These methodological weaknesses make firm conclusions difficult. Tentative conclusions are still possible.

————————————————-

Teaching is Often More Difficult in Multigrade Classrooms:  

It appears that teaching in a multigrade classroom is more difficult than teaching in a single-age classroom. This conclusion comes from those on all sides of the debate, so it is a fairly strong conclusion.

Quotes from the Research:

“It is commonly stated in the literature that multigrade teaching is more difficult than single grade teaching” From Mulryan-Kyne (2007).

“Multigrade classes hold instructional potential for some, but they are potentially onerous for most. Indeed, we argue that multigrade classes lead to a negative instructional effect and that they increase teachers’ stress and may jeopardize teachers’ motivation and commitment to teaching.” From Mason and Burns (1996).

“Most teachers, when asked about their feelings toward and organizational and teaching strategies for combination classes, responded negatively and preferred not to teach them.” From Mason and Burns (1995).

“The professional knowledge and skills that are relevant and necessary to teaching effectively in single-grade contexts are also relevant and necessary for effective multigrade teaching… However, many of these skills need heightened emphasis in the context of the preparation of teachers for multigrade teaching.” From Mulryan-Kyne (2007).

“[Teachers] report that these classes require more planning, are more difficult to teach, and diminish instruction and curriculum coverage.” From Mason & Burns (1997).

“We concluded that the difficulties teachers face in multigrade classes are centered around five problem areas: (a) the efficient use of instructional time, (b) the design of effective instruction, (c) classroom management, (d) the organization of independent practice or learning, and (e) the formulation of clear and collectively agreed-upon goals for making the multigrade school work.” From Veenman (1996).

“…multigrade teachers pressed for instructional time and the mastery of basic skills (e.g., reading, writing, and mathematics) might neglect science, social studies, and other subjects, which would lead to negative achievement effects in these areas.” From Mason & Burns (1996).

————————————————-

Academic Achievement Results from Multigrade Classrooms:

In the two most credible recent reviews of the research, the findings have shown no differences in academic achievement between multigrade classrooms and single-grade classrooms (Veenman, 1995; Mason & Burns, 1997). However, because of the likelihood that this research is tainted in having better students and teachers in multi-grade classrooms, it is possible that multigrade classrooms produce “at least small negative effects” (Mason & Burns, 1997).

Separating the results for combination multigrade classrooms and pedagogical multigrade classrooms is difficult because researchers haven’t always noted this difference. In Veenman’s (1995) review, both combination and pedagogic multigrade classrooms produced non-significant results—in other words, they were found to produce the same academic results as single-grade classrooms. However, as Mason and Burns (1996) pointed out, Veenman didn’t take into account some important potential biases. Mason and Burns write: “We conclude that multigrade classes have at least a small negative effect on achievement as well as potentially negative effects on teacher motivation.”

Unfortunately, in Mason and Burns (1997) follow-up research review, they only focused on combination multigrade classrooms—NOT pedagogic multigrade classrooms. Their results on combination multigrade classrooms were consistent with Veenman’s in finding no statistical differences, but they warned that “all things being equal, combination classes have at least small negative effects.” Such a negative outcome was found in a 2008 study of combination multigrade classrooms in California (Sims, 2008).

Conclusions on Academic Achievement:

The research is not definitive because there are factors that have not really been teased out as of yet.

In terms of academic results, the tentative conclusion is that multigrade classes on average are probably no better and no worse than single-grade classes, but if they are different from single-grade classes, they may be slightly worse.

————————————————-

Social/Emotional Results from Multi-age Classrooms:

Veenman’s (1995) research found that a majority of combination multigrade classes performed no better than single-grade classes in terms of attitudes towards school, self-concept, and personal and social adjustment.  Mason and Burns (1997) examination of combination multigrade classes found similar non-significant affective benefits.

For pedagogic multigrade classes, however, Veenman (1996) found a “very small” effect, showing slight benefits for pedagogic multigrade classes in terms of attitudes towards school, self-concept, and personal and social adjustment. Again, because of the potential biasing effects in terms of student and teacher selection, these results could be due to bias instead of any benefits for pedagogic multigrade classes.

Conclusions on Social/Emotional Results:

The research is not definitive because there are factors that have not really been teased out as of yet.

In terms of social/emotional results, the tentative conclusion is that multigrade classes on average are probably no better and no worse than single-grade classes. If there are very small benefits, they might be obtained in well-designed pedagogic multigrade classrooms—as opposed to logistically-driven combination multigrade classrooms.

————————————————-

Do some Students Benefit More than Others from Multigrade Classrooms? 

Although this was beyond the scope of my review—I did come across one recent study that dealt specifically with the question of how different types of students might be affected by a multigrade approach (Ong, Allison, & Haladyna, 2000).

First, it should be noted that in this one study, multigrade classrooms tended to outperform single-grade classrooms for all students, regardless of their background. Given that as background, the study found the following:

  • It found no differences between boys and girls.
  • It found that regular students seemed to benefit more from multigrade classrooms than disadvantaged students (that is, Title 1 students).
  • It found that overall non-Hispanic students seemed to benefit more from multigrade classrooms than Hispanic students.

Caveat: This was just one study and should be evaluated with caution.

————————————————-

Overall Recommendations Regarding Multigrade, Multiage, Combination Classrooms:

The scarcity, frailty, and equivocation in the research make strong recommendations impossible. Instead, I offer these tentative ideas for consideration:

1. It is likely that the quality of the learning methods utilized and the teacher’s performance in the classroom makes more of difference than whether a multigrade or single-grade approach is utilized. Given this, it appears that choosing a multigrade approach would be acceptable, though the following points should also be kept in mind.

2. If a multigrade approach is utilized, then it should be utilized with due diligence—providing teacher support and development, utilizing team teaching, encouraging many diagnostic opportunities (so that learning can be tailored to learner’s current levels*), and so forth.

* Recent research indicates that teachers, tutors, and other learning professionals tend not to be very good at providing instructional explanations at learner’s appropriate levels—but that they can improve on this by specifically being informed of their learners’ level of understanding (Wittwer, Nückles, & Renkl, 2010).

3. Multigrade teaching seems to offer the possibility for alternative methods of learning, including group work targeted to different developmental levels. On the other hand, multigrade teaching by itself is no guarantee of good teaching methodology.

4. It should be recognized that multigrade teaching probably has costs associated with its use. Specifically, teachers may have to invest more effort and care in the process and may have a tendency to tradeoff other desirable educational goals.

5. In some forms of multigrade teaching, especially those that utilize team teaching, teachers have to learn the skill levels (in each discipline) of many more students. Because one of the most important aspects of teaching is providing learners with just the right level of instruction, this may cause students to be underserved at first as the teacher learns their students’ skill levels. One way to ameliorate this problem is for students entering a multigrade cycle to be diagnosed through testing, problem solving, and other performance metrics early on. This “slow-start” issue has a flip side—because multigrade classes stretch into two (or more) years, students in subsequent years will probably experience accelerated learning due to deeper understanding of each student by teachers and hence better instructional scaffolding.

6. Teachers with less experience may be especially unprepared for multigrade teaching. With experience, teachers learn how to automate aspects of their classroom performance so that they can utilize their limited working memory capacity to focus on supporting their learners in learning (for example, differentiating their instruction, etc.). Newer teachers will be unlikely to provide the necessary instructional scaffolding to perform at a high level.

7. Students who have behavioral problems or who are less competent in learning may make the task of multigrade teaching more difficult. This may suggest that extra effort and guidance in the early grades is warranted or that a multigrade approach should be delayed until a time when most students are ready to engage fully in the process.

————————————————-

Research References:

Burns, R. & Mason, D. (2002). Class composition and student achievement in elementary schools. American Educational Research Journal, 39 (1), 207-233.

Burns, R. B., & Mason, D. A. (1998). Class formation and composition in elementary schools. American Educational Research Journal, 35(4), 739-772.

Kinsey, S. (2001). Multiage Grouping and Academic Achievement. ERIC Digest, January 2001.

Mason, D. A., & Burns, R. B. (1995). Teachers’ views of combination classes. Journal of Educational Research, 89(1), 36-45.

Mason, D. A., & Burns, R. B. (1996). “Simply no worse and simply no better” may simply be wrong: A critique of Veenman’s conclusion about multigrade classes. Review of Educational Research, 66(3), 307-322.

Mason, D., & Burns, R. (1997). Reassessing the effects of combination classes. Educational Research and Evaluation, 3(1), 1–53.

Mulryan-Kyne, C. (2007). The preparation of teachers for multigrade teaching. Teaching and Teacher Education, 23(4), 501-514.

Ong, W., Allison, J., & Haladyna, T. M. (2000). Student achievement of 3rd-graders in comparable single-age and multiage classrooms. Journal of Research in Childhood Education, 14(2), 205-215.

Sims, D. (2008). A strategic response to class size reduction: Combination classes and student achievement in California. Journal of Policy Analysis and Management, 27(3), 457-478.

Veenman, S. (1995). Cognitive and noncognitive effects of multigrade and multi-age classes: A best-evidence synthesis. Review of Educational Research, 65, 319-381.

Veenman, S. (1996). Effects of multigrade and multi-age classes reconsidered. Review of Educational Research, 66(3), 323-340.

Wittwer, J., Nückles, M., & Renkl, A. (2010). Using a diagnosis-based approach to individualize instructional explanations in computer-mediated communication. Educational Psychology Review, 22(1), 9-23.

————————————————-

Some of the articles reviewed but not cited:

Song, R.  Spradlin, T. E., & and Plucker, J. A. (2009). The Advantages and Disadvantages of Multiage Classrooms in the Era of NCLB Accountability. Education Policy Brief, 7, 1-7. Published by the Center for Evaluation and Education Policy of Indiana University.

Why not cited:
This article is not from refereed journal so it may not have been fully vetted. Also, there are hints of bias in the article, for example, (a) providing invited proponents of multiage classrooms to describe its value, without providing a similar counterpoint, and (b) talking about the lack of good research but then plowing ahead with a list of the “benefits (perceived and real)” of multiage classrooms—so in a real sense the article ignores the research by plowing ahead to benefits. Still, I wouldn’t discount this article completely. Its research review seems good when it is limited to reviewing the actual research. Also, it makes recommendations that are mostly consistent with the findings contained in this review, so I can’t dismiss their conclusions. The bottom line is that this review does not add anything to a review of the research, and, because it is not vetted in a refereed journal and appears slightly biased, I think it safer not to cite it as separate evidence.

Hoffman, J. (2003). Multiage teachers beliefs and practices. Journal of Research in Childhood Education, 18, 5-17.

Why not cited:
This article looked at only 4 teachers, all who had chosen to be teaching in multiage classrooms. It was really a descriptive research project and did not look at actual learning outcomes. No single-grade teachers were examined so we don’t really know how different results with single-grade teaching might be.

Linley, L. (1999). Multi-Age Classes and High Ability Students. Review of Educational Research, 69, 187-212.

Why not cited:
This article was a review of other reviews, not a review of research studies itself. Also, it focused only on high-ability students.

Gerard, M. (2005). Bridging the gap: Understanding young children’s thinking in multiage groups. Journal of Research in Childhood Education, 19, 243-250.

Why not cited:
This article utilized an incredibly biased research design. It took one classroom at one school that used multiage grouping and compared it on standardized exams with the national average.

Mariano, S., & Kirby N. (2009). Achievement of Students in Multigrade Classrooms Evidence from the Los Angeles Unified School District. From the RAND Education Working Paper Series (WR-685-IES).

Why not cited:
This article is not from refereed journal so it may not have been fully vetted. It also uses somewhat opaque statistical methods to estimate findings, not looking directly at actual comparisons between multigrade and single-grade classrooms. I must admit that I don’t fully understand all the statistical employments utilized (for example, “doubly robust regression,” “non-parametric generalized boosting,” “the Kolmogorov-Smirnov (K-S) statistic”). Thus, I may be missing the full implications of the research employed.

Research Benchmarking

Research Benchmarking is the process by which your learning interventions are benchmarked against research-based best practices.

While random-assignment between-group research is likely to be too expensive and time-consuming for most of us, and benchmarking our work against other industry players is likely to push us toward mediocrity, Research Benchmarking offers a potent alternative.

Learning programs are examined to determine how well they (a) create understanding, (b) support long-term remembering (minimize forgetting), and (c) motivate on-the-job performance. They will be research-benchmarked against the 12 most decisive factors in learning design.

If you’d like to discuss research benchmarking further, contact me, Dr. Will Thalheimer, at 1-888-579-9814 or email me.

 

Sign Up for Will Thalheimer’s Mailing List

If you really want to know what Will Thalheimer’s up to—if you want to be the first to know when new research and information is revealed, if you want to be the first to get involved in Will’s Book Project—sign up here.

Our Policies:

  • We Never Share Your Info!!
  • You Can Unsubscribe at Any Time

 


Representative Research Citations

The following list of citations from the world’s best scientific refeered journals and publications are representative of the research articles that inform our work. The researchers cited below—although often unknown to practitioners in the learning-and-performance field—are heroes on the front line of learning and educational improvement.


Aggleton, J. P., & Waskett, L. (1999). The ability of odours to serve as state-dependent cues for real-world memories: Can Viking smells aid the recall of Viking experiences? British Journal of Psychology, 90, 1-7.

Allen, G. A., Mahler, W. A., & Estes, W. K. (1969). Effects of recall tests on long-term retention of paired associates. Journal of Verbal Learning and Verbal Behavior, 8, 463-470.

Alliger, Tannenbaum, Bennett, Traver, & Shotland (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50, 341-357.

Anderson, R. C. (1982). Allocation of attention during reading. In A. Flammer & W. Kintsch (Eds.), Discourse processing (pp. 292-305). New York: North-Holland.

Anderson, R. C., & Biddle, W. B. (1975). On asking people questions about what they are reading. In G. H. Bower (Ed.) The psychology of learning and motivation: Advances in research and theory (Vol. 9). New York: Academic Press.

Anderson, R. C., Kulhavy, R. W., & Andre, T. (1971). Feedback procedures in programmed instruction. Journal of Educational Psychology, 62, 148-156.

Anderson, R. C., Kulhavy, R. W., & Andre, T. (1972). Conditions under which feedback facilitates learning from programmed lessons. Journal of Educational Psychology, 63, 186-188.

Antrobus, J. S. & Bertini, M. (1992). The neuropsychology of sleep and dreaming. Hillsdale, NJ: Erlbaum.

Armstrong, T. R. (1970). Feedback and perceptual-motor skill learning: A review of information feedback and manual guidance training techniques. Technical Report No. 25, Human Performance Center, University of Michigan.

Ash, P. (1950). The relative effectiveness of massed versus spaced film presentation. Journal of Educational Psychology, 41, 19-30.

Austin, S. D. M. (1921). A study in logical memory. American Journal of Psychology, 32, 370-403.

Ausubel, D. P. (1966). Early versus delayed review in meaningful learning. Psychology in the Schools, 3, 195-198.

Baddeley, A. D. (1999). Essentials of human memory. East Sussex, UK: Psychology Press.

Bahrick, H. P. (1979). Maintenance of knowledge: Questions about memory we forgot to ask. Journal of Experimental Psychology: General, 108, 296-308.

Bahrick, H. P., & Hall, L. K. (1991). Preventive and corrective maintenance of access to knowledge. Applied Cognitive Psychology, 5, 1-18.

Bahrick, H. P., & Phelps, E. (1987). Retention of Spanish vocabulary over 8 years. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13, 344-349.

Bahrick, H. P., & Phelps, E. (1988). The maintenance of marginal knowledge. In U. Neisser & E. Winograd (Eds.), Remembering reconsidered: Ecological and traditional approaches to the study of memory (pp. 178-192). Cambridge, UK: Cambridge University Press.

Bahrick, H. P., Bahrick, L. E., Bahrick, A. S., & Bahrick, P. E. (1993). Maintenance of foreign language vocabulary and the spacing effect. Psychological Science, 4, 316-321.

Balsam, P. D. (1985). The functions of context in learning and performance. In P. D. Balsam & A. Tomie (Eds.) Context and Learning. Hillsdale, NJ: Erlbaum.

Banas, J. A., & Rains, S. A. (2008, November). A meta-analysis of research on inoculation theory. Paper presented at the annual meeting of the National Communication Association, San Diego, CA.

Bangert-Drowns, R. L., Kulik, C-L. C., Kulik, J. A., & Morgan, M. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61, 213-238.

Bangert-Drowns, R. L., Kulik, J. A., & Kulik, C. C. (1988). Effects of frequent classroom testing. Unpublished manuscript, University of Michigan.

Barnett, J. E., & Seefeldt, R. W. (1989). Read something once, why read it again?: Repetitive reading and recall. Journal of Reading Behavior, 21, 351-360.

Baumeister, R. F., Gailliot, M., DeWall, C. N., & Oaten, M. (2006). Self-Regulation and Personality: How Interventions Increase Regulatory Success, and How Depletion Moderates the Effects of Traits on Behavior. Journal of Personality, 74, 1773-1801.

Baumeister, R. F., Vohs, K. D., & Tice, D. M. (2007). The strength model of self-control. Current Directions in Psychological Science, 16, 351-355.

Baumeister, R. F., Vohs, K. D., & Tice, D. M. (2007). The strength model of self-control. Current Directions in Psychological Science, 16, 351-355.

Baumeister, R.F., Heatherton, T.F., & Tice, D.M. (1994). Losing control: How and why people fail at self-regulation. San Diego, CA: Academic Press.

Bekerian, D. A., & Baddeley, A. D. (1980). Saturation advertising and the repetition effect. Journal of Verbal Learning and Verbal Behavior, 19, 17-25.

Bell, H. H., & Waag, W. L. (1998). Evaluating the effectiveness of flight simulators for training combat skills: A review. International Journal of Aviation Psychology, 8, 223-242.

Belton, T., & Priyadharshini, E. (2007). Boredom and schooling: A cross-disciplinary exploration. Cambridge Journal of Education, 37, 579-595. doi: 10.1080/03057640701706227

Berger, J., & Milkman, K. L. (2010). Social Transmission and Viral Culture. Retrieved February 9, 2010 from http://marketing.wharton.upenn.edu/documents/research/Virality.pdf

Berger, S. A., Hall, L. K., & Bahrick, H. P. (1999). Stabilizing access to marginal and submarginal knowledge. Journal of Experimental Psychology: Applied, 5, 438-447.

Bjork, E. L., & Bjork, R. A. (1988). On the adaptive aspects of retrieval failure in autobiographical memory. In M. M. Gruneberg, P. E. Morris, R. N. Sykes (Eds). Practical aspects of memory: Current research and issues (Vol. 1: Memory in everyday life) (pp. 283-288). New York: Wiley.

Bjork, R. A. (1988). Retrieval practice and the maintenance of knowledge. In M. M. Gruneberg, P. E. Morris, R. N. Sykes (Eds.), Practical Aspects of Memory: Current Research and Issues, Vol. 1., Memory in Everyday Life (pp. 396-401). NY: Wiley.

Bjork, R. A., & Richardson-Klavehn, A. (1989). On the puzzling relationship between environmental context and human memory. In C. Izawa (Ed.) Current Issues in Cognitive Processes: The Tulane Floweree Symposium on Cognition (pp. 313-344). Hillsdale, NJ: Erlbaum.

Bloom, K. C., & Shuell, T. J. (1981). Effects of massed and distributed practice on the learning and retention of second-language vocabulary. Journal of Educational Research, 74, 245-248.

Bouton, M. E. (1993). Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychological Bulletin, 114, 80-99.

Bower, G. H., Monteiro, K. P., and Gilligan, S. G. (1978). Emotional mood as context for learning and recall. Journal of Verbal Learning and Verbal Behavior, 17, 573-585.

Bransford, J. D., Franks, J. J., Morris, C. D., & Stein, B. S. (1979). Some general constraints on learning and memory research. In L. S. Cermak & F. I. M. Craik (Eds.), Levels of processing in human memory (pp. 331-354). Hillsdale, NJ: Erlbaum.

Bromage, B. (1981). Effects of repetition on comprehension of technical prose. Unpublished doctoral dissertation, University of California, Department of Psychology, Santa Barbara. As cited in Mayer (1983).

Bromage, B. K., & Mayer, R. E. (1986). Quantitative and qualitative effects of repetition on learning from technical text. Journal of Educational Psychology, 78, 271-278.

Bruce, D., & Bahrick, H. P. (1992). Perceptions of past research. American Psychologist, 47, 319-328.

Bruner, J. S. (1963). The process of education. Cambridge, MA: Harvard University Press.

Bryan, W. L., & Harter, N. (1897). Studies in the physiology and psychology of the telegraphic language. Psychological Review, 4, 27-53.

Burger, J. M., Messian, N., Patel, S., del Prado, A., & Anderson, C. (2004). What a coincidence! The effects of incidental similarity on compliance. Personality and Social Psychology Bulletin, 30, 35-43. doi: 10.1177/0146167203258838

Cain, L. F., & Willey, R. (1939). The effect of spaced learning on the curve of retention. Journal of Experimental Psychology, 25, 209-214.

Carlson, R. A., & Yaure, R. G. (1990). Practice schedules and the use of cognitive skills in problem solving. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 484-496.

Carrier, M., & Pashler, H. (1992). The influence of retrieval on retention. Memory & Cognition, 20, 633-642.

Challis, B. H. (1993). Spacing effects on cued-memory tests depend on level of processing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19, 389-396.

Challis, B. H., & Sidhu, R. (1993). Dissociative effect of massed repetition on implicit and explicit measures of memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19, 115-127.

Christoff, K., Gordon, A. M., Smallwood, J., Smith, R., & Schooler, J. W. (2009). Experience sampling during fMRI reveals default network and executive system contributions to mind wandering. PNAS Proceedings of the National Academy of Sciences of the United States of America, 106, 8719-8724. doi: 10.1073/pnas.0900234106

Cialdini, R. B. (2009). Influence: Science and Practice, Fifth Edition. Boston: Pearson.

Clariana, R. B., Ross, S. M., & Morrison, G. R. (1991). The effects of different feedback strategies using computer-administered multiple-choice questions as instruction. Educational Technology Research and Development, 39, 5-17.

Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53, 445-459.

Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21-29.

Clawson, D. M., Healy, A. F., Ericsson, K. A., & Bourne, L. E., Jr. (2001). Retention and transfer of morse code reception skill by novices: Part-whole training. Journal of Experimental Psychology: Applied, 7(2), 129-142.

Cobb, T. (1997). Cognitive efficiency: Toward a revised theory of media. Educational Technology Research and Development, 45(4), 21-35.

Cousins, R., & Hanley, J. R. (1996). The effect of environmental context on recall and category clustering scores following relational and individual item processing: A test of the outshining hypothesis. Memory, 4, 79-90.

Craik, F. I. M., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671-684.

Crooks, T. J. (1988). The impact of classroom evaluation practices on students. Review of Educational Research, 58, 438-481.

Crowder, R. G. (1976). Principles of learning and memory. Hillsdale, NJ: Erlbaum.

Cuddy, L. J., & Jacoby, L. L. (1982). When forgetting helps memory: An analysis of repetition effects. Journal of Verbal Learning and Verbal Behavior, 21, 451-467.

Cull, W. L. (2000). Untangling the benefits of multiple study opportunities and repeated testing for cued recall. Applied Cognitive Psychology, 14, 215-235.

Cull, W. L., Shaughnessy, J. J., & Zechmeister, E. B. (1996). Expanding understanding of the expanding-pattern-of-retrieval mnemonic toward confidence in applicability. Journal of Experimental Psychology, Applied, 2, 365-378.

d’Ydewalle, G. D., Swerts, A., & De Corte, E. (1983). Study time and test performance as a function of test expectations. Contemporary Educational Psychology, 8, 55-67.

Dalton, P. (1993). The role of stimulus familiarity in context-dependent recognition. Memory & Cognition, 21, 223-234.

Davies, G. (1986). Context effects in episodic memory: A review. Cahiers de Psychologie Cognitive, 6, 157-174.

Dellarosa, D., & Bourne, Jr., L. E. (1985). Surface form and the spacing effect. Memory & Cognition, 13, 529-537.

Dempster, F. N. (1987a). Effects of variable encoding and spaced presentations on vocabulary learning. Journal of Educational Psychology, 79, 162-170.

Dempster, F. N. (1987b). Time and the production of classroom learning: Discerning implications from basic research. Educational Psychologist, 22, 1-21.

Dempster, F. N. (1988). The spacing effect: A case study in the failure to apply the results of psychological research. American Psychologist, 43, 627-634.

Dempster, F. N. (1989). Spacing effects and their implications for theory and practice. Educational Psychology Review, 1, 309-330.

Dempster, F. N. (1996). Distributing and managing the conditions of encoding and practice. In E. L. Bjork & R. A. Bjork (Eds.) Memory (pp. 317-344). San Diego, CA: Academic Press.

Dempster, F. N., & Farris, R. (1990). The spacing effect: Research and practice. Journal of Research and Development in Education, 23, 97-101.

Dick, W., & Carey, L. (1996). The Systematic Design of Instruction. NY: HarperCollins.

Doane, S., Sohn, Y. W., & Schreiber, B. (1999). The role of processing strategies in the acquisition and transfer of a cognitive skill. Journal of Experimental Psychology: Human Perception and Performance, 25(5), 1390-1410.

Drachman, D., DeCarufel, A., & Insko, C. A. (1978). The extra credit effect in interpersonal attraction. Journal of Experimental Social Psychology, 14, 458-465. doi: 10.1016/0022-1031(78)90042-2

Driskell, J. E., Willis, R. P., & Copper, C. (1992). Effect of overlearning on retention. Journal of Applied Psychology, 77, 615-622.

Dulsky, S. G. (1935). The effect of a change of background on recall and relearning. Journal of Experimental Psychology, 18, 725-740.

Dumay, N., & Gaskell, M. G. (2007). Sleep-associated changes in the mental representation of spoken words. Psychological Science, 18, 35-39.

Durgunoğlu, A. Y., & Roediger, III, H. L. (1987). Test differences in accessing bilingual memory. Journal of Memory and Language, 26, 377-391.

Durgunoğlu, A. Y., Mir, M., & Ariño-Martí, S. (1993). Effects of repeated readings on bilingual and monolingual memory for text. Contemporary Educational Psychology, 18, 294-317.

Ebbinghaus, H. (1885/1913). Memory: A contribution to experimental psychology, (Translated by H. A. Ruger and C. E. Bussenius). New York: Teachers College, Columbia University. (Also available 1964 and 1987, New York: Dover Publications. Original published in 1885).

Eich, E. (1985). Context, memory, and integrated item/context imagery. Journal of Experimental Psychology: Learning, Memory, and Cognition, 11, 764-770.

Eich, E. (1995). Mood as a mediator of place dependent memory. Journal of Experimental Psychology: General, 124(3), 293-308.

Eich, J. E. (1980). The cue dependent nature of state dependent retrieval. Memory and Cognition, 8, 157-173.

Elmes, D. G., Dye, C. J., & Herdelin, N. J. (1983). What is the role of affect in the spacing effect? Memory & Cognition, 11, 144-151.

English, R. A., & Kinzer, J. R. (1966). The effect of immediate and delayed feedback on retention of subject matter. Psychology in the Schools, 3, 143-147.

Ericsson, K. A. (2006). The Influence of Experience and Deliberate Practice on the Development of Superior Expert Performance. In Ericsson, K. Anders (Ed); Charness, Neil (Ed); Feltovich, Paul J. (Ed); Hoffman, Robert R. (Ed), The Cambridge handbook of expertise and expert performance. (pp. 683-703). New York, NY, US: Cambridge University Press.

Ericsson, K. A. (2009). Enhancing the development of professional performance: Implications from the study of deliberate practice. In Ericsson, K. Anders (Ed), Development of professional expertise: Toward measurement of expert performance and design of optimal learning environments. (pp. 405-431). New York, NY, US: Cambridge University Press.

Ericsson, K. A., & Charness, N. (1994). Expert performance: Its structure and acquisition. American Psychologist, 49, 725-774.

Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100, 363-406.

Fang, X., Singh, S., & Ahluwalia, R. (2007). An examination of different explanations for the mere exposure effect. Journal of Consumer Research, 34, 97-103. doi: 10.1086/513050

Fendrich, D. W., Healy, A. F., & Bourne, Jr., L. E. (1991). Long-term repetition effects for motoric and perceptual procedures. Journal of Experimental Psychology: Learning, Memory, and Cognition, 17, 137-151.

Fernandez, A., & Glenberg, A. M. (1985). Changing environmental context does not reliably affect memory. Memory & Cognition, 13, 333-345.

Fontana, F. E., Mazzardo, O., Furtado, O., Jr., & Gallagher, J. D. (2009). Whole and part practice: A meta-analysis. Perceptual and Motor Skills, 109(2), 517-530.

Foos, P. W., & Clark, M. C. (1983). Learning from text: Effects of input order and expected test. Human Learning, 2, 177-185.

Foos, P. W., & Fisher, R. P. (1988). Using tests as learning opportunities. Journal of Educational Psychology, 80, 179-183.

Frase, L. T. (1967). Learning from prose material: Length of passage, knowledge of results, and position of questions. Journal of Educational Psychology, 58, 266-272.

Frase, L. T., & Kreitzberg, V. S. (1975). Effect of topical and indirect learning directions on prose recall. Journal of Educational Psychology,  67, 320-324.

Frese, M., & Sabini, J. (Eds.). (1985). Goal-directed behavior: The concept of action in psychology. Hillsdale, NJ: Erlbaum.

Gailliot, M. T., & Baumeister, R. F. (2007). The physiology of willpower: Linking blood glucose to self-control. Personality and Social Psychology Review, 11, 303-327.

Gailliot, M. T., Baumeister, R. F., DeWall, C. N., Maner, J. K., Plant, E. A., Tice, D. M., Brewer, L. E., & Schmeichel, B. J. (2007). Self-control relies on glucose as a limited energy source: Willpower is more than a metaphor. Journal of Personality and Social Psychology, 92, 325-336.

Gall, M. D., Ward, B. A., Berliner, D. C., Cahen, L. S., Winne, P. H., Elashoff, J. D., & Stanton, G. C. (1978). Effects of questioning techniques and recitation on student learning. American Educational Research Journal, 15, 175-199.

Gardiner, J. M., Kaminska, Z., Dixon, M., & Java, R. I. (1996). Repetition of previously novel melodies sometimes increases both remember and know responses in recognition memory. Psychonomic Bulletin & Review, 3, 366-371.

Gardner, M. K., Rothkopf, E. Z., Lapan, R., & Lafferty, T. (1987). The word frequency effect in lexical decision: Finding a frequency-based component. Memory & Cognition, 15, 24-28.

Garner, R., Alexander, P. A, Gillingham, M. G., & Brown, R. (1991). Interest and learning from text. American Educational Research Journal, 28, 643-659.

Garner, R., Brown, R., Sanders, S., & Menke, D. J. (1992). “Seductive Details” and learning from text. In K. A. Renninger, S. Hidi, & A. Krapp (Eds.), The Role of Interest in Learning and Development (pp. 239-277). Hillsdale, NJ: Erlbaum.

Garner, R., Gillingham, M. G., & White, C. S. (1989). Effects of “seductive details” on macroprocessing and microprocessing in adults and children. Cognition and Instruction, 6, 41-57.

Gartman, L. M., & Johnson, N. F. (1972). Massed versus distributed repetition of homographs: A test of the differential encoding hypothesis. Journal of Verbal Learning and Verbal Behavior, 11, 801-808.

Gates, A. I. (1917). Recitation as a factor in memorizing. Archives of Psychology, 26, 1-104.

Ghodsian, D., Bjork, R. A., & Benjamin, A. S. (1997). Evaluating training during training: Obstacles and opportunities. In M. A. Quiñones & A. Ehrenstein (Eds.) Training for a rapidly changing workplace: Applications of psychological research (pp. 63-88). Washington, DC: American Psychological Association.

Gladstone, G., & Parker, G. (2003). What’s the use of worrying? Its function and its dysfunction. Australian and New Zealand Journal of Psychiatry, 37, 347-354. doi: 10.1046/j.1440-1614.2003.01187.x

Glanzer, M., & Duarte, A. (1971). Repetition between and within languages in free recall. Journal of Verbal Learning and Verbal Behavior, 10, 625-630.

Glenberg, A. M. (1976). Monotonic and nonmonotonic lag effect in paired-associate and recognition memory paradigms. Journal of Verbal Learning and Verbal Behavior, 15, 1-16.

Glenberg, A. M. (1977). Influences of retrieval processes on the spacing effect in free recall. Journal of Experimental Psychology: Human Learning & Memory, 3, 282-294.

Glenberg, A. M. (1979). Component-levels theory of the effects of spacing and repetitions on recall and recognition. Memory & Cognition, 7, 95-112.

Glenberg, A. M., & Lehmann, T. S. (1980). Spacing repetitions over 1 week. Memory & Cognition, 8, 528-538.

Glover, J. A. (1989). The “testing” phenomenon: Not gone but nearly forgotten. Journal of Educational Psychology, 81, 392-399.

Glover, J. A., & Corkill, A. J. (1987). Influence of paraphrased repetitions on the spacing effect. Journal of Educational Psychology, 79, 198-199.

Godden, D. R., and Baddeley, A. D. (1975). Context dependency in two natural environments: on land and underwater. British Journal of Psychology, 91, 99-104.

Godden, D., & Baddeley, A. (1980). When does context influence recognition memory? British Journal of Psychology, 71, 99-104.

Goettl, B. P., & Shute, V. J. (1996). Analysis of part-task training using the backward-transfer technique. Journal of Experimental Psychology: Applied, 2(3), 227-249.

Goetz, E. T., & Sadoski, M. (1995). The perils of seduction: Distracting details or incomprehensible abstractions? Reading Research Quarterly, 30, 500-511.

Gordon, K. (1925). Class results with spaced and unspaced memorizing. Journal of Experimental Psychology, 8, 337-343.

Grant, H. M., Bredahl, L. C., Clay, J., Ferrie, J., Groves, J. E., McDorman, T. A., & Dark, V. J. (1998). Context-dependent memory for meaningful material: Information for students. Applied Cognitive Psychology, 12, 617-623.

Greene, R. L. (1989). Spacing effects in memory: Evidence for a two-process account. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15, 371-377.

Greene, R. L. (1989). Spacing effects in memory: Evidence for a two-process account. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15, 371-377.

Guéguen, N., & Martin, A. (2009). Incidental similarity facilitates behavioral mimicry. Social Psychology, 40, 88-92. doi: 10.1027/1864-9335.40.2.88

Guéguen, N., & Pascual, A. (2005). Improving the Response Rate to a Street Survey: An Evaluation of the “But You Are Free to Accept or to Refuse” Technique. The Psychological Record, 55, 297-303.

Gueguen, N., Pichot, N., & Le Dreff, G. (2005). Similarity and helping behavior on the Web: The impact of the convergence of surnames between a solicitor and a solicitee in a request made by e-mail. Journal of Applied Social Psychology, 35, 423-429.

Guéguen, N., Pichot, N., & Le Dreff, G. (2005). Similarity and Helping Behavior on the Web: The Impact of the Convergence of Surnames Between a Solicitor and a Subject in a Request Made by E-Mail. Journal of Applied Social Psychology, 35, 423-429. doi: 10.1111/j.1559-1816.2005.tb02128.x

Guthrie, J. T. (1971). Feedback and sentence learning. Journal of Verbal Learning and Verbal Behavior, 10, 23-28.

Haidt, Jonathan (2006), The happiness hypothesis: Finding modern truth in ancient wisdom, New York: Basic Books.

Hall, J. F. (1971). Verbal learning and retention. Philadelphia, Lippincott.

Hall, K. G., Domingues, D. A., Cavazos, R. (1994). Contextual interference effects with skilled baseball players. Perceptual and Motor Skills, 78, 835-841.

Hamaker, C. (1986). The effects of adjunct questions on prose learning. Review of Educational Research, 56, 212-242.

Hamilton, R. J. (1985). A framework for the evaluation of the effectiveness of adjunct questions and objectives. Review of Educational Research, 55, 47-85.

Harp, S. F., & Mayer, R. E. (1997). The role of interest in learning from scientific text and illustrations: On the distinction between emotional interest and cognitive interest. Journal of Educational Psychology, 89(1), 92-102.

Healy, A. F., Fendrich, D. W., & Proctor, J. D. (1990). Acquisition and retention of a letter-detection skill. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 270-281.

Heider, F. (1958). The psychology of interpersonal relations. New York: Wiley.

Herz, R. S. (1997). The effects of cue distinctiveness on odor-based context-dependent memory. Memory & Cognition, 25(3), 375-380.

Hinds, P. J. (1999). The curse of expertise: The effects of expertise and debiasing methods on predictions of novice performance. Journal of Experimental Psychology: Applied, 5, 205-221.

Hintzman, D. L. (1974). Theoretical implications of the spacing effect. In R. L. Solso (Ed.), Theories in cognitive psychology: The Loyola Symposium (pp. 77-99). Potomac, MD: Erlbaum.

Hintzman, D. L. (1976). Repetition and memory. In Bower, G. H. (Ed.) The psychology of learning and motivation (Vol. 10) Advances in theory and research (pp. 47-91). New York: Academic Press.

Hintzman, D. L. (1987). Personal Communication. Cited by Dempster (1989, p. 326).

Hofmann, S. G., Moscovitch, D. A., Litz, B. T., Kim, H., Davis, L. L., & Pizzagalli, D. A. (2005). The Worried Mind: Autonomic and Prefrontal Activation During Worrying. Emotion, 5, 464-475. doi: 10.1037/1528-3542.5.4.464.

Hogan, R. M., & Kintsch, W. (1971). Differential effects of study and test trials on long-term recognition and recall. Journal of Verbal Learning and Verbal Behavior, 10, 562-567.

Hyoenae, J., & Niemi, P. (1990). Eye movements during repeated reading of a text. Acta Psychologica, 73, 259-280.

Ivanov, B., Pfau, M., & Parker, K. A. (2009). Can inoculation withstand multiple attacks?: An examination of the effectiveness of the inoculation strategy compared to the supportive and restoration strategies. Communication Research, 36, 655-676.

Iyengar, S.S., & Lepper, M.R. (2000). When choice is demotivating: Can one desire too much of a good thing? Journal of Personality and Social Psychology, 79, 995–1006.

Izawa, C. (1992). Test trials contributions to optimization of learning processes: Study/test trials interactions. In A. F. Healy, S. M. Kosslyn, & R. M. Shiffrin (Eds.) From Learning Processes to Cognitive Processes: Essays in Honor of William K. Estes (Volume 2, pp. 1-33). Hillsdale, NJ: Erlbaum.

Jacoby, L. L. (1978). On interpreting the effects of repetition: Solving a problem versus remembering a solution. Journal of Verbal Learning and Verbal Behavior, 17, 649-667.

Jacoby, L. L. (1983). Remembering the data: Analyzing interactive processes in reading. Journal of Verbal Learning and Verbal Behavior, 22, 485-508.

Jentsch, F., & Bowers, C. A. (1998). Evidence for the validity of PC-based simulations in studying aircrew coordination. International Journal of Aviation Psychology, 8, 243-260.

Jones, H. E. (1923-1924). Experimental studies of college teaching: The effect of examination on permanence of learning. Archives of Psychology, 10, 5-70.

Kahana, M. J., & Greene, R. L. (1993). Effects of spacing on memory for homogenous lists. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19, 159-162.

Kaplan, R. (1974). Effects of learning prose with part versus whole presentations of instructional objectives. Journal of Educational Psychology, 66, 448-456.

Kaplan, R., & Rothkopf, E. Z. (1974). Instructional objectives as directions to learners: Effect of passage length and amount of objective-relevant content. Journal of Educational Psychology, 66, 448-456.

Karraker, R. J. (1967). Knowledge of results and incorrect recall of plausible multiple-choice alternatives. Journal of Educational Psychology, 58, 11-14.

Kausler, D. H., Wiley, J. G., & Phillips, P. L. (1990). Adult age differences in memory for massed and distributed repeated actions. Psychology and Aging, 5, 530-534.

Keller, K.L., & Staelin, R. (1987). Effects of quality and quantity of information on decision effectiveness. Journal of Consumer Research, 14, 200–213.

Keltner, Dacher and Jon Haidt (2003), “Approaching awe: a moral, spiritual, and aesthetic emotion,” Cognition and Emotion, 17, 297-314.

Kernan, M. G., & Lord, R. G. (1989). The effects of explicit goals and specific feedback on escalation processes. Journal of Applied Social Psychology, 19, 1125-1143.

Kiewra, K. A. (1989). A review of note taking: The encoding-storage paradigm and beyond. Educational Psychology Review, 1, 147—172.

Kiewra, K. A., Mayer, R. E., Christensen, M., Kim, S-I., & Risch, N. (1991). Effects of repetition on recall and note-taking strategies for learning from lectures. Journal of Educational Psychology, 83, 120-123.

Klauer, K. J. (1984). Intentional and incidental learning with instructional texts: A meta-analysis for 1970-1980. American Educational Research Journal, 21, 323-339.

Klein, R. D., & Fleck, R. A. (1990). International business simulation/gaming: An assessment and review. Simulation and Gaming, 21, 147-165.

Koulack, D. (1997). Recognition memory, circadian rhythms, and sleep. Perceptual & Motor Skills, 85, 99-104.

Kozma, R. B. (1994). A reply: Media and methods. Educational Technology Research and Development, 42(3), 11-14.

Kozma, R. B. (1994). Will media influence learning? Reframing the debate. Educational Technology Research and Development, 42(2), 7-19.

Krueger, W. C. F. (1929). The effect of overlearning on retention. Journal of Experimental Psychology, 12, 71-78.

Krug, D., Davis, T. B., & Glover, J. A. (1990). Massed versus distributed repeated reading: A case of forgetting helping recall? Journal of Educational Psychology, 82, 366-371.

Kulhavy, R. W. (1977). Feedback in written instruction. Review of Educational Research, 47, 211-232.

Kulhavy, R. W., & Anderson, R. C. (1972). Delay-retention effect with multiple-choice tests. Journal of Educational Psychology, 63, 505-512.

Kulhavy, R. W., & Stock, W. A. (1989). Feedback in written instruction: The place of response certitude. Educational Psychology Review, 1, 279-308.

Kulhavy, R. W., Yekovich, F. R., & Dyer, J. W. (1976). Feedback and response confidence. Journal of Educational Psychology, 68, 522-528.

Kulik, J. A., & Kulik, C-L. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58, 79-97.

Kuo, T. M., & Hirshman, E. (1996). Investigations of the testing effect. American Journal of Psychology, 109, 451-464.

Lachman, R., & Laughery, K. R. (1968). Is a test trial a training trial in free recall learning? Journal of Experimental Psychology, 76, 40-50.

Landaeur, T. K., & Bjork, R. A. (1978). Optimum rehearsal patterns and name learning. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes, (Eds.), Practical Aspects of Memory (pp. 625-632). New York: Academic Press.

Landaeur, T. K., & Ross, B. H. (1977). Can simple instructions to used spaced practice improve ability to remember a fact?: An experimental test using telephone numbers. Bulletin of the Psychonomic Society, 10, 215-218.

Landauer, T. K., & Ainslie, K. I. (1975). Exams and use as preservatives of course-acquired knowledge. Journal of Educational Research, 69, 99-104.

Landauer, T. K., & Bjork, R. A. (1978). Optimum rehearsal patterns and name learning. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes, (Eds.), Practical Aspects of Memory (pp. 625-632). London: Academic Press.

Landauer, T. K., & Eldridge, L. (1967). Effect of tests without feedback and presentation-test interval in paired-associate learning. Journal of Experimental Psychology, 75, 290-298.

Latham, G. P., & Locke, E. A. (1991). Self-regulation through goal-setting. Organizational Behavior and Human Decision Processes, 50, 212-247.

Latham, G. P., & Yukl, G. A. (1975). A review of research on the application of goal setting in organizations. Academy of Management Journal, 18, 824-845.

Lavery, J. J. (1962). Retention of simple motor skills as a function of type of knowledge of results. Canadian Journal of Psychology, 16, 300-311.

Lavery, J. J. (1964). Retention of a skill following training with and without instructions to retain. Perceptual and Motor Skills, 18, 275-281.

Lee, T. D., & Magill, R. A. (1983). The locus of contextual interference in motor-skill acquisition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 9, 730-746.

Levav, J., & Argo, J. (2010). Physical contact and financial risk taking. Psychological Science OnlineFirst, published on April 22, 2010 as doi:10.1177/0956797610369493.

Lim, J., Reiser, R. A., & Olina, Z. (2009). The effects of part-task and whole-task instructional approaches on acquisition and transfer of a complex cognitive skill. Educational Technology Research and Development, 57(1), 61-77.

Locke, E. A. (1968). Toward a theory of task motivation and incentives. Organizational Behavior and Human Performance, 3, 157-189.

Locke, E. A., & Henne, D. (1986). Work motivation theories. In C. Cooper & I. Robertson (Eds.), International review of industrial and organizational psychology. Chichester, England: Wiley.

Locke, E. A., & Latham, G. P. (1990). A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice-Hall.

Locke, E. A., Shaw, K. M., Saari, L. M., & Latham, G. P. (1981). Goal setting and task performance: 1969-1980. Psychological Bulletin, 90, 125-152.

Madigan, S. A. (1969). Intraserial repetition and coding processes in free recall. Journal of Verbal Learning and Verbal Behavior, 8, 828-835.

Madigan, S. A. (1969). Intraserial repetition and coding processes in free recall. Journal of Verbal Learning and Verbal Behavior, 8, 828-835.

Mager, R. F. (1997; 1962). Preparing instructional objectives. Atlanta, GA: Center for Effective Performance.

Magill, R. (1993). Augmented feedback in skill acquisition. In R. N. Singer, M. Murphey, & L. K. Tennant (Eds.), Handbook of Research on Sport Psychology (pp. 193-212). NY: Macmillan.

Magliero, A. (1983). Pupil dilations following pairs of identical and related to-be-remembered words. Memory & Cognition, 11, 609-615.

Marian, V., & Neisser, E. (2000). Language-dependent recall of autobiographical memories. Journal of Experimental Psychology: General, 129, 361-368.

Marmurek, H. H. C., Holt, P. D., & Coe, K. (1978). Presentation mode and repetition effects in free recall. American Journal of Psychology, 91, 483-490.

Martin, M., & Jones, G. V. (1998). Generalizing everyday memory: Signs and handedness. Memory & Cognition, 26, 193-200.

Mattoon, J. S. (1994). Designing instructional simulations: Effects of instructional control and type of training task on developing display-interpretation skills. International Journal of Aviation Psychology, 4(3), 189-209.

Mayer, R. E. (1983). Can you repeat that? Qualitative effects of repetition and advance organizers on learning from science prose. Journal of Educational Psychology, 75, 40-49.

Mayer, R. E. (1989). Systematic thinking fostered by illustrations in scientific text. Journal of Educational Psychology, 81, 240-246.

Mayer, R. E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist, 32, 1-19.

Mayer, R. E., & Gallini, J. K. (1990). When is an illustration worth ten thousand words? Journal of Educational Psychology, 82, 715-726.

Maylor, E. A. (1998). Retrieving names in old age: Short- and (very) long-term effects of repetition. Memory & Cognition, 26, 309-319.

McCaul, K. D., Mullens, A. B., Romanek, K. M., Erickson, S. C., & Gatheridge, B. J. (2007). The motivational effects of thinking and worrying about the effects of smoking cigarettes. Cognition and Emotion, 21, 1780-1798. doi: 10.1080/02699930701442840

McVay, J. C., & Kane, M. J. (2009). Conducting the train of thought: Working memory capacity, goal neglect, and mind wandering in an executive-control task. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 196-204. doi: 10.1037/a0014104

Melton, A. W. (1970). The situation with respect to the spacing of repetitions and memory. Journal of Verbal Learning and Verbal Behavior, 9, 596-606.

Mento, A. J., Steel, R. P., Karren, R. J. (1987). A meta-analytic study of the effects of goal setting on task performance: 1966-1984. Organizational Behavior and Human Decision Processes, 39, 52-83.

Meyer, G. (1934). An experimental study of the old and new types of examination: I. The effect of the examination set on memory. Journal of Educational Psychology, 25, 641-661.

Meyer, G. (1935). An experimental study of the old and new types of examination: II. Methods of study. Journal of Educational Psychology, 26, 30-40.

Meyer, W., Niepel, M., Rudolph, U., & Schützwohl, A. (1991). An experimental analysis of surprise. Cognition and Emotion, 5, 295-311.

Mita, T. H., Dermer, M., & Knight, J. (1977). Reversed facial images and the mere-exposure hypothesis. Journal of Personality and Social Psychology, 35, 597-601. doi: 10.1037/0022-3514.35.8.597

Modigliani, V. (1976). Effects on a later recall by delaying initial recall. Journal of Experimental Psychology: Human Learning and Memory, 2, 609-622.

Modigliani, V. (1978). Effects of initial testing on later retention as a function of the initial retention interval. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes, (Eds.), Practical Aspects of Memory (pp. 652-659). London: Academic Press.

Modigliani, V., & Hedges, D. G. (1987). Distributed rehearsals and the primacy effect in single-trial free recall. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13, 426-436.

Modigliani, V., Rea, C. P., & Hedges, D. G. (1988). Within-story recalls and the retention of ideas in prose processing. Canadian Journal of Behavioural Science, 20, 73-84.

Moore, J. W., & Smith, W. I. (1961). Knowledge of results in self-teaching spelling. Psychological Reports, 9, 717-726.

More, A. (1969). Delay of feedback and the acquisition and retention of verbal materials in the classroom. Journal of Educational Psychology, 60, 339-342.

Moreno, R., & Mayer, R. E. (2000). A coherence effect in multimedia learning: The case for minimizing irrelevant sounds in the design of multimedia instructional messages. Journal of Educational Psychology, 92, 117-125.

Moreno, R., & Mayer, R. E. (2002). Learning science in virtual reality multimedia environments: Role of methods and media. Journal of Educational Psychology, 94, 598-610.

Morris, P. E., & Fritz, C. O. (2000). The name game: Using retrieval practice to improve the learning of names. Journal of Experimental Psychology, 6, 124-129.

Mory, E. H. (1992). The use of informational feedback in instruction: Implications for future research. Educational Technology Research and Development, 40, 5-20.

Muraven, M., & Slessareva, E. (2003). Mechanisms of self-control failure: Motivation and limited resources. Personality and Social Psychology Bulletin, 29, 894–906.

Muraven, M., Baumeister, R. F., & Tice, D. M. (1999). Longitudinal improvement of self-regulation through practice: Building self-control strength through repeated exercise. The Journal of Social Psychology, 139, 446-457.

Muraven, M., Shmueli, D., & Burkley, E. (2006). Conserving self-control strength. Journal of Personality and Social Psychology, 91, 524–537.

Muraven, M., Tice, D. M., & Baumeister, R. F. (1998). Self-control as a limited resource: Regulatory depletion patterns. Journal of Personality and Social Psychology, 74, 774-789.

Muth, K. D., Glynn, S. M., Britton, B. K., & Graves, M. F. (1988). Thinking out loud while studying text: Rehearsing key ideas. Journal of Educational Psychology, 80, 315-318.

Naylor, G., & Briggs, J. (1963) Effects of task complexity and task organization on the relative efficiency of part and whole training methods. Journal of Experimental Psychology, 65, 217-224.

Naylor, J. C. (1962). Parameters affecting the relative efficiency of part and whole training methods: A review of the literature (Report No. NAVTRADEVCEN 950-1). Port Washington, NY: U. S. Naval Training Device Center.

Neely, J. H., & Balota, D. A. (1981). Test-expectancy and semantic-organization effects in recall and recognition. Memory & Cognition, 9, 283-300.

Newman, M. I., Williams, R. G., & Hiller, J. H. (1974). Delay of information feedback in an applied setting: Effects on initially learned and unlearned items. Journal of Experimental Education, 42, 55-59.

Nickerson, R. S., & Adams, M. J. (1979). Long-term memory for a common object. Cognitive Psychology, 11, 287-307.

Nungester, R. J., & Duchastel, P. C. (1982). Testing versus review: Effects on retention. Journal of Educational Psychology, 74, 18-22.

Oldjoke, W. M. (1958). How do you get to Carnegie Hall? Practice, practice, practice!! Journal of Will’s Memory, 1, 1-2.

Pan, S. (1926). The influence of context upon learning and recall. Journal of Experimental Psychology, 9, 468-491.

Parker, A., & Gellatly, A. (1997). Moveable cues: A practical method for reducing context-dependent forgetting. Applied Cognitive Psychology, 11, 163-173.

Parkin, A. J., Wood, A., & Aldrich, F. K. (1988). Repetition and active listening: The effects of spacing self-assessment questions. British Journal of Psychology, 79, 77-86.

Pascual, A., & Guéguen, N. (2005). Foot-in-the-door and Door-in-the-face: A Comparative Meta-analytic Study. Psychological Reports, 96, 122-128. doi: 10.2466/PR0.96.1.122-128

Peeck, J., van den Bosch, A. B., & Kreupeling, W. J. (1985). Effects of informative feedback in relation to retention of initial responses. Contemporary Educational Psychology, 10, 303-313.

Perkins, A. M., & Corr, P. J. (2005). Can worriers be winners? The association between worrying and job performance. Personality and Individual Differences, 38, 25-31. doi: 10.1016/j.paid.2004.03.008

Phye, G. D. (1991). Advice and feedback during cognitive training: Effects at acquisition and delayed transfer. Contemporary Educational Psychology, 16, 87-94.

Phye, G. D., & Andre, T. (1989). Delayed retention effect: Attention, perseveration, or both. Contemporary Educational Psychology, 14, 173-185.

Phye, G. D., & Sanders, C. E. (1994). Advice and feedback: Elements of practice for problem solving. Contemporary Educational Psychology, 19, 286-301.

Pichert, J. W., & Anderson, R. C. (1977). Taking different perspectives on a story. Journal of Educational Psychology, 69, 309-315.

Plihal, W., & Born, J. (1997). Effects of early and late nocturnal sleep on declarative and procedural memory. Journal of Cognitive Neuroscience, 9, 534-547.

Plihal, W., & Born, J. (1999). Effects of early and late nocturnal sleep on priming and spatial memory. Psychophysiology, 36, 571-582.

Pressley, M., Tanenbaum, R., McDaniel, M. A., & Wood, E. (1990). What happens when university students try to answer prequestions that accompany textbook material? Contemporary Educational Psychology, 15, 27-35.

Pyle, W. H. (1913). Economical learning. Journal of Educational Psychology, 3, 148-158.

Racsmány, M., Conway, M. A., & Demeter, G. (2010). Consolidation of episodic memories during sleep: Long-term effects of retrieval practice. Psychological Science, 21(1), 80-85.

Rea, C. P., & Modigliani, V. (1985). The effect of expanded versus massed practice on the retention of multiplication facts and spelling lists. Human Learning, 4, 11-18.

Rea, C. P., & Modigliani, V. (1985). The effect of expanded versus massed practice on the retention of multiplication facts and spelling lists. Human Learning, 4, 11-18.

Rea, C. P., & Modigliani, V. (1987). The spacing effect in 4- to 9-year-old children. Memory & Cognition, 15, 436-443.

Rea, C. P., & Modigliani, V. (1988). Educational implications of the spacing effect. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes (Eds.) Practical aspects of memory: Current research and issues, Vol. 1: Memory in everyday life (pp. 402-406). New York: John Wiley & Sons.

Reder, L. M., & Anderson, J. R. (1982). Effects of spacing and embellishment on memory for the main points of a text. Memory & Cognition, 10, 97-102.

Reiser, R. A. (1994). Clark’s invitation to the dance: An instructional designer’s response. Educational Technology Research and Development, 42(2), 45-48.

Rethans, A. J., Swasy, J. L., & Marks, L. J. (1986). Effects of television commercial repetition, receiver knowledge, and commercial length: A test of the two-factor model. Journal of Marketing Research, 23, 50-61.

Reynolds, J. H., & Glaser, R. (1964). Effects of repetition and spaced review upon retention of a complex learning task. Journal of Educational Psychology, 55, 297-308.

Reynolds, R. E., Trathen, W., Sawyer, M. L., & Shepard, C. R. (1993). Causal and epiphenomenal use of selective attention strategy in prose comprehension. Contemporary Educational Psychology, 18, 258-278.

Riccio, D. C., Richardson, R., & Ebner, D. L. (1984). Memory retrieval deficits based upon altered contextual cues: A paradox. Psychological Bulletin, 96, 152-165.

Rickards, J. P. (1979). Adjunct postquestions in text: A critical review of methods and processes. Review of Educational Research, 49, 181-196.

Roediger, H. L., & Challis, B. H. (1992). Effects of exact repetition and conceptual repetition on free recall and primed word-fragment completion. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 3-14.

Roediger, H. L., III, & Guynn, M. J. (1996). Retrieval processes. In E. L. Bjork & R. A. Bjork (eds.), Memory (pp. 197-236). San Diego, CA: Academic Press.

Rose, R. J. (1992). Degree of learning, interpolated tests, and rate of forgetting. Memory & Cognition, 20, 621-632.

Ross, R. T., & Pirie, M. (1934). The persistence of errors in successive true-false tests. Journal of Educational Psychology, 25, 422-426.

Rothkopf, E. Z. (1963). Some observations on predicting instructional effectiveness by simple inspection. The Journal of Programmed Instruction, 2, 19-20.

Rothkopf, E. Z. (1965). Some theoretical and experimental approaches to problems in written instruction. In J. D. Krumboltz (Ed.), Learning and the educational process (pp. 193-221). Chicago: Rand McNally.

Rothkopf, E. Z. (1966). Learning from written instructive materials: An exploration of the control of inspection behavior by test-like events. American Educational Research Journal, 3, 241-249.

Rothkopf, E. Z. (1968). Textual constraint as function of repeated instruction. Journal of Educational Psychology, 59, 20-25.

Rothkopf, E. Z. (1982). Adjunct aids and the control of mathemagenic activities during purposeful reading. In W. Otto & S. White (Eds.) Reading expository material. New York: Academic Press.

Rothkopf, E. Z., & Billington, M. J. (1975). Relevance and similarity of text elements to descriptions of learning goals. Journal of Educational Psychology, 67, 745-750.

Rothkopf, E. Z., & Billington, M. J. (1979). Goal-guided learning from text: Inferring a descriptive processing model from inspection times and eye movements. Journal of Educational Psychology, 71, 310-327.

Rothkopf, E. Z., & Bisbicos, E. E. (1967). Selective facilitative effects of interspersed questions on learning from written material. Journal of Educational Psychology, 58, 56-61.

Rothkopf, E. Z., & Coke, E. U. (1963). Repetition interval and rehearsal method in learning equivalences from written sentences. Journal of Verbal Learning and Verbal Behavior, 2, 406-416.

Rothkopf, E. Z., & Coke, E. U. (1966). Variations in phrasing, repetition intervals, and the recall of sentence material. Journal of Verbal Learning and Verbal Behavior, 5, 86-91.

Rothkopf, E. Z., & Kaplan, R. (1972). An exploration of the effect of density and specificity of instructional objectives on learning from text. Journal of Educational Psychology, 6, 295-302.

Rothstein, A. L., & Arnold, R. K. (1976). Bridging the gap: Application of research on video-tape feedback and bowling. Motor Skills: Theory into Practice, 1, 35-62.

Ruch, T. C. (1928). Factors influencing the relative economy of massed and distributed practice in learning. Psychological Review, 35, 19-45.

Runquist, W. (1983). Some effects of remembering on forgetting. Memory & Cognition, 11, 641-650.

Runquist, W. (1986). The effect of testing on the forgetting of related and unrelated associates. Canadian Journal of Psychology, 40, 65-76.

Russo, R., Parkin, A. J., Taylor, S. R., & Wilks, J. (1998). Revising current two-process accounts of spacing effects in memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 161-172.

Russo, R., Ward, G., Geurts, H., & Scheres, A. (1999). When unfamiliarity matters: Changing environmental context between study and test affects recognition memory for unfamiliar stimuli. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 488-499.

Salasoo, A., Shiffrin, R. M., & Feustel, T. C. (1985). Building permanent memory codes: Codification and repetition effects in word identification. Journal of Experimental Psychology: General, 114, 50-77.

Salden, R. J. C. M. (2005). Dynamic task selection in aviation training. Report NLR-TP-2004-465. National Aerospace Laboratory NLR. Downloaded September 2010 at http://www.nlr.nl/id~2851/lang~en.pdf.

Salden, R. J. C. M., Paas, F., & van Merriënboer, J. J. G. (2006). A comparison of approaches to learning task selection in the training of complex cognitive skills. Computers in Human Behavior, 22(3), 321-333.

Salmoni, A. W., Schmidt, R. A, & Walter, C. B. (1984). Knowledge of results and motor learning: A review and critical reappraisal. Psychological Bulletin, 95, 355-386.

Sassenrath, J. M. (1975). Theory and results on feedback and retention. Journal of Educational Psychology, 67, 894-899.

Sassenrath, J. M., & Gaverick, C. M. (1965). Effects of differential feedback from examinations on retention and transfer. Journal of Educational Psychology, 56, 259-263.

Sassenrath, J. M., & Yonge, G. D. (1968). Delayed information feedback, feedback cues, retention set, and delayed retention. Journal of Educational Psychology, 59, 69-73.

Sassenrath, J. M., & Yonge, G. D. (1969). Effects of delayed information feedback and feedback cues in learning on delayed retention. Journal of Educational Psychology, 60, 174-177.

Schab, F. R. (1990). Odors and remembrances of things past. Journal of Experimental Psychology: Learning, Memory, and Cognition, 8, 648-655.

Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3, 207-217.

Schmidt, S. R. (1983). The effects of recall and recognition test expectancies on the retention of prose. Memory & Cognition, 11, 172-180.

Schraw, G. (1998). Processing and recall differences among seductive details. Journal of Educational Psychology, 90(1), 3-12.

Schützwohl, A. (1998). Surprise and schema strength. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 1182-1199.

Shah, A. M., & Wolford, G. (2007). Buying behavior as a function of parametric variation of number of choices. Psychological Science, 18, 369-370.

Shaughnessy, J. J., Zimmerman, J., & Underwood, B. J. (1972). Further evidence on the MP-DP effect in free-recall learning. Journal of Verbal Learning and Verbal Behavior, 11, 1-12.

Shea, J. B., & Morgan, R. L. (1979). Contextual interference effects on the acquisition, retention, and transfer of a motor skill. Journal of Experimental Psychology: Human Learning and Memory, 5, 179-187.

Shebilske, W. L., Goettl, B. P., Corrington, K., & Day, E. A. (1999). Interlesson spacing and task-related processing during complex skill acquisition. Journal of Experimental Psychology: Applied, 5, 413-437.

Shiota, Michelle, Dacher Keltner, and Amanda Mossman (2007), “The nature of awe: elicitors, appraisals, and effects on self-concept,” Cognition and Emotion, 21, 944-963.

Shrock, S. A. (1994). The media influence debate: Read the fine print, but don’t lose sight of the big picture. Educational Technology Research and Development, 42(2), 49-53.

Simon, H. A., & Chase, W. G. (1973). Skill in chess. American Scientist, 61, 394-403.

Singh, S. N., Mishra, S., Bendapudi, N., & Linville, D. (1994). Enhancing memory of television commercials through message spacing. Journal of Marketing Research, 31, 384-392.

Sitzmann, T., Brown, K. G., Casper, W. J., Ely, K., & Zimmerman, R. D. (2008). A review and meta-analysis of the nomological network of trainee reactions. Journal of Applied Psychology, 93, 280-295.

Sitzmann, Traci, Brown, Kenneth G., Ely, Katherine, Kraiger, Kurt and Wisher, Robert A. (2009) A Cyclical Model of Motivational Constructs in Web-Based Courses, Military Psychology, 21, 534 — 551.

Smallwood, J., Fishman, D. J., & Schooler, J. W. (2007). Counting the cost of an absent mind: Mind wandering as an underrecognized influence on educational performance. Psychonomic Bulletin & Review, 14, 230-236.

Smith, L., & Klein, R. (1990). Evidence for semantic satiation: Repeating a category slows subsequent semantic processing. Journal of Experimental Psychology: Learning, Memory, & Cognition, 16, 852-861.

Smith, S. M. (1979). Remembering in and out of context. Journal of Experimental Psychology: Human Learning and Memory, 5, 460-471.

Smith, S. M. (1982). Enhancement of recall using multiple environmental contexts during learning. Memory & Cognition, 10, 405-412.

Smith, S. M. (1984). A comparison of two techniques for reducing context-dependent forgetting. Memory & Cognition, 12, 477-482.

Smith, S. M. (1985). Background music and context-dependent memory. American Journal of Psychology, 98, 591-603.

Smith, S. M. (1988). Environmental context-dependent memory. In G. M. Davies & D. M. Thomson (eds.) Memory in Context: Context in Memory (pp. 13-34), Chichester, UK: Wiley.

Smith, S. M. (1995). Mood is a component of mental context: Comment on Eich (1995). Journal of Experimental Psychology: General, 124(3), 309-310.

Smith, S. M., & Rothkopf, E. Z. (1984). Contextual enrichment and distribution of practice in the classroom. Cognition and Instruction, 1, 341-358.

Smith, S. M., & Vela, E. (2001). Environmental context-dependent memory: A review and meta-analysis. Psychonomic Bulletin & Review, 8, 203-220.

Smith, S. M., Glenberg, A., and Bjork, R. A. (1978). Environmental context and human memory. Memory and Cognition, 6, 342-353.

Spear, N. E. (1978). The processing of memories: Forgetting and retention. Hillsdale, NJ: Erlbaum.

Spitzer, H. F. (1939). Studies in retention. Journal of Educational Psychology, 30, 641-656.

Srull, T. K., & Wyer, R. S., Jr. (1986). The role of chronic and temporary goals in social information processing. In R. M Sorrentino & E. T. Higgins (Eds.), Handbook of motivation and cognition: Foundations of social behavior (Vol. 1, pp. 503-549). New York: Guilford Press.

Stammers, R. B. (1982). Part and whole practice in training for procedural tasks. Human Learning, 1, 185-207.

Stickgold, R., & Walker, M. P. (2005). Memory consolidation and reconsolidation: What is the role of sleep? Trends in Neurosciences, 28, 408-415.

Sturges, P. T. (1969). Verbal retention as a function of the informativeness  and delay of informative feedback. Journal of Educational Psychology, 60, 11-14.

Sturges, P. T. (1972). Information delay and retention: Effect of information in feedback and tests. Journal of Educational Psychology, 63, 32-43.

Sturges, P. T. (1978). Delay of informative feedback in computer-assisted testing. Journal of Educational Psychology, 70, 378-387.

Surber, J. R., & Anderson, R. C. (1975). Delay-retention effect in natural classroom settings. Journal of Educational Psychology, 67, 170-173.

Swinnen, S. P. (1990). Interpolated activities during the knowledge-of-results delay and post-knowledge of results interval: Effects on performance and learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 692-705.

Swinnen, S. P., Schmidt, R. A., Nicholson, D. E., & Shapiro, D. C. (1990). Information feedback for skill acquisition: Instantaneous knowledge of results degrades learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 706-716.

Teague, R. C., Gittelman, S. S., & Park, O. (1994). A review of the literature on part-task and whole-task training and context dependency. (ARI Technical Report 1010). U.S. Army Research Institute.

Thalheimer, W. H. (1996). Information-acquisition goals: How questions produce learning through non-strategic processing. Unpublished doctoral dissertation, Columbia University Teachers College.

Thios, S. J. (1972). Memory for words in repeated sentences. Journal of Verbal Learning and Verbal Behavior, 11, 789-793.

Toppino, T. C. (1991). The spacing effect in young children’s free recall: Support for automatic-process explanations. Memory & Cognition, 19, 159-167.

Toppino, T. C., & Brochin, H. A. (1989). Learning from tests: The case of true-false examinations. Journal of Educational Research, 83, 119-124.

Toppino, T. C., & Gracen, T. F. (1985). The lag effect and differential organization theory: Nine failures to replicate. Journal of Experimental Psychology: Learning, Memory, and Cognition, 11, 185-191.

Tsai, C. I., Klayman, J., & Hastie, R. (2008). Effects of amount of information on judgment accuracy and confidence. Organizational Behavior and Human Decision Processes, 107, 97-105.

Tubbs, M. E. (1986). Goal setting: A meta-analytic examination of the empirical evidence. Journal of Applied Psychology, 71, 474-483.

Tulving, E., & Thompson, D. M. (1973). Encoding specificity and retrieval processes in episodic memory. Psychological Review, 80, 352-373.

VanSickle, R. L. (1986). A quantitative review of research on instructional simulation gaming: A twenty-year perspective. Theory & Research in Social Education, 14, 245-264.

Vela, E. (1984). Memory as a function of environmental context. Paper presented at the 30th annual meeting of the Southwest Psychological Association, New Orleans, LA.

Vohs, K. D., Baumeister, R. F., Schmeichel, B. J., Twenge, J. M., Nelson, N. M., & Tice, D. M. (2008). Making choices impairs subsequent self-control: A limited-resource account of decision making, self-regulation, and active initiative. Journal of Personality and Social Psychology, 94, 883-898.

Vohs, K. D., Baumeister, R. F., Schmeichel, B. J., Twenge, J. M., Nelson, N. M., & Tice, D. M. (2008). Making choices impairs subsequent self-control: A limited-resource account of decision making, self-regulation, and active initiative. Journal of Personality and Social Psychology, 94, 883-898.

Wade, S. E., & Adams, B. (1990). Effects of importance and interest on recall of biographical text. JRB: A Journal of Literacy, 22, 331-353.

Wade, S. E., Schraw, G., Buxton, W. M., & Hayes, M. T. (1993). Seduction of the strategic reader: Effects of interest on strategies and recall. Reading Research Quarterly, 28, 3-24.

Wagner, A. D., Desmond, J. E., Demb, J. B., Glover, G. H., & Gabrieli, J. D. E. (1997). Semantic repetition priming for verbal and pictorial knowledge: A functional MRI study of left inferior prefrontal cortex. Journal of Cognitive Neuroscience, 9, 714-726.

Walker, M. P., Brakefield, T., Morgan, A., Hobson, J. A., & Stickgold, R. (2002). Practice with sleep makes perfect: Sleep-dependent motor skill learning. Neuron, 35, 205-211.

Waugh, N. C. (1962). The effect of intralist repetition on free recall. Journal of Verbal Learning and Verbal Behavior, 1, 95-99.

Webb, J. M., Stock, W. A., & McCarthy, M. T. (1994). The effects of feedback timing on learning facts: The role of response confidence. Contemporary Educational Psychology, 19, 251-265.

Weisberg, Deena Skolnick; Keil, Frank C.; Goodstein, Joshua; Rawson, Elizabeth; Gray, Jeremy R. (2008). The seductive allure of neuroscience explanations. Weisberg, D. S., Keil, F. C., Goodstein, J., Rawson, E., & Gray, J. R. (2008). The seductive allure of neuroscience explanations. Journal of Cognitive Neuroscience, 20(3), 470-477.

Weiss, W., & Margolis, G. (1954). The effect of context stimuli on learning and retention. Journal of Experimental Psychology, 48, 318-322.

Wells, J. E., & Kirsner, K. (1974). Repetition between and within modalities in free recall. Bulletin of the Psychonomic Society, 2(4B), 395-397.

Wenger, S. K., Thompson, C. P., & Bartling, C. A. (1980). Recall facilitates subsequent recognition. Journal of Experimental Psychology: Human Learning and Memory, 6, 135-144.

Whitten, W. B., II, & Bjork, R. A. (1977). Learning from tests: Effects of spacing. Journal of Verbal Learning and Verbal Behavior, 16, 465-478.

Whitten, W. B., II, & Leonard, J. M. (1980). Learning from tests: Facilitation of delayed recall by initial recognition alternatives. Journal of Experimental Psychology: Human Learning and Memory, 6, 127-134.

Wightman, D. C., & Lintern, G. (1985). Part-task training for tracking and manual control. Human Factors, 27(3), 267-283.

Wightman, D. C., & Sistrunk, F. (1987). Part-task training strategies in simulated carrier landing final-approach training. Human Factors, 29(3), 245-254.

Wixson, K. K. (1983). Postreading question—answer interactions and children’s learning from text. Journal of Educational Psychology, 30, 413-423.

Wright, D. L., & Shea, C. H. (1991). Context dependencies in motor skills. Memory & Cognition, 19, 361-370.

Wulf, G., & Schmidt, R. A. (1997). Variability of practice and implicit motor learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 987-1006.

Wulf, G., Shea, C. H., & Matschiner, S. (1998). Frequent feedback enhances complex motor skill learning. Journal of Motor Behavior, 30, 180-192.

Wyer, Jr., R. S., Srull, T. K., Gordon, S. E., & Hartwick, J. (1982). Effects of processing objectives on the recall of prose material. Journal of Personality and Social Psychology, 43, 674-688.

Yaniv, I., Meyer, D. E., & Davidson, N. S. (1995). Dynamic memory processes in retrieving answers to questions: Recall failures, judgments of knowing, and acquisition of information. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 1509-1521.

Zajonc, R. B. (1965). Social facilitation. Science, 149, 269-274.

Zaragoza, M. S., & Mitchell, K. J. (1996). Repeated exposure to suggestion and the creation of false memories. Psychological Science, 7, 294-300.

Zechmeister, E. B., & Shaughnessy, J. J. (1980). When you know that you know and when you think that you know but you don’t. Bulletin of the Psychonomic Society, 15, 41-44.

 

 

 

21st December 2009

Neon Elephant Award Announcement

Dr. Will Thalheimer, President of Work-Learning Research, announces the winner of the 2009 Neon Elephant Award, given this year to Ruth Clark for her many years in leading the workplace learning-and-performance field with research-based insights and recommendations, and—by so doing—helping to professionalize our field.

Click here to learn more about the Neon Elephant Award…

 

2009 Award Winner – Ruth Clark

Ruth Clark, EdD, a recognized specialist in instructional design and technical training, holds a doctorate in Educational Psychology and Instructional Technology from the University of Southern California. Prior to founding CLARK Training & Consulting, Dr. Clark served as training manager for Southern California Edison. She is past president of the International Society for Performance Improvement and author of six books and numerous articles. Dr. Clark is the 2006 recipient of the Thomas F. Gilbert Distinguished Professional Achievement Award from ISPI. Ruth is the author of numerous books, searchable at Amazon.com under her full name Ruth Colvin Clark, including:

  • The New Virtual Classroom: Evidence-based Guidelines for Synchronous e-Learning,
  • Efficiency in Learning: Evidence-Based Guidelines to Manage Cognitive Load,
  • e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning,
  • Graphics for Learning: Proven Guidelines for Planning, Designing, and Evaluating Visuals in Training Materials,
  • Developing Technical Training: A Structured Approach for Developing Classroom and Computer-based Instructional Materials,
  • Building Expertise: Cognitive Methods for Training and Performance Improvement. Click to purchase it here: Building Expertise: Cognitive Methods for Training and Performance Improvement

In addition to her lifetime of work, she is honored this year for the 3rd edition of her excellent book, published just a little over a year ago, Building Expertise: Cognitive Methods for Training and Performance Improvement. Although this is said to be the 3rd edition, the research cited is fresh and up-to-date. This book may be Ruth’s masterwork. It covers a wide swath of the learning research. It’s written by a research translator at the height of her powers. It’s a must-read (and must-study) for everyone in the field of workplace learning-and-performance.

It’s not easy to examine learning research from refereed scientific journals and compile it so that it is practical for others. The time commitment is incredible, the research skills must be of the highest caliber—and it requires guts and gusto. Some of what the research reveals cuts against the common wisdom. Sometimes it chaffs and brings angst and heat. Ruth’s continuing perseverance over the last three decades is testament to her passion and tenacity. Her work itself is testament to her integrity and skills.

I would imagine that over the last two decades there is no one in our field who has improved the work of as many instructional designers, trainers, and e-learning developers as Ruth Clark. For me, she continues to be a beacon—proof that research-based work is valued by our profession. For our field, Ruth’s work is simply indispensible.

Using evidence-based reasoning and recommendations is not just useful in practice. It is what respected, successful professions are based on. We owe Ruth Clark our most grateful thanks.

 

Click here to learn more about the Neon Elephant Award…

 

 

20th December 2008

Neon Elephant Award Announcement

Dr. Will Thalheimer, President of Work-Learning Research, announces the winner of the 2008 Neon Elephant Award, given this year to Robert Brinkerhoff for developing the Success Case evaluation method and for advocating that learning professionals play a more “courageous” role in their organizations.

Click here to learn more about the Neon Elephant Award…

2008 Award Winner – Robert Brinkerhoff

Robert O. Brinkerhoff, EdD, professor emeritus, Western Michigan University is planning his retirement from his full-time work as a principal consultant and partner at Advantage Performance Group, where he has worked since 2005. His clients include Anglo-American Corp., Bank of America, Pitney Bowes, the Federal Aviation Administration, Dell, and the World Bank. He is the author of numerous books, including Courageous Training (with Tim Mooney), Success Case Method, High Impact Learning, and Telling Training’s Story.

In addition to his lifetime of work, he is honored this year for the development of the Success-Case Method and for his advocacy that learning professionals play a more “courageous” and integral part in organizational performance. Too many of us play order-taker roles when we should be partners in helping our organization/business get results. Brinkerhoff’s insight that overly-complex methodologies are generally ineffective because they can’t be understood easily by stakeholders is one that more thought leaders should embrace.

The Success Case Method, while it can’t provide a complete picture of the training-impact landscape, is an important tool in any training-evaluation toolkit. It embodies two key insights. First, training doesn’t have to prompt all trainees to utilize training successfully to have a major impact on the organization. If one person implements one insight that nets the organization millions of dollars, the overall impact of the training may hinge on that one result—not the average or median result from all the learners. The second key insight embodied in the Success Case Method is the understanding that we ought to be diagnosing the cause of training failures and then working to fix those failures. By diving into deep case analyses of failure instances, we can uncover obstacles and forces that are limiting training impact. To learn about the Success Case Method, see Brinkerhoff’s book “Telling Training’s Story: Evaluation Made Simple, Credible, and Effective.” You can purchase it by clicking here: Telling Training’s Story: Evaluation Made Simple, Credible, and Effective

Brinkerhoff’s latest book, entitled “Courageous Training: Bold Actions for Business Results,” and written with Tim Mooney, build’s on Brinkerhoff’s years of experience in evaluating learning. He has seen how training succeeds and how it fails. He uses wisdom gained from these evaluations to lay out a comprehensive and practical process for going from needs-identification to results. Mooney and Brinkerhoff’s book challenges us as learning professionals to go outside our comfort zones to make true performance impacts. It is an important book in the mold of Wick, Pollock, Jefferson, and Flanagan’s “Six Disciplines of Breakthrough Learning.” You can purchase it by clicking here: Courageous Training: Bold Actions for Business Results (Bk Business)

 

Click here to learn more about the Neon Elephant Award…

 

 

22nd December 2007

Neon Elephant Award Announcement

Dr. Will Thalheimer, President of Work-Learning Research, announces the winners of the 2007 Neon Elephant Award, given this year to Sharon Shrock and Bill Coscarelli for advocating against the use of memorization-level questions in learning measurement and for the use of authentic assessment items, including scenario-based questions, simulations, and real-world skills tests.

Click here to learn more about the Neon Elephant Award…

 

2007 Award Winners – Sharon Shrock, Bill Coscarelli

Sharon Shrock and Bill Coscarelli have worked together for over 25 years building world-class expertise in the area of learning measurement and criterion-referenced test development. Their lifetime of work embodies the values of the Neon Elephant Award.

The third edition of their book Criterion-Referenced Test Development: Technical and Legal Guidelines for Corporate Training, released late this year, is a standard reference in the learning-and-performance field, having won two professional society awards as the year’s outstanding publication. Their workshops and consulting on Level 2 Assessment and Certification are based on sound methodology and real-world expertise.

In addition to their lifetime of work, they are honored this year for their paradigm-altering recommendation that memorization-level questions are inadequate for use in measuring learning as it relates to workplace competence. This recommendation is absolutely stunning given that current practice in evaluating learning relies, almost without exception, on memorization-level questions. That the most highly-regarded text on criterion-referenced testing advocates for this change—especially given its focus on legal liabilities—should send shockwaves through our industry.

Bios and more information about Sharon and Bill can be found on their website at www.shrockandcoscarelli.com.

You can read a review of Sharon and Bill’s book on Will Thalheimer’s blog at this link.

You can purchase their book through Amazon.com by clicking on this link: Criterion-referenced Test Development: Technical and Legal Guidelines for Corporate Training

 

Click here to learn more about the Neon Elephant Award…

 

 

22nd December 2006

Neon Elephant Award Announcement

Dr. Will Thalheimer, President of Work-Learning Research, announces the inaugural edition of the Neon Elephant Award, awarded for 2006 to Cal Wick of the Fort Hill Company for leading the development of the first commercially-viable training-follow-through e-learning software and his work as co-author of the book, The Six Disciplines of Breakthrough Learning: How to Turn Training and Development into Business Results.

Click here to learn more about the Neon Elephant Award…

 

2006 Award Winner – Cal Wick

Cal Wick embodies the values of the Neon Elephant Award. In 1999, he founded the Fort Hill Company because of his frustration with the way typical management-training programs produced virtually no impact when the learners returned to their jobs. Cal’s answer to this problem was the development of Friday5s—an e-learning platform that enables training results to be channeled to on-the-job implementation. By tracking learners’ progress toward goals and connecting the learners’ managers and learning administrators to the progress of implementation, Friday5s enables a revolution in how we think about training. No longer is training about information. No longer is it an event. With an online training-follow-through platform, implementation can become part of the training contract. Since the 1980’s the field has moved from training to performance improvement. Cal and his team’s invention provides a bridge between training and performance. It’s not one or the other. The goal is on-the-job performance. Now training can be outfitted to support that performance.

Cal and his team at the Fort Hill Company do more than create and market their products and services. By utilizing results from both research and practice and by honestly evaluating their own on-the-job results, they have been able to build a continuing cycle of improvement in their own work efforts. They’ve also compiled their own learning in The Six Disciplines book, published this past April (2006), making it easy for the rest of us to improve what we’re doing. In the book, which I’ve previously reviewed as “nothing short of revolutionary,” they’ve laid out a coherent and well-tested system for getting training results. It’s not just about training-follow-through software. It’s about an attitude and a complete methodology for getting business results. To get the 2nd edition of the book, click here: The Six Disciplines of Breakthrough Learning: How to Turn Training and Development into Business Results (Pfeiffer Essential Resources for Training and HR Professionals)

 

Click here to learn more about the Neon Elephant Award…

 

The Neon Elephant Award

The Neon Elephant Award is awarded to a person, team, or organization exemplifying enlightenment, integrity, and innovation in the field of workplace learning and performance. Announced in December—during the time of year when the northern hemisphere turns away from darkness toward the light and hope of warmer days to come—the Neon Elephant Award honors those who have truly changed the way we think about the practice of learning and performance improvement. Award winners are selected for demonstrated success in pushing the field forward in significant paradigm-altering ways while maintaining the highest standards of ethics and professionalism. 

 

Why “Neon Elephant?”

The elephant represents learning, power, strength, and the importance of nurturing the community. The glow of neon represents enlightenment, illumination, and a spark of something unique and alluring.

 

Selection Methodology

The award is based purely on merit and the criteria detailed above. Proposals are not accepted, nor are any entrance fees solicited or accepted. While advice on the selection is sought from industry thought leaders, Dr. Will Thalheimer of Work-Learning Research is the final arbiter. Awards will only be made in years when exceptional contributions to the workplace learning and performance field are apparent.

 

Winners

The 2018 Neon Elephant Award, given this year to Clark Quinn for writing a book debunking the learning myths, Millennials, Goldfish & Other Training Misconceptions: Debunking Learning Myths and Superstitions—and for his many years advocating for research-based practices in the workplace learning field.

The 2017 Neon Elephant Award, given this year to Patti Shank for writing and publishing two research-to-practice books this year, Write and Organize for Deeper Learning and Practice and Feedback for Deeper Learning—and for her many years advocating for research-based practices in the workplace learning field.

The 2016 Neon Elephant Award, given this year to Pedro De Bruycere, Paul A. Kirschner, and Casper D. Hulshof for their book Urban Myths about Learning and Education—a book that provides a research-based reality check on the myths and misinformation that float around the learning field.

The 2015 Neon Elephant Award, given this year to Julie Dirksen for her book, Design for How People Learn—a book that wonderfully conveys practical, research-based wisdom through the authentic voice of an experienced instructional designer and strategist.

The 2014 Neon Elephant Award, given this year to Peter C. Brown, Henry L. Roediger III, and Mark A. McDaniel for their book, Make it Stick: The Science of Successful Learning—a book that brilliantly conveys scientific principles of learning in prose that is easy to digest, comprehensive and true in its recommendations, highly-credible, and impossible to ignore or forget.

The 2013 Neon Elephant Award, given this year to Gary Klein for his many years doing research and practice in naturalistic decision making, cognitive task analysis, and insight learning–and for reminding us that real-world explorations of human behavior are essential in enabling us to distill key insights.

The 2012 Neon Elephant Award, given this year to K. Anders Ericsson for his many years conducting research on expertise and creating a body of knowledge that has inspired many others to translate his research into recommendations for use by performance-improvement professionals.

The 2011 Neon Elephant Award, given this year to Jeroen van Merriënboer for his many years conducting research on learning and translating that research into practical models for use by learning professionals.

The 2010 Neon Elephant Award was awarded to Richard E. Clark for his many years in leading the workplace learning-and-performance field by bridging the gap between academic research and practical application.

The 2009 Neon Elephant Award was awarded to Ruth Clark for her many years in leading the workplace learning-and-performance field with research-based insights and recommendations, and—by so doing—helping to professionalize our field.

The 2008 Neon Elephant Award was awarded to Robert Brinkerhoff for developing the Success Case evaluation method and for advocating that learning professionals play a more “courageous” role in their organizations.

The 2007 Neon Elephant Award was awarded to Sharon Shrock and Bill Coscarelli for advocating against the use of memorization-level questions in learning measurement and for the use of authentic assessment items, including scenario-based questions, simulations, and real-world skills tests.

The 2006 Neon Elephant Award was awarded to Cal Wick of the Fort Hill Company for his work developing methodologies and software to support learning transfer.