Evidence-Informed Learning Design

Creating Training to Improve Performance

Mirjam Neelen, Paul A. Kirschner

• Publisher: Kogan Page
• Year: 2020
• ISBN:
  9781789661422
• Edition: 1
• Topic: Business
• Subtopic: Human Resource Management
• Index: Business

Learning and Development (L&D) programmes are too often based on fads, the latest trends or learning designers' personal preferences without critical evaluation. Evidence-Informed Learning Design allows learning professionals to move away from this type of approach by showing them how to assess and apply relevant scientific literature, learning science research and proven learning techniques to design their training in a way that will make a measurable difference to employee performance and overall business success. Packed with tips, tools and examples, Evidence-Informed Learning Design enables L&D and training professionals to save both time and money by ensuring that efforts are focused on designing learning that's proven to be effective. Covering techniques like interleaving and self-directed and self-regulated learning, as well as debunking myths and fallacies in the field, it covers how best to test, measure and reinforce learning in both online, offline and face-to-face scenarios. To ensure that employees develop the skills the business needs to succeed and that the L&D function is recognised as adding true organizational value, this book is essential reading for anyone responsible for designing learning.

1

Designing learning experiences in an evidence-informed way

We draw a distinction between evidence-based and evidence-informed, as there’s a difference.

Evidence-based practice is an interdisciplinary approach to clinical practice, grounded in medicine. Sackett et al (1996) see it as a three-legged stool integrating three basic principles (Figure 1.1):

1. the best available research evidence bearing on whether and why a treatment works
2. clinical expertise of the health care professional (clinical judgement and experience) to rapidly identify each patient’s unique health state and diagnosis, their individual risks and benefits of potential interventions
3. client preferences and values.

Figure 1.1 Evidence-based practice

For example, if a decision is made on the dosage, intake, and working of a medicine, then it means that it was tested and approved for a specific sickness or condition and a specific part of the population (eg a person aged between 30 and 50, with a healthy BMI and symptoms X, Y, and Z). The instructions to take that medicine in the morning on an empty stomach allows for a wide range of specific circumstances (at home, in the car, on the beach, when and wherever as long as it’s on an empty stomach in the morning). The problem is that for learning, this doesn’t work.

Evidence-informed still means ‘based on scientific research’. However, in the field of the learning sciences we’re dealing with muddy real-life things, also known as ‘variables’, that can influence what we want to achieve and whether it’s achieved. In our context of designing a learning experience, this is an intervention’s effects. Our field simply doesn’t always allow for ‘straightforward measurement’. We simply can’t usually deliver the same quality of evidence as clinical practice does. This is because in learning environments, we’re dealing with many different variables that interact and are hard to control. Literally, what worked in a workshop or lesson today will not necessarily work in the same workshop or lesson that afternoon, one day later or three months later. Just the fact that the learners are different, with different prior knowledge, different beliefs, different needs and/or different motivations to participate, can change everything. And then there are also environmental factors. Take the room where the instruction or training takes place, for example. If the first room has the ‘right’ temperature and the second is too hot or too cold, it can impact the learning experience. Or, if the first cohort is local (eg either from the same company/department and/or in the same room) and the second global (eg from different companies and/or spread throughout the globe), this will impact how the learners interact with each other and with an instructor, and, thus, how they’ll learn. Hence, when we use evidence, we need to acknowledge that what works in one context might not necessarily work in another.

Also, we often use more qualitative data and this type of data provides weaker evidence than quantitative data. Quantitative data are numerical. They’re about ‘hard’ numbers, measurable variables that can be used for mathematical calculations and statistical analyses. Examples are people’s weight or height, or the number of times they took a medicine, or the score on an achievement test. It’s important to note here that numerical doesn’t mean that it’s per se exact, reliable or valid. A self-report on a Likert scale (eg how much did you think you learned: 1 = very little through 5 = a lot) yields numerical data but is, of course, subjective and thus not very reliable.

Qualitative data are information about characteristics, properties or ‘qualities’, such as quality of the facilitator, engagement, or perceived usefulness of a training. This type of data describes but doesn’t define. It approximates but doesn’t measure the actual attributes (eg how much you think you learned as opposed how much you actually learned). A third problem – and this can be either quantitative or qualitative – is that we sometimes can’t measure what we need to measure and then we use a proxy. For example, in a learning context, we often ask learners if they were engaged or we observe them and rate their engagement. When they indeed felt engaged or were observed as being engaged, we take that as a positive proxy for learning. However, in reality, we haven’t measured learning, we’ve measured either how busy they were (but with what?) and/or their perception of their own engagement. In other words, in our field, the evidence will alert us to what might work and under what conditions (ie it will inform us as to whether it might work, when it might work, how it might work etc), however, this is no guarantee that it will work!

This nuanced description of the difficulties with using scientific evidence in our field might suggest that it’s too difficult or not worth it. This is definitely not the case. Although we need to acknowledge that in our field the evidence is usually weaker than in a field like medicine (though placebos have an effect in medicine and we measure pain via subjective rating scales) or physics, this doesn’t mean we shouldn’t use the evidence. On the contrary. This whole book is about why we should use the evidence that’s available to us. We should use it based on our practical wisdom and based on the context we work in. This combination will help us explain why we make certain decisions and it will make us better practitioners overall.

There are also similarities to medicine. Here too we can speak of a three-legged stool. However, we can also use other types of evidence to decide what works best to achieve a certain goal through a learning experience. Examples are input from learners and stakeholders, data from systems that might be used in the workplace, and of course our own expertise as learning professionals (see Figure 1.2).

Figure 1.2 Evidence-informed practice for learning experience design

Figure 1.2 details

The upper circle is labeled Highest-quality scientific evidence, the lower left is labeled Learning experience, design experience, and the lower right is labeled Input from learners, other stakeholders, systems, etc.

In this book, we focus on the scientific evidence, as this is underused when it comes to designing learning experiences and, in particular, evidence from the learning sciences: an interdisciplinary field focused on developing a deeper scientific understanding of learning. We explore its history and meaning, as well how it’s useful to us as learning professionals, in the next section.

Although the tide is turning in that more professionals in our field are beginning to recognize that scientific evidence needs to be leveraged to make informed design decisions, we still too often base our designs on hunches and beliefs (and sometimes worse – based on myths!), aesthetics and learner or stakeholder opinions and preferences. This book is a strong plea for consistently integrating evidence from the learning sciences into our practice so that we can truly support the organization we work for to support their people to continuously learn so that they can do their jobs better.

We’re both members of the Debunker Club, founded by Will Thalheimer and dedicated to eradicating learning myths and sharing proven evidence-informed insights. We’re committed to this ourselves because we agree with the club that, when we design learning experiences based on myths, we’re spending time and money that could be better spent elsewhere. After all, we’re there to help organizations and the people who work there. Possibly even more important from an ethical point of view, we also hurt learners when we incorporate learning myths and misconceptions into our designs. And last, however, not least, it’s also detrimental to the foundation of the learning profession when we base our designs on hype, myths, anecdotes from gurus, silliness, or sexy bells and whistles. We simply must do all we can to debunk myths and misconceptions, learn from research and practice, and share evidence-informed information. We need to be open yet sceptical.

We imagine you looking at us with slight uncertainty, wondering if this means you need to dive into the learning sciences research yourself. We can reassure you that you don’t. Not necessarily, anyway. There are other ways for all of us to put our research hat on to prevent ourselves from getting fooled or to confirm that something we’re reading, seeing or hearing might actually be true. Daniel T Willingham’s steps come to the rescue (Figure 1.3). Let’s see how we can use them to our benefit.

Figure 1.3 Steps towards designing learning experiences in an evidence-informed way

Figure 1.3 details

The steps lead from one to the next in sequence and return to the starting step as follows: Step 1: Strip it and flip it; Step 2: Trace it; Step 3: Analyse it; Step 4: Should I do it? The final step is unnumbered, and labeled Evidence-informed!

Adapted from Willingham (2012a, 2012b)

Steps to start designing learning experiences in an evidence-informed way

Let’s look at an example excerpt from an article and then figure out what we need to do if we follow Willingham’s steps. The following example is based on an existing article; we have adapted it for the purpose of practice and because it’s about informing and not naming and shaming.

EXAMPLE How to design learning experiences for millennials

Joe Doe

Millennials are still the largest generation in the global workforce (Research Centre Y). Organizations need to ask themselves what this means for training their staff. How should the approach differ from previous approaches taken with Baby Boomers or even Gen Xers? Millennials are the future, so it’s critical that organizations design learning strategies that ensure they get the most out of this unique generation.

How millennials are different

Millennials are also known as the ‘digital native’ generation. Because of their lifelong relationship with technology, they learn differently. Don’t bother millennials with lecture-style training; it won’t work. You need to focus on hands-on, experiential learning, as only then will you get the results that your organization need you to deliver.

A study by We Do Research, comparing the learning preferences of 5,000 people in one organization, clearly shows that millennials want to spend 10 per cent less time on in-person training. Evidence clearly shows that millennials prefer authentic learning contexts, enabling them to connect learning to their jobs.

What works best for millennials?

There’s only one simple answer to this question: video. It is the only medium that grabs a millennial’s attention. The average human brain processes video 60,000 times faster than text and a millennial brain even processes it 100,000 time...