Evidence that Bullet-Points don’t Work

At last, we have some scientifically rigorous evidence to show that slides full of bullet-points don’t work. The research is the work of Chris Atherton, a cognitive psychologist. Chris recently delivered a presentation at the Technical Communication UK Conference and has put up her slides on slideshare. There’s been a tremendous amount of interest in them, but as they were designed to complement Chris’s talk – they only tell half the story. In this post I’ll explain the findings of Chris’s research. I’ve written the post based on Chris’s slides and asked Chris to comment on various aspects. Chris has also reviewed this post to make sure I’ve got all the science right.

The research

Chris tested the effects of using two different types of PowerPoint slides in a presentation. Students were randomly assigned to two groups. One group attended a presentation with traditional bullet-point slides (with the occasional diagram) and the second group attended a presentation with what Chris calls “sparse slides”, which contained the same diagrams, but minimized the amount of text, and broke up the information over several different slides. Both presentations were accompanied by the same spoken narrative. Here are samples of the slides used: A. Traditional bullet point with the occasional diagram

B. Sparse slides

Chris tested the students’ learning in two ways – multiple choice questions and short essay answers. There was no significant difference between the groups on the multiple choice questions. Chris comments:

This is most likely because it’s not very hard to pick out the correct answer from among distractors when you have only recently been exposed to the material and your memory of it is quite fresh.

Before marking the short essay answers, Chris worked with two independent people to identify the themes of information in the presentation. They identified around 30 themes by consensus. The short essay answers were then marked by counting how many of those themes the students wrote about.

As you can see the students who were in the presentation with the sparse slides did much better than those who saw traditional slides.

Theory behind the research

There are a number of theories which can be used to explain these results (if you’re not interested in the theories, scroll down to the next section “What does this mean for your presentations?”):

1. The limitations of working memory

Even the students who did well in recalling themes, remembered only 6-7 themes out of a possible 30. Chris suggests this is due to the limitations of our working memory. Recent work (Cowan 2001) has estimated working memory capacity to be around 4 chunks of information:

2. Two processing pathways

The brain has two major pathways for processing information.

The auditory cortex and the areas around it are involved in processing language – both spoken and written. When a presenter uses bullet-point slides, they’re not using both pathways as effectively as they could. The audience member has to read the words on the slide and listen to the presenter at the same time, leading to overloading of the language areas whilst leaving the visual cortex with very little to do:

Chris notes:

The visual cortex is involved in reading the words on the screen –  it works on the lines and features to assemble the words that are being read, but it’s not really being used to the full, since there’s usually little color or texture information.

3. Cognitive load

The theory of cognitive load was developed by John Sweller. Cognitive load is the amount of work required to understand or learn something. There are two main types:

1. Intrinsic cognitive load – how inherently difficult something is.
2. Extraneous cognitive load – extra work imposed by the thinking/learning environment.

Chris suggests that the sparse slides may minimize extraneous cognitive load by creating fewer competing demands on attention — that is, because we don’t need to spend very long processing the visual elements, we have more attention for what the speaker is saying. She adds:

Having anything on a screen invites people to look at it, the same way their gaze would keep returning to a TV screen in a pub. Since you can’t control the audience’s visual attention, it’s all about controlling what visual information you make available at any given moment, and minimising what is there so it’s not distracting from the spoken narrative, while also ensuring that it is congruent with what you are actually saying.

4. Better encoding of information

Encoding is the process of putting something into your memory. McDaniel and colleagues have shown that a little more effort at the encoding stage can be beneficial to learning. Chris suggests:

Sparse visual cues could lead to better encoding of information — that is, having to work a little bit harder to integrate the speaker’s narrative with the pictures might actually improve our storage of the information (obviously this is only true up to a point; having to work too hard at integrating the two could actually be counterproductive, effectively producing a situation with high extraneous cognitive load).

What does this mean for your presentations?

1. Don’t say too much

Limit what you cover in a presentation. Your audience has limited capacity to take it in.

2. Split the load

Take advantage of the brain’s two pathways. Design your slides so that they can be processed quickly by the visual cortex, allowing the language areas to focus on what you’re saying. This means using more pictures and as few words as you think you can get away with.

3. Get rid of visual clutter on your slides

Do what you can to minimize the extraneous cognitive load on your audience. For example:

1. Only put on your slides things you want the audience to focus on.
2. Split information between slides rather than having it all on one slide, so that you can direct the audience’s attention where you want it.

4. Make your audience work

There’s some evidence that making your audience work a little to understand your point will make your point stick better. A big caveat to this is that obviously you mustn’t make it so hard that they don’t get your point at all. Some ways of doing this are to:

1. Show a picture that the audience has difficulty relating to what you’re saying. Either ask them to guess the relationship, or explain the relationship to them.
2. Show them the axes of a graph, and ask your audience to guess the way the data goes (give enough clues that they’re fairly likely to get it right – without making it too easy).

Thank you Chris

It’s great to have some solid experimental evidence on the use of slides in a live presentation, to back-up what so many presentation authors, trainers and coaches have been saying. I’m also deeply indebted to Chris for her help with this post. I’m looking forward to whatever research Chris does next.