What You Practice Matters

Marksmanship practice doesn't improve scenario performance

Publication

Olma, J., Sutter, C. & Sülzenbrück, S.(2024). When failure is not an option: a police firearms training concept for improving decision-making in shoot/don’t shoot scenarios. Frontiers in Psychology. 15:1335892. doi: 10.3389/fpsyg.2024.1335892

What Was the Issue?

It is critical that police make correct shoot/don’t shoot decisions in the rare situations where they are called upon to do so. It is also important that when officers use deadly force they act quickly and shoot accurately. This paper explores the impact of a short training program on officer decision making, speed, and accuracy in simulated deadly force situations. Although they didn’t state it this way, the key issue was whether task based training produced better outcomes than marksmanship training.

How Did They Look at It?

The researchers recruited 45 experienced police officers for this study. This was a true experiment where half of the participants were randomly assigned to one condition and half to the other. The core test of this study involved using Four-Component Instructional Design (4C/ID) Model for training or not. As the name implies, the model consists of 4 parts. The first is learning tasks. These are the real-life meaningful tasks that the learner must perform. They are presented in a sequence from simple to complex. The second is supportive information. This is information that helps the learners to perform the tasks and understand the principles behind them. This information includes theoretical knowledge, cognitive strategies, and mental models. The third is just-in-time information.  This is feedback given during the performance of the tasks and can also be step-by-step instructions given when performing the task. The fourth is part-task practice. This focuses on practicing skills or sub-skills to develop a high level of automaticity. Because of the short training duration of the study, this step was omitted from the training. Many of you have probably either been trained or designed training that is at least loosely based on this model.

Both groups completed the same pre and posttests. The pretest was completed before training and the posttest after. The first task on these tests was to shoot 2 large circles and two small circles as quickly as possible. Participants could shoot these circles in any order they wanted. Scores on this test were computed using the hit factor that is commonly used in competitive shooting (number of hits divided by the time it took). The second part involved the officers watching 3 realistic videos where they interacted with a suspect and had to make shoot/don’t shoot decisions. The researchers only counted the first two videos of the series (one was a shoot and the other a don’t shoot scenario) and used the third scenario so that the participants couldn’t guess how many shoot or don’t shoot scenarios there were across the pre and posttests. The outcomes of this test were number of correct decisions, response time (how quickly the first shot was fired in shoot scenarios), first hit (how quickly the first shot to hit the suspect landed), muzzle position (how long was the muzzle blocking the participant’s line of sight), and closed eyes (how long the participant closed at least one eye before shooting).

For both groups, the first part of the training was conducted using videos, which served as the supportive information component of the 4C/ID model. In the experimental group, the key factors for officer success were identified as situational awareness, tactical gaze control (focusing on what is important), and visual perception. The training videos covered the fundamentals of human perception and its distortions, situational awareness and what to expect in certain situations. Participants were instructed to focus on the suspect's hands, keep the muzzle of the gun low to maintain a clear view of the suspect until a decision to shoot was made, and understand the speed-accuracy trade-off in response times.

In the control group, the training video focused on the speed-accuracy trade-off with the objective of the training being to increase speed while maintaining accuracy. The session included information on the process of shooting, different shooting modes, the optimal use of sights, and optimal movement of the handgun.

The training for the experimental group involved 4 tasks that increased in complexity as the training progressed. The first involved shooting 6 rounds into a torso-shaped target. The second involved shooting at one of six circles when indicated. The third involved shooting two rounds at a human silhouette when the participant saw the target was armed. The fourth used life-sized pictures and required the officers to shoot the targets if the suspect was armed. During the training, participants were given feedback and input from a trainer (just-in-time information from the 4C/ID model).

In the control condition, the training involved the participants shooting a variety of abstract targets in different sequences. It is unclear if they received feedback from a trainer or not, but I suspect they did.

What Did They Find?

Hit factor improved a little bit for both groups across the tests. The authors had hypothesized that the training given to the control group should improve their hit factor (because it focused on quick target shooting), but thought that the hit factor for the experimental group would stay the same. I’ll come back to this in the So What section.

There was not a statistically significant difference in decisions by group from pretest to posttest. This is because both groups made mostly correct decisions across the phases. The control group made 2 mistakes on the pretest and the experimental group made 5. Neither group made any mistakes on the posttest. A more difficult set of test materials might have been able to ferret out a difference between groups (if there really was one).

The analysis for response time and first hit are presented below. You can see that the experimental group got faster overall and scored their first hit faster than the control group after training. These differences were moderate in size.

The control group increased the amount of time that their muzzle was blocking their sight from pre to posttest and the experimental group decreased it, but the differences were not statistically significant, and the effect sizes were small.

Few members of either group closed an eye before firing, so meaningful analysis was not possible.

In additional analysis, the researchers found that the experimental group fired more shots and was more accurate on the posttest than the control group.

So What?

This is an interesting study that suggests that even a brief training course (less than an hour in this case) can significantly improve performance – even for experienced officers. We don’t know if these improvements represent real changes in skill because we don’t know if the improvements were retained over time, but at least the initial boost was there. I also really like the use of hit factor to score proficiency. It is a really nice way to incorporate both speed and accuracy into the assessment of performance.

I like that the authors used a validated training framework, but this really isn’t a full test of the framework because they didn’t incorporate the forth part of the 4C/ID model. But it appeared to work even with only 3 parts being applied.

Now let me put on my EcoD hat for a minute. If we ignore the other parts of the training process and focus only on the training tasks, we see that if you want people to get good at something, you have them practice doing that thing. That is the only way to create the perception-action loops that allow for success in the real environment. The experimental group got relevant practice. The control group did not. They got marksmanship practice.

If you want your officers to be experts at quickly shooting a set number of holes in pieces of paper when a tone sounds or someone yells “threat”, you should do a lot of regular range shooting. If you want officers who can make decisions about when to shoot and then shoot effectively, you must put them in those situations. At the very least this means having shoot/don’t shoot targets on the range and these targets must be differentiated by being armed or not. They can’t be X’d out or different colors or different shapes. Ultimately, your officers’ training should involve live fire in a shoot house and scenario-based force-on-force.

I also doubt that the supportive information mattered in this case (or really matters in most cases). Telling officers to focus on the suspect’s hands is at least an attempt to get the officers to focus on relevant information and is also focusing them externally, but experienced officers have probably been told this before. Also, people will quickly figure this out without being explicitly told - as they did in Dr. Martaindale’s study. Much of the supporting information also appears to have been internally focused (keep the muzzle low and your eyes open) for example. Internally focused instructions have been consistently shown to be less effective than externally focused ones. In many cases, internally focused cues appear to hamper performance.

Much of the just-in-time information also appears to be internally focused, so I doubt it helped participants much either. 

I can’t prove this with the data in this article, but I suspect the training outcomes would have been even better if all the time was devoted to the training tasks instead of including the supporting information. I also suspect they would be better without the just-in-time information (or with the information delivered in a different way). Maybe we’ll run that study in the future.

One final observation. Proponents of traditional training often claim that you must teach students the fundamentals before you can move on to more complex things. When they say fundamentals, they usually mean specific physical actions done in a particular way. This is the right way to kick a ball. Or in firearms, this is the right way to aim the gun. Only after they can demonstrate that they do the fundamentals the “right way” can we implement more complex game like or real life like elements. Yet, there are a growing body of findings in the EcoD literature that suggest playing more complex games without focusing on the fundamentals first causes people to develop the fundamentals anyway. The data also suggest that these fundamentals may be better retained.  This article – provides an example of this is soccer.

These data might provide another example. The experimental group focused on more complex decision making tasks in their training, and the control group focused on marksmanship skills. In the post test, both groups did better on their marksmanship skills and the experimental group did just as well as the control group. In this study, it is of course possible that the groups did better simply because of being more familiar with the task, and this is the interpretation that the researchers gave the results. But it could also be possible that the experimental group got better at the fundamentals by practicing more complex tasks.