An All About the Jersey Hockey Stat Primer: Goaltending

A lot of the posts in this series have focused on the skaters in hockey. This is largely because there is more information to work with among other issues. However, this does not mean goaltenders are worth ignoring. They are very valuable. Without good goaltending, an otherwise very good team could end up performing very poorly. Likewise, having good or even great goaltending can elevate bad and mediocre teams in games and in the standings. It is a very tricky position as failure almost always costs the team but true success is not easy to identify until you already have seen enough of it. This primer will go into how goalies can be evaluated from a statistical standpoint. As with previous ones, I will incorporate New Jersey Devils-related examples as needed.

The Basic Stat that Still Matters: Save Percentage

Goaltenders have been defined by four stats for a very long time. Their record or wins, their goals against average, how many shutouts they have, and their save percentage. Out of these four, only the save percentage is the only one that really tells us how well the goaltender did at stopping shots. It is the only one that takes into account how often the goaltender makes saves. This matters because a goaltender’s primary job is to stop shots. Their mission on the ice is to make saves in order to keep the other team from scoring.

The other three stats do not take saves into account. A goaltender’s record is more indicative of how the team performs as opposed to the goaltender. A bad team will sink a goaltender’s record regardless of how well they do at their job; and a very good team will boost a goaltender’s record regardless of how bad they are at their job. Goals against average is literally just the number of goals allowed times 60 minutes and then divided by how many minutes the goaltender played. There are no saves in this formulation. While a shutout means the goaltender put up a perfect 100% save percentage, it is only a count of how many times that happened. It does not tell us anything other than that the goaltender had the best possible performance some number of times. They may have their own meaning, but they do not when it comes to figuring out whether a goaltender is great or awful at their primary job.

Calculating a save percentage is simple. It is the number of saves made by the goaltender divided by the number of shots they faced. However, there is a bit more to it when it comes to utilizing it. I’ve written about it in a previous primer, but here is a short summary.

Save percentages are definitely influenced by the game situation. From the 2019-20 season, here were the ranges of save percentages as per Natural Stat Trick:

• All Situations: 88.63% to 92.06%
• 5-on-5 Only: 90.12% to 93.38%
• Power Plays: 83.08% to 98.51%
• Penalty Kills: 83.45% to 90.61%

If you see a general “Sv%” for save percentage, then it is usually referring to overall save percentage which is for all situations. The issue there is that special teams really drag on a save percentage. There is a lot of variation within them since teams do not have the same number of power plays and penalty kills in games and they are played differently than at even strength. Goaltenders in a penalty killing situation will typically be facing the opposition’s best offensive players and have to handle situations where the opposition is often in their end of the rink and trying to set up a specific play or shot to score. Goaltenders during a power play situation often do not see shots, but when they do, they are often very dangerous ones such as breakaways or odd-man rushes. If you want to get a handle on how well the goaltender does without being swayed by special teams, then you need to look at the goaltender’s even strength (5-on-5, 4-on-4, 3-on-3) save percentage like at NHL.com or at their 5-on-5 save percentage like you can at Natural Stat Trick.

For a Devils example, here are the saves by strength by the Devils’ goaltenders from last season at NHL.com.

NHL.com

Notice how MacKenzie Blackwood’s overall save percentage of 91.5% belies a very good 92.4% even strength save percentage. Why is that? It is because of his 86.5% save percentage against power plays and 85.7% save percentage against penalty kills. You can see from the number of saves attempted (SA) that Blackwood faced a lot more at even strength than he did in either special team situation. Due to the Law of Large Numbers, Blackwood’s performance at even strength is more likely to be representative of his true performance than either his power play or penalty killing results.

By the way, out of the 66 goaltenders who played in at least ten games last season, Blackwood’s 86.5% save percentage against power plays ranked 35th per NHL.com. That was still ahead of other goalies like Jordan Binnington, Corey Crawford, and Anton Khudobin. Blackwood’s 85.7% save percentage against penalty kills ranked 59th out of 66 and better than both Ilja Samsonov and Phillip Grubauer. Some very, very good goaltenders just have low save percentages on special teams in some seasons. There is that much variation in the process. It is likely these will be very different next season due to the relatively few amount of shots taken as well as the kind of shots taken. I would not worry about Blackwood on special teams unless he does this every season. And even then, I would question how the team is doing on each before the goaltender. In other words, if you want to see how a goaltender is doing or how a goaltender compares to another, then you are better off using even strength or 5-on-5 save percentage to do that.

Quality Starts

One of the common questions about a goaltender is whether or not they had a good start. Robert Vollman came up with a quick method to do that: Quality starts. A goaltender had a “quality start” if they had a game where the starting goaltender put up a save percentage higher than league average or if the goaltender faced 20 or fewer shots and posted up a save percentage better than a replacement level goaltender (this is usually 88.5%). Likewise, if a goaltender has a save percentage below 85% in a game, then they had a “Really Bad Start.” It is a simple method to judge how often the goaltender had a good game.

The approach is simple although the issue is that, like shutouts, it just counts how many games this was achieved. It does not really tell us how the goaltender does as a whole. It can be useful to identify whether a goaltender’s save percentage is driven by having plenty of good starts or was undercut by a number of really bad ones. For example, per Hockey-Reference, Blackwood had 22 quality starts out of his 43 started games last season. Only 12 goalies had more than that last season. As good as that it ranks, it only really tells us that Blackwood did very well in a little more than half of his starts.

With the development and promotion of expected goals models, this has gone by the wayside. Given that we can look up the expected goals value for a team and then the score, we can quickly see whether the goaltender may have had a rough night on the surface.

Goals Saved Above Average

For the purposes of seeing whether a goaltender is good, we often want to compare the goaltender’s stat to others. We can see how they rank in the league among all goaltenders or a filtered group of goaltenders such as all who have played at least 10 games or 1,000 minutes in a given season. Another way to do this is to compare it to the league average and Goals Saved Above Average (GSAA) is a simple way to do that.

The concept is in the name itself. GSAA takes the league’s save percentage and applies to how many shots the actual goaltender has faced to determine how many goals an average goaltender would have allowed if they faced the same workload. Then that number is subtracted from how many goals the actual goaltender did allow. The result will either be positive or negative and it will represent how many goals the actual goaltender saved above the calculated “league average” goaltender. (Zero would mean they are the league average goaltender.) It is a number that can be used to gauge whether or not a goaltender is better than league average or not. And as with a lot of these stats, you can easily turn it into a rate (like per-60 minutes) to better compare goaltenders with very different time on ice values.

One of the big benefits of GSAA is its application to history. Whereas shot attempts have only been recorded since 2006, save percentages go much further back in hockey history. It can and has been determined for many goaltenders of the past. You can see GSAA at Hockey-Reference for Martin Brodeur’s entire career and most of Jacques Plante’s career.

The other big benefit is that is quick to judge. For save percentage, I typically have to look at how it compares with other goaltenders unless it is obviously high (like 94%) or low (below 89%). With GSAA, the number tells me right away whether the goaltender is doing well or not. For all goalies in all situations last season at Natural Stat Trick, Blackwood’s GSAA of 7.34 immediately tells me that he is doing much better than a league average goaltender.

There are drawbacks for GSAA. For one, it is not always calculated the exact same way at each site. At Natural Stat Trick, for example, it is based on what you filter out for goaltenders. While we expect this when we change the situation from all situations to 5-on-5 only, I have to remember that other filters matter too. If keep the filter to all situations but look at just New Jersey, then Blackwood’s GSAA changes to 14.87. The other goaltenders see a change to their GSAA too: Cory Schneider’s -7.89 becomes -6.05; Louie Domingue’s -11.74 becomes -8.74; and Senn’s -0.29 becomes -0.03. All because of a simple change of the filter. But if you go to another site where that is not a concern like Hockey-Reference, their GSAAs for the Devils goaltenders are a bit different: 7.29 for Blackwood, -7.90 for Schneider, and -10.32 for Domingue. The general conclusions are the same: Blackwood was quite good last season and Schneider and Domingue were quite bad. But the values changing is auspicious.

The bigger drawback of GSAA is that it does not really tell us anything about the goaltender’s performance beyond their results. Whether or not the goaltender faced more difficult shots than another does not matter. It treats all shots as the same, as does a simple save percentage. However, we have a better way to account for that.

Expected Goals & Goals Saved Above Average Expected

I did a whole primer on scoring chances and expected goals. I have touched on goaltending there, but they are important to highlight here. Those two concepts have been used to strengthen what we know about goaltenders. Just as we can count scoring chances based on shot location in the NHL’s play by play log metadata, we can determine how many saves the goaltender made on those shots. The bigger sea change has come with expected goals models.

Just as we determine the value of a shot attempt and sum it up for a skater’s individual expected goal amount from an expected goals model, we can determine how many goals a goaltender was expected to allow. Instead of calculating xG for a skater, it would be xGA - expected goals against - for the goaltender.

This was a big boon for goaltender analysis. Applying an expected goals model for goaltenders means we can take into account the danger of a shot. A goaltender who faces more challenging shots than others will not be as punished. Especially if they get beat by those more challenging shots. I would still recommend using 5-on-5 or even strength situations only for it as to not get swayed by special teams, but you could get away with all situations in so much that all shots are given value in all situations.

Through this method, we can get a better understanding of how the goaltender is performing in general as well. Maybe they are doing better than we may think by their save percentage or GSAA. Maybe they are doing worse. How did the Devils do by it last season? According to Natural Stat Trick for 5-on-5 situations:

Natural Stat Trick

Here you can see Blackwood allowed about as many goals as he was expected to allow. You cannot give up 0.74 of a goal, so rounding that up to 85 would mean Blackwood met expectations. Even if he gave up a few more or a few less, we can conclude that Blackwood did not significantly over-perform or under-perform based on the model. We cannot say the same for Domingue or Schneider. Schneider should have done a bit better and Domingue should have done a lot better than what they have actually done. Through this method, we can conclude (among other things) that the Devils really needed to move on from Domingue and Schneider. They did albeit after the damage was done. We can also conclude that if the Devils wanted Blackwood to allow fewer goals, then they should focus on what kind of shots they were allowing. This is another benefit of using an expected goals model: good goaltenders on bad defensive teams are not punished as much as they may be by save percentage or GSAA.

Taking it up another level, there is Goals Saved Above Expected (GSAx). This is simply the difference between expected goals against and actual goals against. Given that different sites do expected goals differently - the models at Evolving-Hockey, Natural Stat Trick, and Moneypuck for example are a bit different from each other - the GSAx is going to be a bit different from site to site. While Natural Stat Trick does not have GSAx listed as a separate stat, we can see that Blackwood’s would be less than one. Moneypuck does not have GSAx but does have save percentage above expectation. Charting Hockey, a visualization site that uses Moneypuck’s data, does have GSAx and lists Blackwood as being positive (and one of the few that was positive last season). It still has the main benefit of GSAA as it is a simple value that immediately tells you whether the goalie is doing well, while the use of a expected goals model addresses the lack of context within GSAA.

Of course, this also means the drawbacks of expected goals models also apply. Perhaps moreso with goaltenders than with skaters. While expected goals models differ from person to person or site to site, they are generally based around shot location and other shot-related data that is recorded. However, the assumption about rebounds really hits home - to a point where Moneypuck’s site differentiates them from the save percentage. Models may underrate how likely a rebound is to go in, something that Cole Anderson wrote about back in 2018 at his blog Game Theory (which appears dormant today).

Model also do not include shot placement or pre-shot movement since there is no currently available data for either short of tracking it yourself. Both are big factors in terms of whether a goaltender can make a save and how difficult it may be. You and I may see a pass from, say, Nico Hischier across the slot to Kyle Palmieri, who slams in a one-timer and conclude that was a dangerous chance. But expected goals only looks at where Palmieri shoots it and that may undervalue the shot. Until this kind of data becomes available, we just have to deal with the models we have. Even if they do underrate some of the shots that you and I may see as near impossible for goaltenders to stop.

Observations Still Matter

Ultimately, the statistics available for goaltenders is still limited. As such, this is an area of the game that definitely requires a well-trained eye to fully evaluate. Similar to football players, their mechanics are especially important to look for. How a goaltender positions, how they get into stances, how quickly they move within the crease as well as react within it, how they handle screens, and how they handle rebounds are all important to determine whether a goaltender has the skills and the technique to play. It can identify weakpoints in a goaltender’s game as well recognize whether a prospective goaltender has a future in pro hockey.

Ideally, we would have data to support this. But all we have are the number of saves they make and limited information about the shots they face. As gains are being made with respect to measuring pre-shot movement, expected goals models and other methods for measuring goaltending performance may improve. If there is a breakthrough to measure shot placement beyond someone manually tracking it, then we could have more. As much as the “eye test” is correctly derided as a way to solely judge a player, a team, or a game, goaltending is an area where you need it to provide the full picture. The challenge is that recognizing goaltending mechanics is a challenge. I fully respect those who are able to do so and communicate to the fans, the players, the coaches, and the management involved with the goaltender.

What’s Next & Your Take

Goaltending is a tricky position. A goaltender can keep their team in the game and secure a win for their team. But they do not score goals, so they cannot win them entirely. A very good goaltender can elevate a team to perform above their level. However, it takes time to identify whether a goaltender is legitimately good and if they struggle at the beginning, then it could undercut a chance at a full career. While a skater can have a bad game and not necessarily cause their team to lose, a goaltender having a bad game usually means the team loses. And it is not uncommon for teams to bet big on someone having a few good seasons only to find out that variation was going to swing the other way for the goaltender. It is a vital position for any team, but with only two spots per team, there may be more players than available spots - and combined with teams being burnt in the past - it does not garner a lot of the big money deals you may see a top forward or defenseman get.

The stats available for goaltending that should be used should be the ones centered around their main job: stopping shots. The basic ones of the past still hold value and there can be uses for quality starts and GSAA. However, more advanced ones that involve expected goals models or even just filtering out high dangers shots are providing a more sophisticated look at how a goaltender is performing. I encourage their use and I do hope they evolve as the data available for expected goal models also evolves.

In the meantime, I would like to know what you think and what you have learned about goaltending and their stats. Please provide any further questions about these stats in the comments. Next time, I will go into two general and major drawbacks for all hockey stats. If you have been reading these primers, then you might have a good idea as to what they are. Thank you for reading.