Updated 1/18/18: Added p-values to last two tables in place of r-squareds.

There’s a glossary of hockey analytics terms and traditional statistical terms at the end of this piece for anyone unfamiliar with things like HDCF% in hockey or r^2 in statistics. If you don’t know those terms, but you’re interested in how teams like the Devils can lose possession battles but win games, scroll allllllll the way down.

The Anomalous Devils

The purpose of a hockey game is to score more goals than your opponent. The most typical approach to this end is to possess the puck and, therefore, likely be in control of the ratio of scoring opportunities. Should that happen repeatedly over the course of the year, that team would be likely to perform favorably in comparison to their competition. The New Jersey Devils, though, are not your typical team. The Devils lose the puck possession battle the vast majority of games (they are 12-26-3 in shot attempt battles), and yet they have a positive goal differential and are 2nd in the division at the halfway point of the season. How has this happened? Well it has a lot to do with a statistic that is intuitive in its inception, but mysterious in its behavior, called High-Danger Scoring Chances (HDC). You see, the typical team produces and allows chances proportional to shot attempts (Ex: 1 dangerous chances per 5.5 shot attempts). But not the Devils. The chart below is using the 5v5 score- and venue-adjusted data from Natural Stat Trick. For anyone who skipped the assigned reading, CF% is percent of shot attempts for as opposed to against (50% is even, at 100% a team gets all the shots, at 0% they get none). HDCF% is the same, except for exclusively high-danger chances. This is an image of the danger, and high-danger areas.

Although the Devils have an adjusted CF% of only 47.82 (25th in the NHL), they have a HDCF% of 54.54% (5th in the NHL). That difference of 6.72% is 2nd largest in the NHL only to the Wild. Note how far away from the trend line the Devils and Wild are — that means their shot attempts are disproportionately dangerous in comparison to their opponents. Conversely, Chicago and Washington have the lowest HDCF%-CF% differentials (-5.33 and -5.69 respectively).

This is clearly an identity the team has and it’s likely that it’s intentional. Is this good? Well, that’s a complicated question. In this article, I will aim to clarify what the tendencies of this statistic are and what the implications of these behaviors are for teams like our Devils.

Team-Level HDCF Behavior

Splitting team seasons into first half, and second half, can we make guesses about the latter, given information from the former.

HDCF% Is Repeatable and Predictive, but Less So Than CF%

So we start of with a little bit of bad news for Devils fans. In order to determine whether a specific stat was repeatable and if it was a good predictor of future goals, I split the data provided in Natural Stat Tricks game logs (example here) into the first 41 games and the last 41 games for every team. I then found how the first half statistics correlated with themselves in the 2nd half (how repeatable were they?) and how well they correlated to GF% -- the percent of goals scored by the team compared to opponents (how predictive were they?). I originally had used data going all the way back to 2008, but was motivated by findings spawned by this dialogue with Woodguy, and confirmation from Natural Stat Trick to use only data from 2010 on. The x-y shot location data is incomplete before 2010. I’ve provided both R and R^2 values for each of the stats discussed in addition to FF% (CF%, but unblocked shots only) and SF% (CF%, but only shots on goal), and SCF% (shots in the “danger” zone — HDCF are a subset of SCF).

As you can see, when we remove events (blocked shots, missed shots) we lose both repeatability and predictivity. Scoring chances actually end up being slightly more predictive than raw attempts because we are choosing which observations to exclude. It still has a pretty high sample size, but successfully sifts out a lot of noise. By the time it gets to high-danger chances, however, too much information is ignored, and the value decreases. That’s not to say it has no value, it just doesn’t have as much value as CF%, and is only slightly better than GF%.

Worth noting here, that my CF% R^2 terms are within 0.01 in predictivity and 0.06 in repeatability of Micah Blake McCurdy’s (of Hockeyviz fame) findings whose results were pretty closely replicated by Dawson (DTMAboutHeart) Sprigings. So these projections seem reasonable in comparison to established norms.

Does the lower comparable repeatability mean that the disproportionate amount of high-danger chances shouldn’t be expected to keep up?

High Danger Chances per Shot Attempt is Repeatable

On average this year, NHL teams record one high-danger scoring chance in every 5.24 shot attempts. The Devils record a high-danger chance in every 4.82 attempts, which is 2nd only to the Rangers (4.15!). Also, the Devils only allow one high-danger chance in every 6.21 attempts which is, once again, 2nd in the NHL, this time to the Wild (6.43). All said, 20.8% of the Devils attempts are high-danger, and only 16.1% of the attempts against them are. That differential is right behind Minnesota for NHL-tops.

But can we keep that up? Well, actually, the ratio of attempts that are high danger definitely significantly repeatable.

These numbers contend with those of raw HDCF60 and HDCA60 with respect to repeatability. This could indicate that the ratio of shot attempts that come from dangerous zones is, at least in part, a product of scheme. This is to say, for example, that John Hynes likely specifically targets high-danger territories. In a bad offensive game, like when the Sharks shut out the Devils in October, NJ would have 27, 5v5 attempts, and 7 high-danger (26%). In a good offensive game, like when the Devils put a 7-spot up against Chicago, they had 44 attempts, 11 of which ere high-danger (25%). Notice that the ratios, 26% and 25% were pretty similar. That’s because the percentage in those case was a product of scheme more so than performance. A good team will limit an oppositions overall attempts, but their tendencies will remain constant.

I’ll plant a flag here that HDCF/CF is only slightly more repeatable than HDCF/60 and less repeatable than CF/60, so there are certainly other possible narratives for why this attribute exists. For instance, HDCF/CF may just be repeatable as a simple consequence of each of its components being repeatable. The preceding paragraph is merely conjecturing one possible explanation.

So, when you get news from John in the December Review, or from Alex this past Saturday, that the Devils Corsi is improving. IMO, that is still definitely good news, even though we are an HDC-focused team. First of all, CF is more reliable and predictive, second of all, it will affect HDCF. You can’t spell HDCF without CF!

Player-Level HDCF Behavior

Okay, so the Devils are trying to be selective with shots attempted and surrendered, but, what about player evaluation? How can we use this to assess whether or not players perform well in this system? I’ll look into player-level behavior in this section, for which most of the work can be found somewhere in this tweet thread.

HDCF%Rel is Repeatable/Predictive for Forwards, but not Defenders

After fiddling around with the stats a bit, I decided to go with relative stats in this section to control for players who switched teams mid-year and remove the cross-correlation between team performance and predictors/responses. The results, sub-divided by position, are shown here:

The defender correlation coefficients are 0.088 for repeatability and 0.092 for predictivity. That translates to p-values of ~0.02 which, depending on your threshold, could be viewed as statistically insignificant. Regardless of threshold, it is far inferior to CF%Rel (r=0.385) and SCF%Rel (r=0.225) and only narrowly better than GF%Rel (0.054)

However, for forwards, there was a significant correlation in all metrics. HDCF%Rel (r=0.186) was much lower than SCF%Rel (0.407) and CF%(0.496) but it was still comfortably in the “statistically significant” territory.

This means that HDCF%Rel is not a particularly good statistic to use for projecting Defenders forward in the year. There’s not enough guarantee that it will a) stay consistent, or b) correlate to goals. So there is not a lot of reliability in saying that Will Butcher, Ben Lovejoy, and Damon Severson will remain the top 3 Devils D-men in HDCF%, or that the stat would power a positive GF%.

High Danger Chances per Shot Attempt is Repeatable

This mirrors what was found at the team-level. It is actually also somewhat intuitive. defenders have more repeatable HDCA/CA (ratio of attempts against that are high danger) and forwards are roughly even.

In all cases, the correlations were significant and the the ratio is repeatable.

So to recap what we know so far, the ratio of high-danger chances that a team produces while a player is on the ice is not particularly repeatable for defenders, but is for forwards. Also, the percentage of attempts that are high danger (both for and against) is repeatable across positions. So if HDCF is less reliable and predictive than CF, why would we use/trust it? Well, that’s where it get’s interesting. One such justifications could be it’s inertia relative with regards to zone starts.

HDCF% is not affected by Zone Start Ratio in Forwards

My original train of thought was aired here on twitter. But the basic idea is that these stats that we use exist on a spectrum. That spectrum goes from possession to goals. The closer we are to the possession end of the spectrum — something like CF%, for instance -- the more it matters where the puck starts. This is because if you win a faceoff after getting an OZS (offensive zone start), you are very likely to get at least 1 shot attempt (CF) out of it. However, the more you move towards the goal end of the spectrum, the less given it is that you would record such an event, because more skill-related events need to fall the right way. After winning the faceoff, you need to move the puck, open up lanes, move without the puck into the lanes, make successful passes, and get the shot off, and score. Only the last aspect of that is taken away for a high-danger chance. You still need to do so much right, that the fact that you started in the offensive zone ends up being of negligible importance. We already know the zone starts are only relevant for about the first 20 seconds of the shift, and high-danger chances take a while to create, so it likely runs up against the limits of beneficial impact.

In order to analyze this, I split the season in half by date (totally for convenience, smarter people should do better things), measured the change in offensive zone start for all players that did not change teams, and the change in other stats like CF%, HDCF%, and GF%. The change in the stat is prefaced by a “d” for delta. dCF% was significantly impacted by dOZS% to the tune of about 1 percentage point per 10% change in OZS%. So, if a player with a 48 CF% increased his OZS% from 30% to 40%, his CF% would be projected to rise to around 49%. Note the pretty tightly paired and obviously positive lines in this scatterplot.

However, that significance drops off quite a bit in dHDCF%, and drops off completely for forwards (p-value ~0.6). The graph for the positions is shown below. Note how the lines have collapsed towards the x-axis here and the orange one for forwards is basically flat. I do find it interesting that defenders, who had a lower p-value for CF%, seemed significantly more resistant to the OZS% change.

And finally, by the time we get to GF%, so much more has to go right other than the zone the play starts in, that OZS% becomes completely irrelevant.

Why is this helpful? Well, although HDCF% is not very repeatable or predictive, it IS still descriptive. HDCF/CF correlates to shooting percentage (r=0.21), HDCA/CA correlates to save percentage (r=-0.15) and the difference correlates to the sum of Sv% and Sh% -- PDO (r=0.17). So when you hear people call PDO the “luck” statistic, that’s not entirely true -- it is partially a product of a team having disproportionately more higher danger chances than their opponents which are more likely to lead to goals. So if it is descriptive of what a players performance has been like, we can contextualize that performance more properly by being aware of these behaviors. Let’s use Steve Santini as an example.

How Do We Use This? The Santini Example.

Steve Santini has absurd usage this year with regards to how often he starts in the defensive zone (Lowest OZS% in NHL among D). Both Dom Luszczyszyn over at The Athletic and your’s truly, have looked into this phenomenon. However, something has happened since those articles ... Steve Santini has figured out how to Andy Greene. Yes, I just used Andy Greene as a verb. What I mean by this is he has figured out how to mitigate the danger of these disadvantageous situations. We can learn this by being aware of the behavior of things like HDCF and HDCA. Santini still has the lowest CF%Rel in the league among defenders. That doesn’t look good. But look closer through the lens of our new information.

Since the start of December, Santini is still 510th/570 qualified skaters in CF%Rel, but the actual results that matter on the ice have been pretty shocking. Despite being lowest on the team in OZS%, Santini leads the team in GF%. That is to say, that despite starting in the defensive zone more than any other player on the team, the Devils outscore their opponent more proportionally with him on the ice than with any other player. Goals are so fluky and rare though, so if we could go up a notch in robustness, without sacrificing too much influence from OZS%, that’d be a good description. As it turns out, Santini is right there at the top of our defenders in HDCF%. He trails only John Moore and Will Butcher, but all of them are between 57% and 58%, which is excellent. We now know that part of the reason that Santini has seen a favorable goal differential despite unfavorable zone starts is that, while he may cede a lot of attempts, very few of them proportionally are dangerous. This is one possible explanation that can add more sophisticated context to his GF% than simply “he is lucky.” It is also possibly a reason why Hynes employs such extreme usage for him.

Concluding Thoughts, Future Work Suggestions, and Feedback

Some of the work I’ve done here are replications of work done by others who are more experienced than I and other work is, at least to my knowledge, new to the literature. My hope is that this will begin a more informed dialogue on what to expect from teams like the Devils moving forward in the season. The next steps to build on what I’ve done here are to iterate the number of predictive games beyond the crude split-half models I utilized, similar in nature to the work of Micah and Dawson. Also using game cutoffs as opposed to dates for players should be investigated. I also only looked into on ice HDCF didn’t investigate iHDCF at all here, but it is worth investigating.

Leaves your thoughts in the comments below. Whether it’s critiques of my work, comments on the Devils, suggestions for future analyses, or just snide remarks. All are welcome. Thanks for reading! Now here’s the aforementioned Glossary.

...

...

...

Glossary

Glossary of Hockey Terms: This article makes heavy use of the hockey statistics CF and HDCF which mean shot attempts and high danger attempts, respectively. Context statistics such as OZS% -- the percentage of shifts started in the offensive zone by a player -- will also be referenced. In addition you will see relative stats which compare a players performance to that of his team, per60 stats which is a version of a per minute measure, and score/venue adjusted stats, explanations for which can be found here.

Glossary of Statistical Terms: I reference statistics such as correlation (r) which is on a scale from -1 to 1 where -1 is perfect inverse correlation, 1 is perfect positive correlation, and 0 is noise; and p-values which are the probability something happens randomly — a low p-value indicates a response is likely caused by some predictor or set of predictors. In referencing those values, I will use E-notation which is scientific notation (Ex: 4.56E-3 = 4.54 * 10^-3 = 0.00456). It will also use the terms “repeatable,” meaning a statistic is likely to more likely to stay consistent over a year and “predictive,” meaning it is helpful in predicting something of value in the future — in our case, goals.

If you are reading this and there is another word that I used somewhere that you didn’t understand, say so in the comments and I will add it to the glossary.