Reviewing Potential 2018-19 NHL Scorer Bias in Hits, Blocks, Giveaways, & Takeaways

Last year, I provided a long over-due update to something I originally dug into in 2010. I looked at scorer bias for shots, missed shots, blocked shots, hits, giveaways, and takeaways for both the New Jersey Devils and the NHL as a whole from the previous five seasons. You can check those posts here: Part 1 for hits, giveaways, and takeaways; Part 2 for shots, blocked shots, and missed shots. My conclusion in both posts is that scorer bias is definitely a thing and it has at least some impact in how we analyze the game.

This remains a blind spot for analytics. Sites may have adjustments for venue (and zone and score) but there is yet no public explanation of why the adjustment is accurate or whether the adjustments should change over time. There certainly is no public knowledge in how the game is scored or whether all 31 teams are working off of the same set of definitions or even the tools to do so. This has always been a problem - Tom Awad called it out as early as 2009 and Gabe Desjardens has done the same with respect to shot locations. It is even more of a problem as EvolvingWild’s GAR has taken off to a degree and GAR absolutely uses NHL Real Time Super Stats (RTSS) which includes hits, giveaways, and takeaways. You can read EvolvingWild’s explanation at Hockey-Graphs; Part 2 has the meat and potatoes of what’s in it. Apparently, the second most important factor for a forward’s offense at even strength is taking hits. Not sure if it is to actually take them or avoid them but that’s more valuable than, say, taking a shot on net. And the second most important factor for a forward’s defense at even strength is taking a block - because blocks = defense, right?

Anyway, today I want to point out how RTSS stats worked out in the 2018-19 season across the league. NHL.com collects this data and through their game-by-game set, you can split it up between home and away games. This is important as it will allow us to identify which arenas may have been very loose or stringent with a stat. At a minimum, I will demonstrate that the New Jersey Devils were more physical than you may have thought.

The Method

I collected all of those stats, calculated the league average and standard deviation, and color coded any values that were high or low. These numbers for all situational play; no games were removed. These stats are not necessarily distributed normally but I will assume a normal distribution to prove the larger point of which arenas may be suspect for certain stats. Any stat that is higher than one standard deviation above the mean is highlighted in yellow; any stat that is higher than two standard deviations above the mean is highlighted in orange and is bolded for being notably high. Any stat that is lower than one standard deviation below the mean is highlighted in blue; any stat that is lower than two standard deviations below the mean is highlighted in green and is bolded for being notably low.

NHL.com

Immediately, you can see through the league standard deviations at the bottom that there is less variation among the road stats than the home stats. Which makes sense. A scorer bias at home impacting a team for 40 or 41 games is going to be more significant than two or three games in someone else’s building.

The Devils

This is a New Jersey Devils blog so let us focus on the Devils first. I recognize that the Devils started their season in Sweden last season, so the scorer bias at the Rock really only impacted 40 games instead of 41. The sheer amount of data makes it so that one game is hardly going to make a difference.

Anyone who complained about the Devils being “too soft” last season and “needed to get tougher,” should be pleased by this finding. On the road, the Devils threw the second most hits in the league. They were above the league mean and second only to Los Angeles on the road. The issue was at home. The Devils threw 220 fewer hits; the largest negative gap (home minus away) between home and away hits in the league last season. Were the Devils that much different at home? They were a Bad Team everywhere last season so I doubt that. I think it is more likely that the scorer at the Rock undercounted hits. This appeared to be the case from 2013 to 2018. This seems to me like more evidence in support that the Rock is on the low end for hits.

It also appears to be on the low end for blocks too. The Devils were below the league mean by at least one standard deviation at home with 506 blocks. On the road, though, they blocked 652 shots. That’s a difference of 146 blocks; also the largest negative gap (home minus away) between home and away blocks in the league last season. If the difference was not so stark, then I could buy the idea of the skaters using their body more to deny shots away from home. Instead, I think the Devils made more blocks at home than they received credit for. This was also something I concluded last season when I looked at the past five seasons. The Devils scorer appears to still undercount blocks as well as hits.

There appears to be some more consistency with giveaways. The Devils were credited for 386 giveaways at the Rock compared to 359 away from the Rock. That’s a difference of 27, which is not so large. Neither value was higher or lower than the league mean by a standard deviation, so there is not a lot to suspect. There is a little more to suspect with takeaways. The Devils were credited for 359 takeaways at home and only 271 on the road. The difference of 88 takeaways is on the higher end of the home-away differences in the league last season. While neither value exceeded the league mean, it suggests there could be overcounting of takeaways for the Devils at home compared to when they are on the road.

To recap it for the Devils: the scorer at the Prudential Center appears to undercount hits and blocks while possibly overcounting takeaways at home. The 2018-19 Devils were actually very physical by way of throwing checks and sacrificing body parts to stop shots. It was only notable in the numbers on the road due to this apparent scoring bias. Of course, the 2018-19 Devils were abysmal and finished 29th out of 31 teams so I would take that point of potential pride with a grain of salt.

As a quick aside: if you want a player to exalt for being so gritty, then exalt Blake Coleman. He was credited for 225 hits last season. Kyle Palmieri came in second with just 98.

The League

Let us go back and look at the other thirty teams in the NHL. The Pittsburgh Penguins were far and away ahead of everyone else when it came to hits. They were above the league average by two standard deviations, which is a staggering amount. It is also very likely that hits were overcounted by a lot in Pittsburgh. On the road, the Pens were credited for 967. While that is a healthy number of hits in of itself, that is a difference of 427 hits compared to their play at home. The Devils had the largest negative gap between home and away hits; the Pens easily had the largest positive gap. No other team had a difference of even 300 hits much less over 400. The scorer in Pittsburgh just gives out hits like candy on Halloween.

In terms of other hit-friendly arenas, there were several other teams who had at least 150 more hits at home compared to the road: Vegas (+278), Carolina (+265), Montreal (+263), Edmonton (+254), Ottawa (+226), Chicago (+224), Arizona (+204), and Our Hated Rivals (+172). Vegas, Edmonton, Montreal, Carolina, and OHR all beat the mean by a standard deviation or more. Chicago is interesting because they had a relatively low amount and on the road, they threw the fewest amount of checks in the NHL. It may be fair to state that the Blackhawks were among the least physical teams last season.

As far as undercounting hits go, not as many teams were so suspect as the Devils. The Devils had the biggest gap between home and away hits at -220. Only two other teams had thrown 150 more hits on the road compared to playing at home: Nashville (-186) and Columbus (-161). Columbus was within the league average by a standard deviation for both home and away games so the swing was not that noticeable. It was for Nashville, who was closer to normality on the road compared to home games.

Moving on to blocks, Ottawa credited their players with more blocks than any other home team in the league. As with Pittsburgh and hits, it was not even close. Unlike Pittsburgh and hits, Ottawa was credited for just 41 fewer blocks. Ottawa was atrocious last season so it is possible that they legitimately blocked as many shots as they did. The relatively small difference between the home and away count suggests that if there is an overcount, it is not as big as we think. And it is not at all as big as it is in Vegas. Goodness, gracious, the Golden Knights were on the higher end of blocks at home and on the lower end of blocks on the road. Their home-away gap is 200 - no other team had a gap as large as even 100. I am more confident in stating that they overcount blocks in Vegas. No wonder EvolvingWild’s GAR tags Mark Stone as one of the most valuable players in the league. I predict he will continue to be up there unless Vegas changes scorers or something.

While there were not many large overcounters of blocks, there appeared to be plenty of undercounters. Again, the Devils has the largest negative gap of home and away blocks at -146. The following teams had negative gaps of -100 or lower: Florida (-126), Detroit (-117), Nashville (-105), and Boston (-104). Those four teams were below the league average by a standard deviation in blocks at home; all four were within the league average by a standard deviation in blocks on the road. Those are good signs that their scorers have been giving them a short end of the stick when it comes to credit for blocks.

No team was above the league average by two standard deviations in giveaways. Florida was the closest at 744. I’ve pegged them last year for having an audaciously high number of giveaways at home. It still is but the thing is that they also led the league and broke the two standard deviation mark for giveaways on the road. It is possible that Florida’s scorer overcounts giveaways and that the Panthers just had serious issues maintaining possession last season. I’m honestly impressed by that sign of futility. On the flipside, shout out to Minnesota for having the fewer amount of giveaways at home and on the road. I expected severe undercounting but it appears to be slighter than that.

But there were some severe overcounters at home for giveaways. Ten different teams credited their teams with 200 or more giveaways at home than they had on the road. Calgary (+383!), Our Hated Rivals (+341), Dallas (+315), the New York Islanders (+312), and Edmonton (+304) were the worst offenders. The second half of that list was Montreal (+281), Toronto (+281), Detroit (+270), the aforementioned Florida (+254), and Ottawa (+232). These are massive swings in counts. Out of all ten, only Ottawa exceeded the league average by a standard deviation. The other nine were well within the mean or even lower. But at home, each of these teams were credited for 600 or more giveaways. That is simply bonkers to me. Their respective scorers kept tagging them for this error over and over and over.

What’s more is that there are few teams undercounting this stat. You would think a team would want to have as tight of a definition for giveaways as to not make the home team look bad. Nope. Only two teams really stood out for undercounting giveaways based on home and away stats: St. Louis (109 more giveaways on road than at home) and Tampa Bay (94 more giveaways on road than at home). Neither were particularly bad at giveaways on the road; but their scorers seemingly helped them look better at home.

Lastly, takeaways. Similar to giveaways, this is another stat where there were more teams overcounting this stat than undercounting them. For the overcounters, Vegas and Carolina were heads and shoulders above everyone else in the NHL with a home count above the league median by two standard deviations. They were also #1 and #2 in terms of the gap between home and away takeaways. Vegas credited 345 more at home than they did on the road, which was just 279 takeaways. Carolina credited 299 more at home than they did on the road, which was just 295 takeaways. Yes, the difference in the home and away count was larger than the away count itself. I’m convinced that Vegas and Carolina seriously overcounted their takeaways last season. I would add Calgary to that group, even though the difference in home-away takeaways (279) was not as large as their road takeaways (283).

Other teams I suspect overcounted this stat would include San Jose (+171), Boston (+165), Edmonton (+157), Our Hated Rivals (+155), Washington (+154), and Florida (+151). The Devils’ gap was +88, which points to why I think there was a slighter overcount compared to other teams that counted many more takeaways. On the other end, only one team counted over a hundred fewer takeaways at home compared to the road: Los Angeles. I could not tell you why but the LA home scorer was remarkably stingy when it came to takeaways. Detroit (home-away difference of -85) and Buffalo (-57) also could be considered places where this may be undercounted.

Overall

A big reason why hits, giveaways, and takeaways really have not been used in analytics is because of what I demonstrated. There appears to be a home scorer bias for those stats and more. That there is a smaller amount of variation on the road points to that immediately. I can agree that some teams may throw more hits than others or give away the puck more. I can even agree to a degree that teams play differently on the road than they do at home. But to do so at a large amount such as 100+ times more or less between home and away games is much larger than those explanations. Among the teams with high counts, Florida’s high giveaway count is an exception not the rule. Most that have a high count at home do not have as high of on the road, which brings doubt as to whether the team is or is not good at it. These are stats from two groups of 41 different games so any exceptional nights would wash out over time. I find it hard to believe that a Devils team that was bad on the road and bad at home somehow threw 220 more hits away from the Rock. The more likely explanation is in how the stat is counted, not that the Devils were just meaner away from Newark. Since the stat’s count is suspect, then how can we assign value to it?

The challenge here is that for all we know, the Devils’ scorer (or scorers) is right about hits and blocks and other stats and everyone else is wrong. Or maybe Vegas and their incredibly generous approach to counting blocks and takeaways are doing it right. This is an issue of inconsistent inaccuracy. It would be one thing if all 31 teams were scoring games inaccurately but doing so consistently. A consistent error can be pinpointed and addressed through an adjustment or a systemic corrective action. But this is not the case. In just these four stats alone, teams are all over in terms of what they are over and/or undercounting. And the over or under counting teams are not necessarily all good or all bad teams. As it stands, with the potential scorer bias already in place, we do not know what is really wrong - and without that, we cannot figure out a solution.

There is some good news along the way. A solution may be coming soon. While it may not (ever?) be public, puck and player tracking will be implemented next season. Teams, such as the Devils, would be very wise to use the tracking data - among other purposes - to establish a more consistent count of these kinds of actions in a game. Even if the count definition is not exactly what it should be, being consistent at least allows for the discussion of whether what is being tracked is meaningful or not. Even if you and I may not know whether team hits truly matter, the teams who have to make decisions based on performance would have a more reliable baseline to make that call. Whether that will lead to changes in how games are tracked by the current scorer(s) is up in the air.

At the least, if you see a hit count or a block count out of the Prudential Center, then you have some reason to be at least a little doubtful of its accuracy. The count may be lower than what actually happened. Similarly for takeaways, although it would be in the other direction. These biases are real and they can be easily forgotten about. There have been many games where I’ll see, say, Devils-Rangers and both teams would not even be combined for 40 hits even though the two teams were basically piling on each other for sixty minutes. I doubt it then. But a week from that game, I’ll just see the number, accept it, and move on without a question. Such is the impact and while it may be small, it is there and it could impact how we look at how a team played or how a team did over a season. To quote yet again from Award, the castles have always been built on this sand. Given that there are stats that incorporate RTSS, the castles are only being expanded.

But, hey, at least we know the 2018-19 Devils weren’t shy about throwing their weight around for all the good that it didn’t really do.

Now that you’ve read through what I think of the scoring bias lied with the Devils and the league in hits, blocks, giveaways, and takeaways, what do you think is the issue here? Will more precise tracking help solve this? Do you feel better or worse knowing that the Devils may have been more physical than you may have thought last season? Please leave your answers and other thoughts about scorer bias in the comments. Thank you for reading.