On-Ice Analytics and the 2019 Devils

I, perhaps naively, would like this article to be accessible to anyone no matter how well-versed in analytics they may be. So, for those who are already acquainted with the stats we at All About the Jersey use regularly, keep scrolling until you see a word you’re not familiar with.

Once the NHL upped their tracking game in the 2007-2008 season, the first innovation of the analytics era was to move from explicit “individual” statistics (things that a player did personally like goals, assists, shots, etc.) to on-ice metrics — a compilation of all events that happened for or against a players team while he was on the ice.

One of these early stats was “Corsi,” named for Buffalo Sabres coach Jim Corsi who pioneered the useful metric. Corsi was initially conceived as a proxy for possession by now-Capitals-consultant, Tim Barnes (went by pseudonym Vic Ferrari at the time). A “Corsi” event is a shot on goal, a missed shot, or a blocked shot — put simply: all shot attempts. A player’s on-ice Corsi For (CF) tells how many shots his team took while he was on, and similarly his Corsi Against (CA) tells how many the opponents recorded. CF% is the percent of all on-ice shot attempts were recorded by the player’s team when he was on the ice (formula: CF / (CF + CA) ). This strucure of for, against, and percentage stats also applied to “Fenwick” events (same as Corsi without the blocks; notated FF, FA, FF%), shots on goal (SF, SA, SF%), and goals (GF, GA, GF%). These metrics were hosted by several sites including Gabriel Desjardins’s BehindTheNet, David Johnson’s HockeyAnalysis (defunct), and ExtraSkater (defunct) by Darryl Metcalf who’s now special assistant to Maple Leafs GM Kyle Dubas.

Either instantly, or very soon thereafter, an important adjustment was made to these on-ice metrics. They were good for teams, but for player evaluation, it was important to know impact. To infer a player’s impact, the most common approach was to take a player’s on-ice stat — say CF% — and subtract his team’s CF% with him off the ice. The difference is a proxy for the impact of the player on that metric and is called a “relative” metric (Ex: CF%Rel). David Johnson actually had an alternative approach as well called “Relative to Teammate” that would be later picked up and displayed on Manny Perry’s Corsica (defunct) and Luke and Josh Younggren’s Evolving-Hockey — more on that later.

Before long, it was clear that this was a metric that rewarded possession, but not dangerous possession. It would behoove us to isolate specific types of Corsi events that were dangerous. This is where AC Thomas (recently dismissed from Minnesota’s analytics team) and Sam Ventura (Penguins’s Director of Hockey Research) came in with their site, WAR-on-Ice (defunct). The first innovation they made en route to a potential hockey WAR model was creating objective parameters for a “scoring chance” which they explain here. This method of “tiering” shots by danger would be perpetuated in Brad Timmins’s web site Natural Stat Trick. The same naming conventions from Corsi were observed for these new metrics; scoring chances (SCF, SCA, SCF%), and high-danger chances (HDCF, HDCA, HDCF%). These also had Rel stats associated with them.

These are helpful because people understand what a “scoring chance” is — we’ve used those words forever. But it does something many analytics folks don’t like called “binning.” This means we take shots and put them in “bins” — low, medium, and high danger — and in doing so, forsake a holistic view of the shot’s total value. The notion of taking the circumstances surrounding a shot and using it to place a danger value on it goes back to 2004 with Alan Ryder’s shot quality piece but first was applied to this Corsi-esque, on-ice format by Brian MacDonald (eventual Director of Analytics for Panthers 2014-2018) here and made mainstream by Dawson Sprigings (originally known as DTMAboutHeart and eventually hired by Avs ownership, Kroenke Sports) here. From then on expected goals (xGF, xGA, xGF%) would be a mainstay in analytics. They incorporate the same types of elements the scoring chances models did, but do it on a spectrum and assign each shot a probability of becoming a goal.

That was the last big on-ice metric innovation. But how we compile these metrics was still evolving. I mentioned that David Johnson had used another method of calculating relative stats. He created a chart called a With or Without You (WoWY) table. Some of you may remember WoWY tables like this one from Johnson’s site. A player’s WoWY is a list of every skater that guy was on the ice with, and how they performed in selected metrics together and apart. Johnson’s new relative metric took all of the individual differences between the together/apart metrics, weighted them by TOI, and came up with a Relative to Teammate (RelTM) metric that was a skaters sum impact on every one of his teammates, rather than just on-ice minus off-ice as described above. For a thorough description with examples, check out EvolvingWild’s (the Younggren Twins) write up here (part 1, part 2)

The aforementioned twins would make one more amendment to the “impact” discussion. Rather than just adding up all the impacts on individual teammates, they wanted to control for all players on the ice and other contextual variables not attributable to the player’s actual performance (score/strength situation, home/away, back-to-back game, where the shift started, opponents). They did this using a technique called regression which allows us to control for all those variables, and conclude that the remaining plus-minus impact was attributable to the player. This Regularized Adjusted Plus-Minus (RAPM) has a full writeup here and it is the most rigorous publicly available adjustment for on-ice metrics.

Regression methods are used all over the place in analytics. They’re used in the RAPM but also in most xG models and perhaps most prevalent of these applications is in WAR (Wins Above Replacement). WAR is a concept borrowed from baseball in which we attempt to gather the entire value of the sum total of a player’s contributions and consolidate into one number. How many more wins are they worth to their team than a player who they could just sign as a free agent or promote from the minors — a “replacemet-level” player. WAR-on-Ice made this their mission statement and succeeded in a beta-WAR shortly before discontinuing operations. Sprigings, then Perry, then the Younggrens (one, two, three) all took up the mantle after them — the last of which is the only one still publicly available.

WAR takes all of these on-ice innovations, plus indivudal player metrics you know and love (goals, assists, etc.), and feeds them into a machine that decides what proportions of which of those ingredients make the best recipe for assessing the value of a player in units of goals. The Evolving-Hockey Goals Above Replacement (GAR) is the most mainstream one at the moment and it using the RAPM method with GF as the “offensive” target variable to predict, and xGA as the defensive — the assumption being that the difference between xGA and GA is largely attributable to the Goaltender, not the skaters — they use an ensemble of machine learning techniques to quantify the value of a player.

And that brings us to where we are today. We have Corsi, Scoring Chances, Expected Goals, and Goals; we can compile them as Relative, Relative to Teammate, or Regularized Adjusted Plus-Minus; and they are all important predecessors or even components of the model for Wins Above Replacement. If you need a reminder of what any of these skip to the Glossary* at the end.

So enough of my nerdy stroll down analytical history. Here’s what you came for: the Devils performance in these metrics. All Relative stats were retrieved from Natural Stat Trick, and all other metrics are from Evolving Hockey. GAR is in “per 82” units. This is simply GAR per game multiplied by 82 to figure out how much a player would be worth over the course of a season. You can navigate through these charts on your own at this tableau.

Depth, Traded, or Waived

Brian Boyle was having a good impact in all situations which is likely part of why we were able to trade him for such a good price. It seems like the hot streak heading into the deadline also helped Mojo’s trade value because his results all over these tables are thoroughly unremarkable, performing near or below average everywhere except for PEN GAR (goal impact of penalties taken and drawn). Ben Lovejoy had been hyper-efficient ever since he was paired with Will Butcher in 2017. He is also one of the best penalty killers in analytics history. I commented on both of those things in my farewell post here.

Kurtis Gabriel had a surprising goal impact which seems to be inflating his GAR since all of his other on-ice metrics, as well as a lifetime of mediocrity, indicate otherwise. JSD had a notably positive impact on expected goals and it would have been interesting to see what he did with a little more time. Gryba was and is irrelevant, managing to be a sever negative in one of the only areas he purports to be of use, the PK (-2.46 SHGAR)

Miles Wood had positive impacts in the RAPMs across the board, and was a positive on the man-advantage. Rel and RelTM metrics also indicate that, at the very least, his shot attempt impact (CF%) is positive. His individual production was definitely a disappointment, but it’s nice to see his peripherals don’t depict a total spiral. We cannot say the same for Noesen who, historically a solid analytical player, was narrowly above average in expected goal impact and negatives across the board everywhere, including a ridiculous -25% GF%Rel. A restricted free agent this off-season, the Devils are going to need to make the tough call on determining if, and to what degree, this year was a fluke.

Young Guys

Due to a litany of injuries, the Binghamton Devils played in New Jersey for a while at forward. Unsurprisingly, there’s a lot of red here, and no that’s not just Devil pride. Most of them sucked so let me just acknowledge here the ones that didn’t in some way. Nathan Bastian is the big plus here. At even strength, he’s got blue everywhere in this chart, and sticks out like the only not sore thumb because of it. In his 7 games and 90 minutes of hockey, he showed a physical game that fans appreciated, some production (3 goals), and an overall positive impact in shots, chances, and goals. If any of the B-Devils earned more time next year, it’d be him and Kevin Rooney. Rooney had a positive expected goal impact likely influenced by his positive impact on high-danger shots. Rooney was also a plus in the SHGAR department and the prospect of a Rooney-Coleman PK is titillating. Seney spent a lot of time up with the big club and was a handful. His creativity and shiftiness on the puck made opposing players need to take penalties to keep up (+2.57 PEN GAR). But his overall game needs work (negative in all Corsi impacts). All told, he was probably about replacement-level this year.

Legit Defenders

If there is anything to take away from this, it is that Will Butcher is far and away the best analytical defender on the team and has been the entire time he’s been here. I was one of the two votes for him in the AAtJ awards for best defender. My logic? ...

He succeeded.... That’s basically it.

As simple as it sounds, that’s a bar that all the other everyday defenders failed to clear. The winner according to our writers, Damon Severson, is in the red in every category of every impact calculation.

Is Butcher given easier assignments? Yes. As you can see, the Butcher in a strict on-ice, off-ice standpoint is stratospheric. Then when we account for all teammate impact, it comes down a little. And then he’s only slightly above average when we account for other variables like zone and competition in RAPM. So the ease of his use is relevant to some of his stats, but even after accounting for those things, he’s STILL the Devils best.

Mueller has a good season on-ice season, particularly in the expected goal area, but his results (goal totals) didn’t follow. Hopefully his luck turns around next year.

For that matter, having Nico on an ELC, Hall on a team-friendly contract, and now the #1 overall pick also on an ELC, it’s officially time: Shero needs to address the defense. Ty Smith comes up next season, but Shero needs to go out and get a bona fide #1 defender because this team hasn’t had one since Andy Greene was in his prime...which was like 100 years ago.

Top 6 Forwards

FINALLY ... BLUE. The Devils top 6 forwards are where all the value was hiding. The Devils actually have a decently strong top end. Get the obvious out of the way first: Hischer, Hall, and Palmieri are all very good. The reason that Hall isn’t in a league of his own this time is likely because of goal impact. He’s a Corsi monster and his position atop that Corsi throne remains more or less unchallenged. As you can see though, the three of them all have the same GF%Rel (upper left chart, rightmost column). That’s despite Hall missing half the year. In other words, that line’s goal results were just fine without him. And so the RAPM model decides Nico and Kyle were more responsible for those goal results as a consequence. Despite declined PP individual scoring, Hall was also still the most valuable player with the man advantage (+3.98 PPGAR). Anyone who watched the Devils down the stretch can testify to the conspicuousness of Halls absence on our atrocious powerplay.

Moving to other players — what more could possibly have been asked of Kenny Agostino. As John said in mid-March, he was absolutely making the most of his opportunity after being added, so much so that he was promoted to the first line en route to being named the AAtJ Devil of the Month. Joining him in the hot streak down the stretch was Pavel Zacha who after just 5 points in the first 31 games(5G, 0A) scored 20 in his last 30 (8G, 12A) including averaging nearly a point per game with 11 points in the final 12 games. Including the entirety of his season though, his expected goal impact was catastrophically low thanks to poor scoring chance AND high danger chance ratios. Couple that with his lack of production, and you get a below-replacement-level player at even strength (-2.96 GAR). His positive impact on the powerplay and penalty kill make him a worthy depth player even when his production has dried up though.

Blake Coleman objectively had himself a breakout season, being one of just two Devils to clear the 20-goal mark this season. With a shooting percentage of 10.3% (career 9.3%) it seems fair that that goal mark is an accurate portrayal of his offensive contributions too. With regards to his on-ice numbers, it’s clear that he has a positive impact on quantity, though the negative high-danger impact somewhat dilutes the overall expected goal impact of his performance, and ultimately his goal impact ended up negative. This could be a consequence of his aggressive style of defense. It causes a lot of turnovers, but also springs players loose and opens gaps. The next step in his game has to be getting a hold on his penalties. He draws a lot but has a negative PEN GAR because he takes so many. Part of that is just his game, but the 4 high-sticks, 6 trips, 2 interferences are some examples of time he shouldn’t be spending in the box. His PK expertise is of no use if he’s the one in the box.

Zajac and Bratt are each other’s inverses. Zajac has strong shot metrics but hasn’t seen it reflected in goals. Bratt has struggled in possession, but has compensated with scoring. This is reflective of who we think they both are. Bratt scored at higher rate at 5v5 than anyone on the team other than Taylor Hall. Meanwhile, Zajac’s expected goal RAPM impact was better than anyone outside Hall and Hischier. When accounting for each of their strengths and deficiencies, both are clearly deserving of spots in NHL lineups, probably 3rd line or above, and Bratt’s arrow is pointing up.

Concluding Thoughts

The Devils have a decent top of the lineup, driven by an exceptional top line. The entire defense, save Butcher, is barely performing at NHL level at this point. The Binghamton crew is mostly still unqualified to get NHL minutes — Rooney may have earned a roster spot, and Bastian may have earned another chance. But in general, this team struggled whenenver the one of the top ~5 forwards wasn’t on the ice. And this year the goalies weren’t there to bail them out. Perhaps next year, with added health, Hughes/Kaako, improved goalie play from Blackwood/Schneider, and maybe an acquisition or two, these charts will look a lot more blue.

What are your thoughts on the Devils players performance in on-ice metrics and GAR? Are any surprising? Are any REALLY not surprising?

Thanks as always for reading and leave your thoughts below.

~~~~~~~

* Glossary

On-Ice Metrics

Corsi — A shot event (shot on goal, missed shot, blocked shot). Most commonly used for player evaluation are the on-ice metrics: CF, CA, and CF% which are the shots a player was on ice for, against, and the ratio of the two respectively.

Scoring Chance/High Danger Chance — Corsi events determined to be dangerous or very dangerous depending on variables such as the shot location, shot angle, and events that preceded the shot. On-ice calculations are SCF, SCA, SCF% and HDCF, HDCA, and HDCF% respectively.

Expected Goal — A number associated with the probability of a shot event at becoming a goal based on the circumstances of the shot such as location, angle, game state, and events preceding the shot. On-ice calculations are xGF, xGA, xGF%

Goal — I mean ... duh. On-ice calculations are GF, GA, and GF%

Impact Calculations

Relative or “Relative to Team” — For a given metric, the teams performance with a player on the ice minus their performance with the player off the ice in games the player played. Notated with “Rel” after the stat such as CF%Rel.

Relative to teammate — The TOI-weighted impact of a player on all of his linemates performance in a given metric. For some player “A” find out how a linemate “B” performed with “A” vs how they performed without “A” to determine the impact “A” had on “B.” Do this for every player, multiply by the percent of “A”s ice time that was played with “B”, and add the impacts together. Here’s a dumb example using fake players. Notated with a “RelT” or “RelTM” after the stat such as xGF%RelTM

Regularized adjusted plus-minus — The impact of a player on a given metric as determined by a regression method that attempts to control for variables beyond the players control such as score/game-state, venue, fatigue (back-to-back games), and the other 11 players on the ice. This is not ubiquitous enough to have universal notation yet, but here it’s notated with RAPM such as in GF%RAPM

Goals/Wins Above Replacement — Ideally, the net impact of all a players contributions in units of goals/wins. In practice, this process feeds all the metrics we have into an algorithm that determines what mix of them is most efficient and effective at determining goal for impact or expected goal against impact.