Analytically Controversial Devils: Damon Severson and Pavel Zacha

If you’re not particularly plugged into either hockey twitter or the analytics community, you may be unaware of the war on WAR that’s been transpiring. It’s been going on a while, but reached critical mass when James Mirtle of The Athletic posted a piece (paywall) in which Tyler Dellow and Matt Cane discuss the merits/lack thereof of WAR (Wins Above Replacement) — a catch-all metric for player value. It was critiqued by some for not including (or even mentioning) the premier modelers such as Emmanuel Perry of Corsica and the Evolving Wild twins, Josh and Luke Younggren. Others came to the defense of some of Dellow’s criticisms of the metric, acknowledging that, though anecdotal examples are not themselves sufficient, many examples seem to be symptomatic of larger problems. The managing editor of this very blog is among such individuals. But I’m not going to re-litigate this 5-day WAR war here.

However, the the consequences of the WAR stat that frequently flies in the face of intuition got me thinking about players for whom the jury is out even among analytically-driven writers. Sometimes it seems that there are players that are loved by analytics (Stefan Noesen and Will Butcher this year or Beau Bennett last year) and players that analytics hates (Brian Boyle this year or Devante Smith-Pelly last year), but there’s a lot of middle ground that even us Corsiphiles and WAR-apologists disagree on.

For one example, view John’s piece on Andy Greene, and my subsequent, dissenting piece. John and I are both very analytically literate individuals and he’s been doing this for 12 years, myself 5. We simply came to, what I consider, equally informed/legitimate, but differing conclusions based on preponderance of the data. Some players aren’t even unanimously agreed upon on the same type of statistic — Sami Vatanen is a positive player in EW’s WAR and a negative in Corsica’s. These, of course, aren’t the only players on the roster that would fall into this area of controversy. Here I will review two more: Damon Severson and Pavel Zacha.

Damon Severson

Pro: A3Z metrics, performance w/o John Moore, historical shot metrics

Virtually all of Severson’s metrics improve when he played away from notorious analytical black hole, John Moore. This noteworthy because it might be part of why his shot and chance metrics took a dive this year after having been quite good historically — enough so for me to label him the best defender on the team earlier this year, before his metrics took a dive. There’s reason to believe in a turnaround though because, not only will he be free of John Moore, but he’s doing things well that normally translate to on-ice results. I presented a study at RITSAC indicating A3Z metrics (entries, exits, shot assists, etc.) may predict future performance better than on-ice metrics for defenders. This is good news for Severson because he’s performed quite well the last two years in these metrics. Severson pushes the puck forward very well and that skill is both valuable and highly repeatable.

Con: WAR, preventing “dangerous” shots, goal metrics

WAR has consistently hated Seves. If you look at the breakdowns, the most obvious area for is struggles is called WAR QA which is “Shot Quality Against.” This makes sense since his HDCA/60 Rel and SCA/60 Rel have both been positive basically every year and SKYROCKETED this year. The Devils had to withstand over 3 more scoring chances against and over 2 more high-danger scoring chances against per hour when he was on the ice. This jives with the Severson’s eye-test advocate detractors who say he’s weak in the defensive zone especially at clearing the crease of bodies and pucks. If you believe that there’s signficant information left out of xG models that can reliably be found in goal metrics (an opinion not without controversy) then Severson looks even worse as he’s only once had a RelT GF% over 0. This could lead you to a narrative that the Devils outshoot opponents with Damon, but get outscored — and perhaps both WAR and the coaches, who sat him in the playoffs, recognized that impact.

Should you be pro or con?

If you think it’s likely that John Moore was an anchor on Severson’s season, and that his transitional and offensive talent will more likely than not force a positive correction in his on-ice results next year, you’re “pro.” If you think that the reason his goal ratios are consistently negative and chance prevention consistently poor is that the liability of his defensive game trumps any contributions he may make offensively then you’re “con.”

Pavel Zacha

Pros: Improved shot metrics, improved 5v5 scoring rate, passing

According to Natural Stat Trick, Zacha’s shot rate metrics have improved modestly, and his chance rates have improved dramatically since 2017. This would explain why, according to Corsica, his RelT xGF% has gone up from -1.28 to +1.76 and his goal rates have followed. In addition to these improved on-ice metrics, he’s shown indications that he could still peak at a top 6 forward level. His 5v5 P/60 and P1/60 have both increased dramatically (0.82 to 0.135 and 0.75 to 1.05 respectively), and he’s always been a notably above average passer. If we put him with a scoring winger, there’s evidence to indicate a significant increase and vault into possible top 6 point production wouldn’t be totally unexpected.

Cons: WAR, Decline in penalty differential, scoring rate still low

Despite all those improvements, the WAR models are torn about whether or not he was more valuable this year than last. Why? He was worse on the PP but better on the PK so that’s a wash. His shot, chance, and scoring metrics have all increased. So what gives? Well one big thing could be that his penalty differential decreased by 17. A +10 in penalty differential in 2017, Zacha plummeted to a -7 in 2018 (according to Corsica). Some disagree about if WAR models weight this appropriately, but just realize that that’s a 34 minute swap. Given the Devils PK and PP rates, that’s about a 4 goal swap all by itself.

It’s also worth noting that, even if we ignore penalties and focus on the improvements elsewhere, despite those improvements according to Dom Galamini’s player evaluation tool, Zacha’s scoring still most closely mirrors that of a 4th line forward.

Should you be pro or con?

If you think the Zacha’s improved on-ice results are a result of his improved overall game and that his point production is low due to the lack of sufficient time with scoring wingers that he’d likely work well with due to his strong passing numbers, then you are a “pro.” If you think his point results are low because he’s simply not good at scoring, and any increase in on-ice stats is eclipsed by the impact of a massive slide in penalty differential, you’re a “con”.

Concluding Thoughts

It’s possible to occasionally run into dogmatism among those both inside and outside the analytics movement. I hope that examples like these serve as a counterpoint to that. More (good) data is always better, not because it confirms preconceptions but because it challenges them. I personally have changed my stance on several Devils players (mostly defenders) because I keep researching their performance and keep finding new data.

For these two players I completely understand why one could be pessimistic or optimistic. I personally still like Severson as a 2nd pair defender and still think Zacha will never find his scoring touch and tops out as a 3C, but I’m open to counterarguments both statistical and otherwise for why that may not be the case. Whether you believe in WAR, expected goals, Corsi, points, or the eye test; not growing too sure on the assessment of any player is generally going to be beneficial down the line.

What do you think of these two players? How about Andy Greene and Sami Vatanen? Do you have any other players that you can’t quite figure out? How do you use anlaytics to determine thoughts on players? Thanks, as always, for reading. Leave your thoughts below!