Questioning Player Assessments: Which Devils’ Skill Sets are Misunderstood?

In science and philosophy, there is a term called “consilience.” Consilience is the principle that the strength of a conclusion is enhanced if it can be arrived at from multiple data sources or multiple lines of thought. This is a concept that is at the bedrock of many philosophical claims.

It’s true of things like, say, natural selection. It was internally consistent as a theory using basic logic, it was then determined to be predictive of things (ex: Xanthopan) and finally observed in real-time (Peppered Moth). We need not delve into the annals of science to see this philosophy in action, though. You are doing using consilience all the time.

When examining any physical structure, you are triangulating it’s existence using several senses. Is that an apple? It looks like one ... maybe it’s plastic. Nope, it feels squishy ... could just be cushioned, though. Smell’s a lot like an apple ... but, I guess you could perfume or something. It tastes like an apple — I think it’s time to admit this is probably just an apple. Could one sense be eluded? Perhaps. It’s quite unlikely that all of them are, though.

Why am I talking about this stuff in a blog post about the New Jersey Devils? Because I think this is one area in which analytics skeptics, no matter how antithetical to the revolution they may be, can admit some utility. If we can confirm a player is good/bad using multiple methods — ex: eye test and numerical analytics — than we can be more confident in the assessment than if the two methods disagreed.

And example of this working is Blake Coleman. Eye-test Devils fans will tell you that he is a relentless forechecker with an active stick who is first to the puck and often physically overmatches larger opponents. As it turns out, his 200-ft game is, analytically excellent as well. He’s an elite defensive forward, and a heavily positive offensive impact as well. This also has intra-analytical consilience as multiple distinct methods (ex: Threat Impact and GAR) agree that he’s very impactful. Your next question should be: “So, what player assessments can we NOT say this about?” I have 3 current candidates that fit the bill, plus a bonus historical one!

1) Nico Hischier is a 200-ft player, Jesper Bratt is not.

Nico Hischier claims he models himself after Pavel Datsyuk — one of the greatest analytical players of all time, and one of the greatest defensive forwards who ever lived. Nico has a Coleman-esque active stick, and has already surpassed his previous career-high in PK minutes. Type “hischier selke” into the search bar in twitter and watch the tweets roll in. Even former coach John Hynes said he’d likely win a Selke one day. He has the reverence of the fans, and the trust of the coaches to play all over the ice.

Jesper Bratt has had a very different story. He came out the gate as a teenager that was given time in all 3 situations. His PK time has vanised ever since. Former coach John Hynes made him a healthy scratch in consecutive games earlier this season. I argued that the reason for this type of usage was likely due to Rooney and Hayden’s defensive superiority.

Whether Hynes’s approach struck a chord with Bratt, or he has just naturally aged towards competence — this assessment no longer holds. Comparatively, the assessment for Nico may never have been right in the first place.

In the spirit of this “multiple methods” philosophy, allow me to present completely different analytical methods on these players — one from Micah McCurdy, operator of Hockeyviz, and one from Luke and Josh Younggren of Evolving-Hockey.

These are completely different approaches with, ultimately, the same goal. Isolating for circumstances, how has a player performed. The EvolvingWild twins’ RAPM metric is a single-year figure (though a 3-year version exists that is not updated for 2020), and Micah’s is “chained” using a Bayesian technique called “priors” — rather than starting naively, you assume they’re as good as the were the previous year with aging impacts. Their treatment of things like priors and certain idiosyncracies like zone usage are sufficiently different so as constitute two non-redundant methods of player evaluation. And they both agree here:

Nico Hischier is thoroughly average in the defensive zone, but his offensive impact at both 5v5 and the PP is pretty clear. Bratt, conversely, has a terrible PP impact, but is positive up and down the ice at 5v5 and, in fact, is BETTER defensively than he is offensively when it comes to observed, empirical impact.

Am I saying that Nico Hischier is not as good a defensive player as Jesper Bratt? Not necessarily. I think Nico has the skills to become an excellent defensive forward, actually., and is already a very good offensive forward. I also think his contract is going to be a steal. What I am saying is that, there is enough analytical evidence here to at least question why Hischier is lauded as a future Selke nominee, while Bratt is getting sat for defensive ineptitude.

2) Damon Severson Brings It on Offense, but Struggles on Defense

Severson is one of the many players on this team that profiles as a “puck-mover”. The redundancy is actually why I lobbied for trading Sami Vatanen. Damon catches the wrap of being prone to boneheaded defensive plays. I’m not here to argue that he doesn’t or that this biases our assessment and he’s actually much better than you think (I’ve done that in the past, I’ve grown less certain in recent years). What I am saying is that his impact may not be what you think it is in either direction.

As it turns out, when it comes to impact numbers, Severson is suppressing shots everywhere, not adding them everywhere. His offensive numbers impact is currently quite low, and his defensive numbers have been great this year, and slightly above average even if we use last year as a prior. Interestingly, this is NOT even limited to 5v5. Over the past 3 seasons, Severson has been worth 1.5 GAR on the PK and 0 GAR on the PP.

Is Severson a stay-at-home defender? A defensive specialist? Obviously that’s not what I’m saying. What I am saying is that there’s evidence to question whether Severson’s strengths and weaknesses are what many seem to think they are.

3) BONUS: Kovalchuk was a powerhouse over his time in NJ

During Kovalchuk’s last two seasons in NJ, he played over 24 minutes a game — absolutely absurd for a forward — and recorded votes for both the Hart and the Selke trophies. Him dragging us to the Cup in 2011-12 was not viewed in an entirely dissimilar fashion to Hall dragging us to the 2017-18 playoffs. His departure was considered a crippling blow to an already-aging Devils team — it was not possible to overcome the loss of his impact....

Or was it?

Here’s a side-by-side comparison of the Devils two best players during Kovy’s NJ tenure. Kovalchuk, and should-be-hall-of-famer, Patrik Elias.

Wow, that’s not even close. In both models, Elias is transcendent and Kovalchuk is ordinary and, debatably below average. How did we miss this? Here’s one data point from the 2011-12 season: Elias’s usage pitted him against the other team’s best players more than average armed with the Devils’ 4-6th forwards. Kovalchuk had it FAR easier, playing primarily with a line of Zach Parise and Adam Henrique against a fairly unremarkable distribution of competition. Kovy’s linemates could, and did, a lot of work at 5v5. Meanwhile, Elias was saddled largely with soon-to-be-retired Petr Sykora and stay-at-home forward Dainius Zubrus, and somehow dragged them to a 50% xGF ratio against other team’s top lines. How can we be sure? RelT numbers (in comparison to RAPMs and Isolates) are relatively primative in that they only account for the TOI-weighted impact of linemates. Therefore, the fact that his RelT xG numbers are negative over that span as well would support the claim that the driver of his numbers had something to do with the teammates with whom he shared the ice.

If you think that sounds convoluted, don’t bother, with the fancy stuff. Elias outscored Kovy 176 to 174 from 2010-2013 despite playing 600 fewer minutes.

This may go to help explain how the Devils, despite losing their dynamo, went from a 0.500-Pts% team in 2013 to a 0.537-Pts% team in 2014. The Devils added a 41-year-old Jaromir Jagr, who, as it turns out, was worth WAY more GAR and xGAR than a 29-year-old Kovalchuk.

With the benefit of perspective, this loss seems like far less of a tragedy than it was at the time. Furthermore, if we give reverence to consilience, we may avoid a similarly misguided overreaction in the future.

Your thoughts

What do you guys think of this mode of thinking? Do you look to the eye-test primarily? The analytics? Or do you use consilience?

What are your assessments of Hischier and Bratt? Severson? Elias and Kovalchuk?

Leave your thoughts below, and as always, thanks for reading!