Has Ryan Murray been good this season?

It’s okay, take your time to answer. I had no idea when I started writing this piece, which is why I asked Twitter. To my eye, he started the season rough, getting exposed on the top pairing with Subban and the first penalty-killing unit. Lately, I haven’t really noticed him. For a defensive defenceman like Murray, that’s normally good news.

Has Damon Severson been good this season?

I feel like this answer was a bit easier for me. The short answer is “Yes.” Ty Smith was floundering with aspiring AHLer, Matt Tennyson, to start the season and then he looked like a true future stud alongside Damon. Ryan Murray’s improvement has also coincided with his time next to Seves. And not too long ago, I declared Andy Greene’s career revived while playing with Severson. How many partners in a row need to improve before we say that Damon Severson, who plays key minutes in all 3 situations, is the Devils best defender?

So we’ve got here, two blueliners — one for whom the assessment seems pretty straightforward, and the other for whom it’s muddier. They contribute to the game in very different ways, so sometimes it can be difficult to say who is “more valuable” and by how much. This is an are where it can be helpful to use an “all-in-one” metric like Goals Above Replacement (GAR) or Expected Goals Above Replacement (xGAR).

GAR and xGAR

GAR and xGAR are each attempts by Luke and Josh Younggren (@EvolvingWild) to quantify the entirety of the value of a player’s performance in terms of the goals contributed. They include components for even-strength offense and defense, powerplay offense, shorthanded defense, and penalty impact. You can find the writeup for GAR on the front page of their revamped website, Evolving-Hockey. The xGAR model is closer to the WAR-on-Ice approach here by Andrew Thomas, et al. I often use them both as sanity checks for my eye test. You could imagine my frustration, then, when the results for these two D-men turned out to be ... let’s say “less than conclusive”.

When looking at GAR, Severson is the Devils worst defender, costing the team 2.4 goals relative to a replacement-level player; and Ryan Murray is the most valuable, adding 5.9 goals. In xGAR they basically entirely switch roles.

Why is this happening? How can both of these players switch from best to worst depending on the metric we use? And who is having the better year? For this, we need to understand the difference between the two models. A peek under the hood can help point us in the direction of the cause for this discrepancy.

What Is Causing This Difference?

The entirety of the difference between the GAR and xGAR values of these two players is accounted for by the difference in their EVO (even-strength offense) value. When switching from xGAR to GAR, Severson’s even-strength offense goes from +3.8 to -3.9 goals of value, and Murrays goes from -2.3 to +5.3. The reason for this boils down to how each of the models measures offensive impact. And if you look at the on-ice results the two players have experienced, you might be able to take an educated guess as to what each model targets. This is how the Devils have performed with Severson on the ice as opposed to Murray.

According to Natural Stat Trick, the Devils generate fewer shots and fewer scoring chances per hour with Murray on the ice than any other Devil, whereas Severson’s scoring chance numbers trail only Ty Smith. In total, we’d have expected the Devils to score 2.04 goals per hour with Murray given their shot and chance rates, and a team-leading 2.54 with Severson. What actually happened, was the Devils score 2.96 per hour with Murray (team high), and 2.07 with Severson (above only Kulikov). The result is that the Devils scored 11 more goals than expected with Murray, and 8 fewer goals than expected with Severson.

As you may have pieced together by this point, the GAR metric that prefers Murray values goal scoring more than shot and chance production and the xGAR that favors Severson values the opportunities more than the conversion of them. The logic here is generally that you can’t control the ability of your teammates to convert scoring chances. Therefore, especially for defenders, a metric like xGAR would likely be more indicative of the impact a player has, even if it’s less descriptive of the goal results on the ice.

For people who are familiar with the impact of luck on things like goal production, this would seem to make the story pretty clear: Damon Severson has been good buy unlucky, Ryan Murray has been lucky, but bad.

But there is one more wrinkle...

Quantifying The Impact of “Luck”

If the difference in their production relative to their shots/chances is simply attributable to luck, then you’d think it would come and go randomly year-to-year. Interestingly, that’s very much not been the case for these two, historically.

In Damon Severson’s 7 years in the NHL, his GAR value has been higher than his xGAR value only once. On average, his value when looking at scoring is 4 and a half goals lower than when looking at shots. Meanwhile, in Murray’s 8 NHL seasons, his GAR has been worth less than his xGAR only once, and, on average, has been worth 3 and a half more goals when measuring conversion over opportunity.

A preference for xGAR over GAR is predicated on the assumption that, while players may be able to impact the amount of scoring chances their teammates get, they cannot impact the ability of their teammates to convert those chances. But if that’s the case, how come Damon’s teammates seem to always underachieve and Murray’s always seem to overachieve?

Perhaps these players do have the ability to impact the conversion rate of their teammates. It’s tough to really say without extensive investigation, but let’s assume that this is a true feature of their talent. How much of the difference is due to their talent? All of it? Most? Barely any?

As a last endeavor at measuring their “true” value, let’s assume that their true talent at converting xGAR to GAR is just their career average rate per hour (-0.2 for Severson, +0.16 for Murray). This is what the adjusted GAR values will be under that assumptino.

Damon Severson falls and Ryan Murray rises, but not enough to make up for the gaping chasm between them in shot and chance generation. For reference, among the top 186 NHL defenders (31 teams, 6 D each) with the most minutes played this season, Damon Severson would rank in the around 110th (back-end 2nd pairing) and Murray would be around 140th (fringe NHLer) in these adjusted metrics.

Concluding Thoughts

I don’t want to make it seem like this piece has solved the problem of the disagreeing value metrics. For one, these aren’t the only value metrics out there. For another, we don’t know if it’s reasonable to expect players to be able to impact GAR-xGAR differential in the manner I described here. There were 943 NHL skaters this season. Surely a few of them just lucked into differentials this extreme and maybe this article just happens to feature two guys at opposite tails of the bell curve. Or maybe there are team effects and the differences between Columbus and New Jersey have contributed here — in that case we only need to explain one lucky Murray season as opposed to a career’s worth. Or maybe it’s something about their very different roles.

The point is that there’s still a lot to be learned. But knowing how value metrics measure the impact of a player can somewhat demystify the numbers in general, as well as the discrepancies between them.