clock menu more-arrow no yes mobile

Filed under:

How Do the Devils Determine If Ruff and His Staff Succeed In 2020-21?

How does one assess a coach performance objectively? What are some ways to isolate the impact of a coach on performance?

New York Islanders v New York Rangers Photo by Bruce Bennett/Getty Images

Ultimately, I agreed with the decision of the Devils management to dismiss John Hynes from his position as head coach, but I was one of the last ones to do it. Some may argue that I was wrong and simply was forced to admit it by the time he was fired. From my perspective, we were getting new data every day and right around the time Hynes was relieved, the data swung against the argument of keeping him. But the fact that I had such a fundamentally different assessment of the situation led me to wonder if I’m doing it wrong, other folks are doing it wrong, or if everyone is doing it wrong.

This season seems poised to be another tough one for New Jersey and, when we know the team is unlikely to perform well, what does a “good” coaching job look like? How can we assess it? Here are a few things that cross my mind as important aspects of analyzing coaches. In the comments let me know which things you agree with and disagree with.

Consider player age and past performance

The coach of the President’s Trophy-winning team is almost always in the conversation for the Jack Adams. It happened again this year with Bruins coach Bruce Cassidy being a finalist. We all know, though, that not every coach is dealt an equal hand. The mistake people make in assessing coach performance, though, is in take the straight difference between the point totals of the team last year and the year before. In reality, fully understanding the trajectory of a team’s roster is essential to assessing the quality of the coaching. It’s important to know, among other things. the consensus around aging (one, two, three, four) — the peak age for NHL skaters is in the 22-24 range and there is a precipitous decline starting around age 30.

How should this impact our assessment of Ruff? The Devils are a young team, so, collectively, the roster should probably improve next year. It’s very reasonable to expect Hischier, Bratt, Hughes, Zacha, Anderson, Boqvist, and even Blackwood to improve. We probably should not expect improvement from Palmieri or Gusev — yeah, Gusev is only a year younger than Palms. And it would be understandable for Zajac to become less and less playable each year.

Look at Lineup Decisions with a Critical but Open Mind

So we’ve determined that the sum of the improvement of all players relative to expectation is valuable in evaluating coaches. But, if Hughes becomes Connor McDavid this season and still only plays 16 minutes, we won’t see the benefits as much as we should and that will totally be on the coaches shoulders. This is because talent development is only one part of the puzzle — deployment is another key feature. I think Hynes was a perfect example of this last year. There were some lineup decisions that were decried that I thought were defensible, and explained why. But there are other things that, even if you play Devils Advocate, are not easily explained.

The sitting of Jesper Bratt is a good example. Bratt was one of the best scorers on the team and his total impact on the ice was distinctly positive, yet Hynes sat him multiple games. The argument in favor of this move is that Bratt’s defensive play improved as the season went on and that Hynes sitting him had something to do with that. Even if you’re inclined to be that charitable, you still need to explain how an NHL head coach is unable to train a very good young player to be responsible in his own end without hurting the team by sitting a top 6 forward who is not even playing particularly poorly.

With Ruff, I’ll be looking for two things. First, I want to see if he’s putting our young forwards in positions to succeed. Gusev didn’t get going last year until he was put with Coleman and Zajac — where does he get placed to keep the momentum going? Hughes was, by far, most productive when he centered Palmieri and Hall — how do we replicate that with Hall gone? Second, I’ll be looking at the deployment — particularly on the blueline. I’m willing to give him a chance to prove us wrong early in the season, but if his first two weeks of 2021 look anything like he did in 2020, then P.K. Subban cannot be given first pairing duties. And, as I’ve argued in the past, Will Butcher needs more minutes.

Look at underlying metrics

The first impact of the coach is on the individual players. The next impact is on the deploymant of those players. So lastly, the finish product should be a competent team with positive results. But what are “results” in a fluky game like hockey? There is no one best metric to look at for assessing the “true” quality of a team. An unambiguously good team, in my opinion, would be strong in CF%, xGF%, and GF% in both 5v5 (score/venue-adjusted) and all-situation performance. I don’t expect us to be good in every one of those metrics this season, but if we are as bad or worse than last year in all 3, that doesn’t bode well for Ruff, even if the 2019 addition did have more weapons (partial seasons from Coleman, Hall, etc.).

The reason we want to look at underlying metrics, though, is that they can help suss out some of the element of luck. The PDO components (goaltending and shooting) are likely only marginally impacted by coaching, and I doubt there is any impact at all at the head coach level. So, the Devils do very well in at least one of the two other categories (xG or Corsi) then we can say that there is a true plan and the product of that philosophy is tangible. During the Hynes era, the Devils had the 4th worst GF% (45.2%), the 4th worst CF% (47.0%), and the 11th worst xGF% (48.4%). I’ll be looking to see where Ruff’s Devils fall relative to those figures. If the Corsi can improve or the xGs can threaten 50%, I’ll feel good about the trend.

This also goes for the assistants. If the powerplay xGF/60 and penalty kill xGA/60 are improved from last season, we’ll have some evidence as to the validity of the scheme. Though, Nasreddine is going to have a harder job with not Coleman and no Greene.

Concluding Thoughts and Other Things to Look At

I think that these three components are good ways to approach coach assessment, but I’d be remiss to not mention that there is also one public coaching impact model. Micah McCurdy of Hockeyviz created a model that, essentially, treats the coach as another skater and measures the impact the same way. Since they are part of Dr. McCurdy’s larger model, they are informed by priors, which incorporates the aspect of my first point about the hand a coach was dealt.

Other than this, what ways would you assess coaches? Is there something that I missed? Is there something that I said that you don’t think is fair? What will you be looking for to determine if Ruff is the man for the job?

Thanks as always for reading and leave your thoughts in the comments below.