Let’s Experiment in EHM: How a WAR-Minimized Team Performed in 2019-20

Last week, we completed the experiment to ruin Our Hated Rivals in Eastside Hockey Manager. We did a lot of damage. Not only did they put up one of the worst seasons in franchise history, the long-term impact of trading away picks and young players plus some terrible GM work by the AI meant they went another 15 seasons without the playoffs. The Blue Line approved (Risto Remes may not). It is time to get away from Manhattan and get back to WAR.

One of the earliest experiments in this series was to see how a team optimized for Wins Above Replacement would do in 2019-20. EliasStillRocks suggested to try the opposite:

Just a suggestion…. If you’re trying to make a near-perfect/excellent-playing team using GAR/WAR, you should try to do the complete opposite as well. Make the worst team possible filled with NHLers, and with reasonable amounts of rookies and/or AHLers (1-3 max on the team), and see how they fare in a full season. My guess would be that they still perform admirably, but are in the bottom third of the standings. Who knows, maybe they shock us?…. but probably not.

And so this is what was run in this past week.

The Experiment

Construct a team made up of players who put up the worst WAR values in 2019-20, replace the New Jersey Devils with this team, and play through the 2019-20 season.

The Approach & The Team

In the WAR-optimized experiment, CJ put together an algorithm to determine a 23-man roster optimized for GAR (Goals Above Replacement, which leads to WAR) while remaining cap compliant and giving me at least six defensemen and twelve forwards. In response to EliasStillRocks’ suggestion, CJ ran it again for his suggestion of the worst GAR roster. This is what he came up with (GAR values in parentheses):

Goalies (2): Devan Dubnyk (-9.4), Jimmy Howard (-10.1)

Defensemen (8): Niklas Hjalmarsson (-6.1), Jake Gardiner (-7.3), Brendan Smith (-7.4), Jack Johnson (-9.5), Korbinian Holzer (-6.6), Jacob Trouba (-8.5), Johnny Boychuk (-10.1), P.K. Subban (-10.1)

Wingers (8): Justin Abdelkader (-5.0), Max Jones (-5.2), Adam Erne (-6.0), Michael Frolik (-7.2), Wayne Simmonds (-4.9), Phil Kessel (-6.1), Brendan Perlini (-6.6), Kaapo Kakko (-9.6)

Centers (5): David Kampf (-6.2), Jesper Boqvist (-6.2), Jesperi Kotkaniemi (-6.2), Valtteri Filppula (-6.7), Frans Nielsen (-7.5)

This is a team filled with players from teams who were really poor in this past season. By team representation, the Red Wings led with five. We have three Rangers, three Devils, three Coyotes, two Ducks, and a single representative of the Islanders, Flames (or Sabres), Canadiens, Hurricanes, Penguins, Blackhawks, and Wild.

Given that GAR is meant to judge a player’s performance, negative values mean that their play is detrimental to their team’s cause. What they do is causing their team to not win games. Given how a WAR-Optimized team crushed the NHL in an 82-game season and in the postseason, this team should be closer to the opposite: utter trash.

I do not know if I agree with that. Having played the game a number of times, we do have evidence of Kessel, Trouba, and Subban being Actually Good. Devan Dubnyk has been known to also be good in EHM. There are a lot of question marks with the younger players. I’ve seen them thrive and also suffer. Who knows what version of Jones, Kotkaniemi, and Kakko we will get. (I am assuming Boqvist will not even be ) The larger concern would be with older players. Only the very good players age gracefully in EHM, older players who have fallen off just continue to fall. Nielsen, Boychuk, and Filppula are targets for that. On paper, this team’s defense is half good and the wingers are half good provided we get lucky with Kakko. But the depth is iffy at best, the goaltending is almost a coin flip, and the centers are a real sore spot. Two 30+ players, a young Kotkaniemi, a young Boqvist, and David Kampf does not instill me with any confidence. I do not think this is a good team, but I also doubt it will be the worst as the negative WAR may suggest.

Going back to EliasStillRocks’ suggest, he thinks they will finish in the bottom third of the standings. I think he will be right. But we shall see.

The Limitations

As the intent of this experiment is to contrast this squad with the WAR-Optimized team that just dominated the NHL, we need to keep the same limitations in place. This means I have to use CJ’s roster as much as possible. I cannot trade players. I cannot pick up players off of waivers. I can only call up players if I have fewer than eleven forwards, six defensemen, and/or two goalies.

I also have to play the game the same way I did in the WAR-Optimized experiment. I can adjust the roster as needed during the season. I can organize the lines and tactics as needed. If I call anyone up, then I have to call up players from Binghamton. As I kept John Hynes as the head coach for the WAR-Optimized team, I need to keep him in charge as the WAR-Minimized team. Additionally, I am using the same base database file as I did for the WAR-Optimized team. This is not going to be run using Version 12.4 of The Blue Line’s rosters.

The Experiment Results

I created Tom FitzGARless as the GM, took over the Devils, and I was informed by the board of their rather modest expectations:

I also quickly discovered a major problem. The roster that CJ put together could not be used as-is. That roster was worth over $84 million. The salary cap ceiling is $81.5 million. EHM starts you in the offseason so you have time to be cap compliant. But I needed to make a move and do so quickly. Instead of getting a different roster from CJ, I made the executive decision to trade away the player with the highest WAR value. Fortunately for me, that player was Wayne Simmonds with his -4.9 WAR value. He has a cap hit of $5 million. After one board and fan-enraging trade to Ottawa (Simmonds and Case McCarthy for two third round picks), I was back to being cap compliant in August. To keep with the limitations of the experiment, I kept with 22 on my active roster unless someone was hurt and I could call someone up based on my own-imposed ruleset.

After a largely unremarkable preseason, the season began. The media in-game predicted the Devils to finish 31st. This would be in-line with what one would expect with a WAR-Minimized roster. After all, all of the players in real life did not contribute to what the model defines as winning hockey. This was a promising sign for the experiment. Then the season began.

Within the first month, I quickly learned that my concerns about the centers were real. Nielsen went from do-everything center in the first few games to a rotating fourth-line center with Boqvist. Kotkaniemi had to be kept to a more limited third line role. Filppula did quite well to start the season and so I leaned on whatever I could get out Kampf. The good news was that Subban and Trouba were the goods, Erne and Perlini proved to be effective left wingers across Kessel and Frolik, and I got Good Dubnyk in net. He posted high save percentages early on and would maintain them throughout the season.

What this resulted was a mixed bag of actual results instead of a string of losses like the team we intentionally wrecked in the last experiment. Imagine a team that would drop three in a row and then win two, then go alternate wins, and then continue on in a similar vein. This was a team that would never go on a long winless streak. This was a team that was also not good enough to go on a long winning streak. These WAR-Minimized Devils were able to get some big wins over top teams, fall to the worst, and stay competitive in general. They did not get blown out very often - thanks in large part to Dubnyk - but they also did not win a ton of games by more than three goals either. (They did crush Columbus 7-2 in March out-of-nowhere so that was fun.)

This was a bit more impressive that it seems given the number of injuries they faced. Most were less than four weeks, but the timing of their injuries often led to a call-up or two for a game at points in the season. For example, it was not enough for, say, Kessel to be out for ten days. Nielsen would then get hurt a day or two later and then Frolik would also be out for about a week, which would then cause me to call up Mikhail Maltsev or Nathan Bastian for a game or two. I wished I could keep them since even they were doing better than Nielsen or Boqvist, but I had to hold to my rules. There was one extended call up and that was because something happened to me in EHM that I had not experience before.

On November 23, 2019, Jimmy Howard was seeking out his first win of the season in a start against Detroit. He made 11 saves and then something snapped. Dubnyk needed to come in to replace the 35-year old goaltender who was in a lot of pain. The Devils dominated the game in a 4-0 win. Howard would get the win since he was the goalie of record, but he lost big time:

I have seen some long injuries on other teams. But I cannot recall a player tearing his MCL in an EHM season from my own team. A seven month period of recovery meant his season was effectively over. It would get worse in the next month. Gilles Senn was called up to be the #2 goalie to Dubnyk, who was taking the lion’s share of starts anyway.

After this injury to Howard, the Devils would reach 50% point percentage with a record of 10-10-3 on November 26. The Devils would remain around that level for the remainder of the season. Senn did not play a lot but he performed at about the same level Howard was when did go between the pipes. Senn would need to continue this as Howard announced in early December that he would retire after the end of the season. On December 17, 2019, the major event concluded earlier than expected:

Howard straight up retired and the NHL gave me $4 million for it. Senn now had to be kept on the Devils roster. I had to hope that neither Dubnyk or Senn would get hurt. They would not. Others would as the Devils floundered around the 50% point percentage mark.

This continued to be the case when other injuries arose. The other major injury was when team captain Boychuk suffered a fractured shin on March 7, 2020. He would be out for three months, which meant he was knocked out of the season. This did not necessitate a call-up unless someone else on defense suffered an injury. There were a few minor ones so Matt Tennyson, Joe Morrow, and Josh Jacobs did get a little time with this roster. It did not really impact their performance one way or another. Dubnyk would tear his UCL on April 4, but that was the last game of the season so it was moot that he would be out for four months.

As expected, this team was not a playoff team. As not expected, this team was not crummy or battling Los Angeles or Ottawa for dead last. This team was the definition of a below average team. Nobody on the roster was named to the NHL All-Star Game. Nobody scored 30 or more goals or put up 60 or more points. They never flirted with a playoff spot, although they were rarely more than 15 points outside of the final one in the East. The only players that were among league leaders were Dubnyk’s 91.8% save percentage finishing among the top twenty goalies in the NHL (tied for 12th) and Kakko’s 45 points placing him fifth among rookie scorers. They reached the 50% point percentage area in November and just hovered around there for the most part. They were never above tenth in the East but they were just about always fourteenth or better. Here were the final standings for the Eastern Conference:

You can see that this team was not a high-scoring team but they were not goal-starved like Ottawa. They were not the stingiest but they did not have a paper wall in net either. They had a good PK. They had a not-bad PP. Had a few games broke their way and they won a few more shootouts, then they could have been in the playoff picture. Instead, they finished nine points out. They were not good. However, the main point is that they were not the disasters one would have expected from a team filled with negative GAR players. To further support that, here are the overall standings:

EliasStillRocks figured a WAR-Minimized Roster would finish in the bottom third of the league. With the league’s tenth worst record, they finished just inside that third. Again, flip a few games here and there, and the WAR-Minimized Devils would be closer to Actually Mediocre to Below Mediocre. By the way, shout out to Edmonton being a point behind the Devils despite winning 12 out of 19 shootouts.

Here are the stats for the WAR-Minimized Devils at the end of the season. First, by points:

Despite changing centers and being on the second line for most of the season, Perlini led the team in scoring and came quite close to going 30-30. It did help I put him on the left wing for the first power play unit. But as Erne was doing well on the first line, I kept Perlini in place. Ditto for Max Jones, who was the third line LW and the LW on the second power play unit. He maximized his minutes. Purely from a production perspective, the wings carried the offense by quite a bit. If only this team had better centers and depth.

Second, by average rating:

Dubnyk was the team’s MVP and their best player by far. A 91.9% save percentage does not seem that great but in this EHM season, it was among the better ones in the league. He kept the Devils in a lot of games and secured leads that would guarantee at least a point many times. An average rating of 7.88 is fantastic and so the Wild man was a cornerstone for a team that was not good but not terrible. This may be a bit much, but he was a real human being and a real hero for this simulated season.

Additionally, the average ratings for the wingers and the top half of the defense were all quite good. Kessel, Perlini, Jones, and Erne were very, very good. Trouba-Subban was my primary pairing and they performed admirably. Gardiner was very solid as well. Kakko had a very fine rookie season and Frolik was pretty good too.

The average ratings also point to the issues I figured this team would have: depth and center. The bottom half of the defense were either just below seven (Boychuk, Johnson) or quite a bit below it (Holzer, Hjalmarsson). The team had five centers and they took five out of the six lowest average rating spots. Filppula started off the season well (which was when the headline photo was taken) and his performances fell off a cliff. Kotkaniemi was frustratingly inconsistent. Boqvist and Nielsen traded 5’s and 6’s in rotation. Kampf eventually became the first line center and he was not first line quality. Senn was also not that good in his spot duty, but that was the least of the team’s issues. If you gave this team two to four legitimately good centers and one legitimately good defenseman to replace some of these names, then may be they would have been in the playoff bubble instead of consistently being eight to fifteen points from the final playoff spot.

And here are the final goaltender stats. For some reason, EHM does not retain the stats of retired players. So when Howard announced his retirement, it was like he never existed. That is why you only see Dubnyk and Senn here:

Again, Dubnyk ruled, Senn was not good, and Howard - while not listed - was better than Senn and nowhere near as good as Dubnyk.

I continued the season just to see what would happen with the playoffs and the World Championships. Nielsen announced his retirement on April 21 (the season ended on April 4). Several Devils went to the WCs. Sharangovich from Binghamton represented Belarus, who was relegated. Studenic, also from Binghamton, represented Slovakia, who did make it to the knockout round and was knocked out quickly by Russia, 5-0. Kampf represented the Czech Republic, who got whacked in the Quarterfinals to America, 4-0. Holzer represented Germany, who did not qualify for the knockout round. Senn put the pads on for Switzerland and was their #1 goalie - they were eliminated by Canada 3-1 in the Quarterfinals. Lastly, Trouba and Kessel represented America, who won the Bronze. I would like to have thought Frolik (who was hurt in the offseason) and Dubnyk would have went if they were not injured. Still, plenty of WC representation. As for the Cup, Colorado beat Buffalo in five games. For some reason, EHM has a lot of time for the Sabres being good - perhaps Kevyn Adams can get a copy of it and learn what could be.

As one final postscript, at no point was I at risk of losing my job. Sure, the board did not like the Simmonds trade that had to be done to be cap compliant. Sure, they were notably unhappy after a handful of signficant losses to the NY teams. Yet, every month, I received the same message: “The New Jersey directors are generally satisfied with your performance as General Manager. That was upgraded to “pleased” in May. I was expected to improve the roster; I did nothing of the sort; and they were still pleased given that the Devils went 37-34-11 and finished 12th in the East. I suppose they wanted not much and I overdelivered.

Experiment Conclusions

EliasStillRocks turned out to be right that a WAR-Minimized roster would finish in the bottom third of the league. I do not think he expected that they would finish that high up. I certainly did not.

It is arguable that I or CJ could have put together a worse roster that was comparable in terms of negative GAR. It is true that I was lucky that Perlini, Erne, Kakko, and definitely Dubnyk were good; as well as the major injuries striking the #2 goalie in Howard and a team captain in Boychuk that was not a big-minutes player like Trouba or Subban. It is also true that since EHM does not contain the same information as is required in the GAR model, using it as a platform for comparison’s sake is not ideal. I recognize it is comparing cantaloupes to honeydews. However, my interpretation of the findings from this experiment do cast some doubt on the value of this model. A team optimized in GAR was absolutely amazing. A team intentionally built to be very bad by the model was not as bad as we would have expected. The roster was filled with players thought to be detrimental to their teams and the squad finished at 51% point percentage - even with injuries considered. The media in the game thought they would be the worst. They were not; Ottawa were pure trash, but not this New Jersey team. This suggests that the negative value identified by the GAR model could be specific to their situation and not necessarily in general. In other words, if you took Subban and put him on another team, then perhaps he would provide more (or less) value.

Ultimately, this drives one of my complaints about what the model results. I can agree that, for example, Subban was bad in real life. But by no means was he among the worst defensemen on New Jersey much less in the entire NHL as the least valuable. Not on a Devils team that featured Mirco Mueller getting dominated more often than not. As much as I do not like defending players from that franchise, I doubt that Kakko was the worst forward in the NHL last season either (recall he had the lowest GAR among forwards at -9.5). Phil Kessel may have not done as much as Arizona expected but I do not think he made them actively worse. The larger point is that while the GAR model may identify players who are good, I question its efficacy in identifying players who are truly bad.

Yes, this team did not make the playoffs but they were competitive. That puts lie to the notion that this team was filled with some of the suckiest sucks whoever sucked in 2019-20. This experiment helps support that thinking despite its limitations. And a model that appears to only work one way is not really a model I have a lot of time for, much less $5 for the Youngren’s Patreon. Your mileage may vary on that, though.

What’s Next & Your Take

Thank you again to EliasStillRocks for the idea for this experiment and to CJ for providing the roster I mostly used to make it happen. What is your reaction to how this team performed? Did they do better than you thought? Did they do worse? How would you have made them even worse if you could? Please let me know in the comments.

The next experiment comes from an idea brought up by alslammerz in response to the Let’s Ruin the Rangers Part 2 post last week:

As a future FM inspired experiment, how about a “you can’t win anything with kids” roster. The salary cap might be tough to do and the goalies might be bleh, but a full roster of just young players, like a cap compliant Young Guns team from the lone World Cup we’ve gotten in awhile.

Downstream in the comments, I figured out that this was possible to do as an under-23 team. It would not be a U-21 team as I mistakenly wrote in that comment, but an age limit of 23 would make it possible. Cap compliance was my concern since most players are on entry level contracts and 23 players with contracts making no more than $925,000 would not hit a floor of $60.4 million. However, thanks to CapFriendly, Auston Matthews, forward Sebastian Aho, Matt Tkachuk, Patrick Laine, Zach Werenski, Charlie McAvoy, and Jakob Chychrun, it is possible to hit the floor with them and sixteen players on maximum value ELCs. (Aside: Which one of those seven do not belong? Related: Seriously, Chayka, how do you GM?) So I will work that out in the coming week and run the experiment. If you have a good idea on which team I should use for this and who the GM of this U-23 squad should be, then please let me know in the comments.

Lastly, if you have any additional suggestions for future EHM experiments, then please leave them in the comments as well. Thank you for reading.