The Oilers partially based their acquisition of Mark Fayne on information gained from analytics. James Mirtle of the Globe and Mail referred to Fayne as an “analytics darling.” (Not a term I would use, although somewhat common in the analytics lingo.) He was one of several players the Oilers acquired based on advanced statistics, which included David Perron, Teddy Purcell, Benoit Pouliot, and possibly Rob Klinkhammer. Because Fayne was considered a top-4 defenseman, a lot was expected from him. After all, he did play top-pairing minutes with Andy Greene in New Jersey.

At the beginning of the 2014/15 season, then coach Dallas Eakins put Fayne up against the toughest competition, along with Martin Marincin. Fayne (and Marincin) did not flourish. He, along with the Oilers, crashed and burned by December. Even after Eakins was fired, and Todd Nelson was brought on as interim coach, Fayne never did recover. He continued to struggle mightily. “But that’s because he faced the toughest competition,” one may interject.  Another could argue, “He’s the best defender on one of the worst defense corps in the league.” A few many even believe that Fayne played well, or as well as can be expected on a bottom-dwelling team. Finally, an embittered fan might explain Fayne’s performance with a more succinct, yet familiar refrain: “Because Oilers.” All these explanations could have kernels of truth to them, or not. Does the data support any of these explanations? Going the way of the scientist, I wanted to investigate the data and see what I could conclude.

My approach was scientific in that I began my data quest with questions (i.e., hypotheses) and then attempted to figure out how, or if, the data answered these questions (i.e., supported or didn’t support my hypotheses).  My questions included: Did Fayne feign struggling this season? (Did he actually struggle or am I seeing things that are not there?) If his struggles were real, what was the influence of the toughness of competition and quality of teammates? Assuming that his under-performance cannot be blamed on his competition or teammates, was their anything in his time with New Jersey that could have been a red flag? Was there some bit of data that should have given management (and analytics-inclined fans) a sign as to whether he could handle top-pairing, or even 2nd pairing minutes?

Scoring Chance +/-

I’ll begin with the most direct evidence by borrowing data from David Staples of the Edmonton Journal, who diligently tracks scoring chances from every game. Specifically, he records whether players contributed to offensive scoring chances or made errors in scoring chances against. He then computes a plus-minus of scoring chances. (Originally, the metric was named after Roger Neilson, who created it.). A positive value shows a player contributed to more offensive scoring chances than to errors when defending against scoring chances, and a negative value shows the opposite. The complete Oilers 2014/15 season can be viewed here. Fayne’s scoring chance differential was -2.52 per 60 minutes, which was 2nd worst among the Oiler’s regular defense; slightly worse than Ference, whose differential was -2.44. Only counting his errors on scoring chances against, Fayne averaged 7.76 errors per 60 minutes. This was actually better than most Oilers, except for (former Oilers) Petry and Marincin. Hence, his limited offensive contributions to scoring chances was the main culprit in his negative scoring chance differential. (Spoiler alert: His limitations on offense are a pattern throughout my analysis.) In any case, I submit that this is the first piece of evidence that Fayne under-achieved.

Passing Metrics

A second piece of direct evidence is passing data collected by Ryan Stimson and his colleagues for his Passing Project. These passing and shot attempt metrics are measures of a player’s offensive contributions that go beyond what is collected by the NHL. The metrics in the graph below include:

• CC% and CC/60 = Corsi Contribution (or Shot Attempt Contribution), which are individual shot attempts, primary passes leading to shot attempts, and secondary passes leading to shot attempts. These are given as a percentage (i.e., proportion of shot attempts a player is involved in when on the ice) and per sixty minutes. These metrics tell you how much offense goes through that player while on the ice and also how often they contribute.

• Composite SAG and SG represent the total number of shot attempts and shots a player generated from both primary and secondary passes per sixty minutes. SAG/60 is solely for the player’s primary passing contributions.

• Entry Assists represent the number of controlled entries a player assisted on. This is determined by the number of passes in transition (prior to entering the offensive zone) that was recorded for each player.

• SC Contribution% and SCC/60 are identical CC% and CC/60, but represent only the scoring chances a player was involved in. Passing data for scoring chances was combined with War-on-Ice’s scoring chance (link to definition) data to arrive at a player’s total number of scoring chance contributions. SC SAG/60 represents the number of scoring chances set up from a player’s primary passes.

 

Here is the way I read this graph. The top one-third percentile (67%+) is top-pairing defenseman range, 33%-66% is 2nd pairing, and below 33% is bottom-pairing. Reading from left to right, Fayne’s offensive contributions involving shot attempts (first 4 columns), he sits around the 2nd-pairing range. He is also 2nd-pairing for entry assists (i.e., controlled passes into the offensive zone). That’s one thing I remembered from watching games. That he had a solid first-pass on defensive zone breakouts. Where he drops off rapidly is his contribution to quality scoring chances, especially as a set-up guy. He was 10th percentile in primary passes leading to scoring chances. He then again sits around the 2nd-pairing range for scoring chance contributions, which includes scoring chance shots, as well as primary and secondary passes.

Is this evidence that Fayne under-performed? Depends on what we were expecting. As a top-pair, he under-achieved. As a 2nd-pair, he performed as expected, except when it came to setting up quality scoring chances. Surprisingly, Sekera is not that strong in setting up scoring chances either. Based on this, Sekera paired with Fayne might not be a good idea. Unfortunately, Schultz appears to be the only defender particularly good at setting up quality scoring chances. Both Fayne and Schultz are right-handed, so it’s unlikely, although not impossible, to pair them.

Quality Shots: Scoring Chances & High-Danger Scoring Chances

Next, I look at indirect evidence, namely, group measures such as a team’s shot attempt and scoring chance metrics. Based on these measures, especially the relative metrics, we make plausible inferences or guesses about a player’s contributions. I begin with Scoring Chances (SC) and high-danger scoring chances (i.e., shots from the slot area) provided by war-on-ice. These metrics are especially useful because they account for shot quality; locations and shot types that greatly increase shooting efficiency. (Unless otherwise stated, I present these scoring chance measures per 60 minutes.) We know that about 20% of shots from the slot (High-Danger) go in. Suppressing shots from the slot, then, has much more impact compared to shots from the blue-line, which only have a 4% efficiency rate.

Starting with suppression of quality shots, and relative to 170 defenders who played at least 750 minutes, Fayne’s High-Danger Scoring Chances Against  (17.16) and Scoring Chances Against (28.39) ranked 144th and 132nd, respectively. How do these metrics compare to other Oilers’ defensemen? Fayne’s High-Danger SC Against is 3rd worst, ahead of  Martin Marincin (19.13) and Keith Aulie (21.29).  With respect to general Scoring Chances Against, Fayne’s rank is better, but nothing special. When Fayne is on the ice, the Oilers are no worse, or better, when it comes to suppressing quality shots. In other words, the team’s defense is average with Fayne involved.

But when comes to the Oilers generating scoring chances with Fayne on the ice, the lack of quality shot production is glaring. Compared to other regular defenders, he ranked 169th in High-Danger Scoring Chances generated (11.12) and 167th in Scoring Chances generated (28.39). I’m hesitant to say this, but the numbers are pretty clear: Fayne’s scoring chance generation metrics are among the worst in the league. This wouldn’t be so bad if his defense compensated for this lack offense, but it doesn’t.

I could look at other numbers, like shot attempts for and against, to support the idea that Fayne struggled, but I think what I showed so far is sufficient. With the Fayne on the ice, the team’s defense in suppressing quality shots is par for the course , which isn’t saying much. What’s worse is that the team’s ability to generate quality scoring chances is severely hindered. So how does this all translate into team scoring? Based on shots from all areas, war-on-ice provides a Net Goals metric for each player. With Fayne on the ice, the team’s expected Net Goals was -0.41 (per 60 min.), which when multiplied by his time-on-ice, is -7.41 net goals. Only Aulie was worse (-0.54 per 60 min.).

Quality of Competition

The Oilers were expecting a solid top-4 defender, which his SAT% metrics with New Jersey suggested, but his Oilers’ numbers presented so far don’t support this notion. One explanation could be that Fayne faced the toughest competition on the team, which he did. However, as I have shown in previous posts (e.g., Gryba vs. Marincin & Sekera vs. Hamilton), a player’s quality of competition can be controlled. Specifically, using David Johnson’s WOWY (With-or-Without-you) tables, a player’s shot-attempt differential (SAT%) can be measured against different levels of competition. Moreover, the player’s SAT% can be compared to other players across similar levels of competition. If Fayne’s SAT% took a big hit only against tough competition, then we could conclude that quality of competition was major factor for his struggles. Let’s see if that is the case.

In the graph below, I present Fayne’s SAT% at different levels of competition (i.e., quality of competition [QualComp] SAT% ranging from 56% to 46%), I also compare him to Ference, who many would consider bottom-pairing at best, and Schultz, who some consider bottom-pairing (like myself), but on a defensively weak team like the Oilers, he is passable as a second-pairing defenseman. Before the analysis, my expectation was that all three were dominated by the strongest competition (i.e., QualComp SAT% of 54% or greater), but would improve considerably as QualComp softened to 50%. Then as competition dropped to players with a negative SAT differential (QualComp SAT% < 50%), Fayne’s SAT% would become positive. Finally, I was hoping that Fayne’s break-even point–the level of competition that players hold their own with a neutral SAT differential (50%)–would also be at least 50%, if not more. Here is what I found (click on graph to enlarge).

 

The first thing I noted was that Fayne was more similar to Ference than to Schultz, which is to say, not that good. In fact, Schultz looks amazing in comparison, but I don’t want to get carried away. Even against weaker competition, Fayne barely kept his head above water. This is most clearly seen in his break-even quality of competition SAT% of 46.9%. Notable players at this level include Devant Smith-Pelly, Nate Prosser, and Dalton Prout. Smith-Pelly I’ve know because he sometimes played with Anaheim’s top line, but in reality, he’s a bottom-6 forward (ranking 310 out of 410 forwards with 1.12 points/60 min). Prosser is in a bottom-pairing role for Minnesota and Prout is Columbus’ 4th more used defenseman. Overall, then, Fayne is holding his own against the bottom-half of opposing teams. This is what I would expect from a marginal second-pairing defenseman, or a solid third-pairing defenseman.

Quality of Teammates

I was still questioning whether Fayne was really that bad. Could it be because he played more minutes, compared to Schultz, with weaker possession players? One combination is the Oilers shutdown line, mainly Boyd Gordon and Matt Hendricks, and the other is any line with Yakupov. (Unfortunately, Yakupov is the weakest possession forward among the Oilers’ regulars). Fayne played 188 minutes with Hendricks and Gordon, whereas Schultz played 120 minutes; so that’s a significant difference. But then he played 136 minutes with Yakupov and Roy, and Schultz played 174 minutes with them. Moreover, when these players were on the ice with Schultz, they had a better SAT% (50%) then with Fayne (46%). Does this mean Schultz is a better defenseman than Schultz? I’m not suggesting that, but as these metrics indicate, Schultz appears to do a better job at generating offense than Fayne, which helps compensate a little for his deficits on defense.

Who Fayne had as a defensive partner also impacted his SAT differential. With Nitikin, it was almost respectable: 49.6%. With Marincin, it was only 46.3%. (Fayne’s adjusted SAT% was 46.7%.) Seems that being paired with a rookie, especially against the toughest competition, did not help Fayne. A few players did help Fayne gain a positive SAT differential. The main possession driver was Eberle, who helped Fayne exceed 50% when he was paired with either Marincin or Nikitin. I could go on to describe other combinations, but fortunately Micah McCurdy has done this work graphically with what he calls WOWY diagrams. This one below, if you follow it carefully, shows how Fayne and his teammates’ SAT differentials changed depending on whether they were with him or without him. I’ll use Marincin (#85) as example to help you understand the diagram (click on the diagram to enlarge).

 

(You may want to increase your font size to for this part.) The bottom-right half, below the red line, is positive possession (good). Above the red line is negative possession (bad). Red-boxes indicate a teammate’s SAT differential without Fayne. If you look to the top-right and below the red line, you will see #85 boxed in red, which is Marincin (SAT% = 50.7%) without Fayne. Then if you follow the line from #85 to the left, you’ll cross the cut-off line (where SAT% = 50%), into negative possession (SAT% < 50%), you’ll see #85 in black. That’s Marincin and Fayne together with a 46.3% SAT differential. In short, Marincin appeared to be better without Fayne than with him, although that’s probably too simple a conclusion. If you look directly above this point (85 in black), you’ll see #85 in blue, which shows Fayne without Marincin. Fayne, then, whether he played with or without Marincin, his SAT differential changed very little. Still, there was small effect in defense with Marincin, namely, there were fewer shots against.

I’ve only explained the effect of one teammate. So as you can see, this elegant diagram packs a lot information into one visual image, which is very cool.

The main take-home message of this graph is how many teammates’ SAT% improved with Fayne (numbers in black). You can barely see it, but one player’s SAT% was positive and that was Pouliot (#67 in black, which is behind #67 in red). That’s it. If Fayne was driving possession, if players gained an advantage offensively or defensively with Fayne, we would see a pattern in which “red” numbers move from left to right toward “black” numbers. We don’t see that pattern, even for Pouliot.  Pouliot’s SAT% is unchanged playing with or without Fayne. What we can conclude from this pattern is that Fayne is not a possession driver. Indeed, several player are worse with him than without him, which we can see in all the red numbers moving from right to left (e.g., Eberle #14 & Hall #4), from positive to negative SAT differentials.

What did Oilers management or analytics followers not know?

I believe I’ve shown that Fayne struggled as a top-pairing, and even 2nd-pairing defenseman this season. As per Staples’ scoring chance data, Fayne had the 2nd lowest plus-minus scoring chance differentials among the defense. Based on Stimson’s passing project data, Fayne’s passing effectiveness puts him in the range of an average 2nd-pairing defenseman. Notably, though, he rarely sets up quality scoring chances through primary assists. From the perspective of the Oilers’ generating and suppressing quality shots, the team (a) allows more (compared to the team’s average) High-Danger chances with Fayne on the ice, and (b) generates fewer high quality shots. Overall, the team’s expected Net Goals with Fayne on the ice is about -7. Is this due to his toughness of competition? I don’t see evidence for that . What I noted is that the team’s SAT differential with Fayne on the ice was not that strong even against weaker competition. As to the reciprocal influence of teammates, I noted that Fayne tended to reduce a teammate’s SAT differential. Some teammates improved Fayne’s SAT%, like Eberle, but given the one-way relationship, it supports the notion that Fayne is not a possession driver.

How did a strong possession player, a top-pairing defenseman on a strong defensive team like New Jersey, become what almost looks to be a possession liability? What did management miss (if anything)? What did the casual analytics follower miss? I say “casual” because Ryan Stimson, the Devils’ analytics blogger for SB Nation, was not a fan of Fayne and he had analytical reasons for that.

First, there is a hint that Fayne’s SAT% in 2013/14 was inflated by certain teammates, specifically, Greene and Jagr, who was the Devils’ best possession forward that season. When Fayne was paired with defensemen other than Greene (time-on-ice = 248 min), his SAT% fell significantly by over 10% to 46%. When Fayne wasn’t with Jagr, his SAT% fell by 5% to 52%.

Second, Stimson had collected passing data from the Devils 2013/14 season. (That was before he expanded the project league-wide.) One statistic that he focused on was a player’s passing efficiency, in particular, what percentage of shot attempt passes lead to shots on goal. He referred to this metric as Shot Attempt Generation Efficiency (SAGE). I think the idea is that if a player’s whose passes lead to more shots on goal, the player presumably has better passing skills, which, seems to me, involves awareness of the developing play, mobility, puck control, and accurate passing. In Stimson’s analysis of the Devils’ defense, Fayne had the lowest SAGE, which was about 30%. As Stimson concludes:

“Mark Fayne generated the third most shot attempts [compared to Devils’ defenders], but was the least efficient by a considerable margin. I think the Devils will be just fine in replacing him in the lineup. Certainly the volume [of shot attempts generated] was solid, but if there’s no efficiency, it’s just empty possession and inflating Corsi totals.”

I think Stimson’s findings provide a reasonable explanation of why Fayne did little to help the Oilers’ offense. We see confirmation of his passing inefficiency in the 2014/15 passing metric data, which shows a mediocre rate (45th percentile) of scoring chance contributions and an abysmal rate (10th percentile) of primary passes that lead to scoring chances. This 2013/14 data was not available to the public until after Fayne was acquired by the Oilers. Hard to say if Oilers management would have considered it seriously, given that it’s not official NHL data. Nonetheless, I think this is evidence that helps explain Fayne’s offensive weakness, which when combined with evidence of his SAT% being heavily dependent on Greene and Jagr, helps explain to some degree his struggles with the Oilers.

Recommendations

Based on this analysis, I offer the following recommendations. (1) Fayne needs to be on the ice with a defenseman who is a puck-carrier and who is more efficient in their passing, especially in the offensive zone. Although I noted above that Sekera might not be a good idea, he does fit the bill of being a strong passer. And Fayne is better than Sekera in executing passes that allow controlled offensive zone entry. In other words, they do appear to compliment each other. As a 2nd pairing combination, Fayne with Klefbom could also work. Klefbom’s passing strength is in the offensive zone, which may help offset Fayne’s limited passing efficiency. (2) Against opposing top-6 forwards, Fayne should not be paired with weaker (i.e., bottom-pairing) defenders, especially rookies and prospects. (3) Reduced time-on-ice with weak possession players, otherwise these line combinations can be a recipe for puck possession disaster. Here I’m thinking of the Oilers’ shut-down line (Hendricks, Letestu, & Klinkhammer) and any line with Yakupov. I do realize Fayne was often put on the ice with the shutdown line, but their SAT% together–47.8%– is simply not that strong. Being realistic, though, the Oilers depth on defense is still lacking. There might not be a better choice than Fayne for this kind of deployment.

This analysis took me much longer than expected. Part of the reason is that I was surprised by some of my findings. I wanted to make sure I wasn’t missing something important, or concluding too much from too little. As always, I’m receptive to new evidence and lines of reasoning. If you believe that I have missed important information or reasoned wrongly, please comment below. Also, if you have any questions or confusion about what I presented, please ask away. My underlying approach in this blog is to make analytics as accessible as possible to hockey fans. Thanks for reading.