Before I start this blog, I want it to be clear, I have no problem with analytics. I respect the field and follow many great analytics people on twitter. It is a great tool. It’s not something I plan to write about often, as there are some great people out there that do a wonderful job of breaking it down and I wouldn’t do them justice.
Notice I did not say it was new. It’s not, coaches and fans have been talking about various components of analytics before many of the people responsible for bringing in more mainstream were born. Zone starts; driving play, creating more than you give up, etc. are as old as the game itself.
Very recently, an attempt has been made to bring analytics to the masses and for the public to be able to look at and quantify those numbers. I applaud this. Any tool that makes us enjoy and understand the game is welcome.
Now that doesn’t mean that people who don’t enjoy it or follow it are burying their heads in the sand or don’t understand it. It doesn’t mean that you have to follow analytics to understand the game. It doesn’t mean that everyone that writes an article about analytics is using it correctly or understands it. It doesn’t’ mean that everything in analytics is correct or that it’s not open to criticism or that anyone that dares criticize it doesn’t understand it. Most importantly it doesn’t mean that people that use analytics do not have confirmation bias and those that only watch the game do.
Most of the above paragraph is fairly obvious. There are many hockey executives that don’t follow analytics and their knowledge of the game can’t be questioned. Writing about any subject doesn’t make you an expert. Analytics is a complicated field even people that are looking at the same numbers may not agree. The last point though I believe needs some expansion because some of the analytics world have assumed analytics has an advantage over watching by “eye” because it eliminates confirmation bias.
One of biggest concerns in science is confirmation bias; it is a danger that you will unconsciously interpret your data to prove your hypothesis. Once you have spent hours, days and months of your time doing experiments and crunching numbers to form a conclusion. Your experiment results will skew toward your conclusion or hypothesis. It is why peer review is an essential part of science. There are many articles that touch on this I choose one.
The same thing is true in hockey analytics, if you A) come up with a conclusion of a matter before looking at the numbers, how you look and interpret those numbers will be affected B) if you spend hours and days trying to make a conclusion and then watch the player you will automatically see those things that confirm your numbers.
We all suffer confirmation bias, whether I subscribe to analytics or not. We need to remove this myth that analytics is the ONLY unbiased way to interpret the game. There is NO such thing. No matter what our view is and how we came to that conclusion we can be questioned. Analytics is no different than any other science or opinion. It is open to criticism and debate.
It’s a great tool and used in conjunction with watching the game can enhance your viewing pleasure and understanding of some aspects of the game that you may not normally think of. It’s a fun and new way to enhance our view of the game.
Enjoy the game! If you want to analyze it and crunch numbers do so to your hearts content, and if you do a great job I will enjoy reading it and yes, I may debate it. Just as you would debate my analysis of a player based on over 40 years of watching and playing the game. But make no mistake, there will be debate and you are not automatically right because you broke it down on a spreadsheet, anymore then I am right because I have watched the game for 40 years, but it will be fun to debate it and watch it. Isn’t that the point?
On Twitter recently, I was told that I shouldn’t be writing a hockey blog simply because I referred to Taylor Hall as an elite player. I couldn’t let this ironic opportunity pass me by, so here is why, from an analytics perspective, I think Hall is an elite player.
First, let me define elite player. Statistically, I define elite as any measure that is about 2 standard deviations above the mean, or in simpler terms, 98% better than all other comparison players. Typically, we would compare forwards with forwards, and defensemen with defense. Also possible to be more specific by comparing centers and wingers separately, but in presenting evidence for Hall, I’m going long and comparing him to all forwards.
Second, what would be a reasonable time-line? I think 4 seasons gives pretty reliable measurement over time. Goal-scoring can be highly variable from season to season because of shooting percentage variability, but after 4 seasons, shooting percentage approaches a “true” or reliable value.
Third, I focus on even-strength (5v5) metrics because 80% of the game is determined at 5v5 and moreover, it is more challenging to produce as compared to the power-play. These 2 factors, then, would be my basis for arguing that for 5v5 represents a more accurate assessment of a player’s ability.
Fourth, what measures would I use as evidence, that is, what are nuts and bolts of an elite player? The 6 measures (5v5; per 60 minutes) are :
Goals
Primary Assists
Primary Points
Points
GF% Relative-to-Team (i.e., team’s goal differential with the player on the ice vs. the team’s average goal differential)
Individual Proportion of Points
Why primary, but not secondary assists? Recent analysis has shown that Primary Assists are repeatable over time, but Secondary are more random. Thus, primary Assists tend to represent a player’s “true” skill as a playmaker. This then, in theory, affects which points are the more reliable points. By excluding secondary assists, we are left with primary points (goals + primary assists). But does that mean secondary assists are meaningless in assessing a player’s ability? I would argue, no.
In the sample (297 forwards, 2012-16), I computed the correlation between primary & secondary assists and found a statistically significant correlation of .49 (p < .001). This moderate correlation suggests to me that players who are better playmakers–because they are likely strong passers in general–tend to have more secondary assists. The points metric, then, is included because it captures three meaningful metrics: goals, primary assists, & secondary assists. The whole is greater than the sum of the individual parts, or something like that.
Finally, the last 2 metrics attempt to the player’s impact on the team. GF% Relative-to-Team answers the question: Relative to his teammates, how well does the team do in outscoring the opposition with the player on the ice? Individual percentage of points tells us the what proportion of points is the player directly involved in when the team scores.
So how many players are in the 98th percentile of any given metric? Using the last 4 seasons, including the current (2012-16), with players over 2000 minutes of ice-time there are 297 forwards, which represents the top-9 forwards of all teams (i.e., 270 players). There is a few ways to compute 98th percentile. The simplest way, which I think makes intuitive sense for the non-statistical-minded person, is to look at the upper 2% of players on any metric. Thus 2% X 297 = 6. In short, we want to know the top-6 players on a metric.
So how does Hall rank in each metric? I present the rank plus the metric in parentheses (per 60 minutes; except where otherwise noted):
Goals: 58th (0.82)
Primary Assists: 3rd (1.68)
Primary Points: 7th (1.91)
Points: 3rd (2.5)
GF% RelTM 17th (+9.2%)
IPP 1st (87.6%)
Thus, in 3 of 6 metrics, Hall is in the upper 2% of the league: Primary Assists, Points, & Individual Proportion of Points. In fact, he ranks 1st overall in his proportional contribution to the team’s points while on the ice. His Primary Points rank of 7th is so close that I might as well call it upper 2%. So that’s 4 metrics that rank as elite (as I have defined it). Next, he ranks 17th in GF% RelTM, which is about 95th percentile, and 58th in goals, which is about 80th percentile.
Overall, 4 out 6 of Hall’s metrics are at the elite level, the GF% RelTM metric has Hall performing better than 95% of forwards, and the goal metric has him scoring at a rate better than 80% of forwards. For me, I think that’s a pretty solid case for being an elite forward. Others may disagree, but if you’re going to post your disagreement, I would kindly ask for your rationale. (Remember, the measures do not include the power-play.)
Thanks for reading and please post your comments below.
Walter
Data courtesy of David Johnson’s stats.hockeyanalysis.com
The short story (spoiler alert) is that the Oilers have improved over last season. If that’s all you wanted to know about the Oilers’ progress, there you have it. If you want to know what specific ways they have improved, read on.
2014/15 Season Series Comparison
To account for quality of competition, I compared even-strength (5v5) shot metrics from each Oilers’ game with metrics from the 2014/15 season series against the same team. I referred to the difference in metrics as a Progress Index. As you might expect, a positive Progress Index indicates improvement over last season, whereas a negative Progress Index indicates the metric is worse relative to 2014/15.
Over the long run, the quality of competition balances out in a team’s schedule. The 4 metrics I tracked were:
(1) Weighted Shots (1 shot attempt = 1 point; 1 goal = 5 points),
(2) Shot Attempts (in following analytics blogger, Micah McCurdy (@IneffectiveMath), I will now refer to as ‘Shots’),
(3) Scoring Chances, &
(4) High Danger Scoring Chances (i.e., Scoring Chances from the slot).
(These last two metrics were developed by war-on-ice to help account for shot quality.)
In my report after game 30, I noted that the Oilers had improved significantly to game 25. But then because of injuries (e.g., McDavid, Pouliot, & Yakupov), certain players struggling (e.g., Fayne & Nurse), and other factors (e.g., the team stagnating in their execution of systems), the team’s metrics were trending downward to levels worsethan last season. The good news: Although requiring 10-15 games, the Oilers recovered! Let’s take a look at the differential shot metrics below.
Shot Metric Differentials
As we can see, from games 26-35 the Oilers were performing worse than last season on all metrics. But then something happened from game 36 onward, especially from games 41-45, in which they played some of their best hockey of the season. Sure, Pouliot and Yakupov returned from injury–and Yakupov only very recently. But they also lost their best defenseman, Klefbom, at game 30. What changed?
I think the answer is in Todd McLellan’s observation that the Oilers had plateaued in December. He stated that he looked forward to practice at home in January to get the team back on track. With these critical practice sessions to get everyone back on the same page, which includes a rehabilitated (overly dramatic, I know) Fayne, who was down in Bakersfield (their AHL affiliate) to get his game and confidence back. the team responded extremely well. There was even a period of 6 consecutive games (Jan 8-18/16) in which they sported a 54% (scored-adjusted) shot differential (Reminder: I use “shot” instead of “shot attempt.”) and a top-5 shot suppression rate.
Seems to me this dramatic improvement kind of went unnoticed by the regular fan because the Oilers were not winning much. But for numbers person, this kind of progress leaves me feeling hopeful and not only because the team is performing better. More importantly, when the team is off track, the coaching staff is able to correct what is going wrong and correct it in a major way. Even after losing Davidson, who has surprisingly earned his way into being a top-4 defenseman, the team has stayed the course.
Not let me breakdown where the improvements have happened, namely, separating offense from defensive.
Offense
Generally, shot generation has been a roller-coaster ride. Still, from game 36 onward, the progress indices have been positive. Over the last 15 games, compared to last season, they have generated +5.9 shots and +3.3 scoring chances per hour. Great to see! On average (over 50 games), the Oilers have generated 2.6 shots and 2 scoring chances more per hour.
In addition to practice time, the return of Pouliot and Yakupov, as well as the addition of Zack Kassian, has helped the team’s offense. Nugent-Hopkins and Eberle had also picked up the pace; both who were struggling, possibly due to slow recovery from illness and injury, respectively. Unfortunately, Nugent-Hopkins was recently injured blocking a shot with his hand and breaking it. At least McDavid will be back next week, which means that even without the Nuge, the Oilers will still have a competitive group of top-6 forwards. (We can thank Draisaitl and his breakout season for that luxury.) What about defense?
Defense
In contrast to offense, progress on defense has been linear. The progress indices in defense was fairly stable over the first 25 games, but then dropped substantially over 15 games (games 26-40). Losing Klefbom at game 30 didn’t help. Perhaps this is the “plateau” (to put it kindly) that McLellan observed. McLellan and crew then got to work, serviced the defensive engine, and it’s been full speed ahead since game 41.
Overall, the improvement in defense appears to be minor because of that terrible 15-game slump. Specifically, the average progress indices per hour are only +1.3 shots, +1.5 scoring chances, and +0.5 high danger chances. But if we focus on the positive, over the last 10 games the Oilers have improved–relative to last season–in suppressing more 6.3 shots, 5.8 more scoring chances, and 1.8 more high danger chances per hour. These rates are an amazing improvement. I don’t expect the numbers to to last, but if the trend stabilizes to half that for the remainder of the season, I’ll be happy.
Altogether, the team’s metrics have improved, especially on defense!
Score Situations
Teams play differently depending on whether they have a lead or are trailing. Leading teams tend to play safer, whereas trailing teams tend to be more aggressive. This means trailing teams, more often than not, out-shoot their opposition during the time they are behind. So I wanted to examine the Oilers in different score situations. Also important to note is that weaker teams tend to be trailing more often than tied or leading, which makes intuitive sense.
Compared to last season, I was interested in knowing three things: Did the Oilers spend less time trailing? When trailing, were they more aggressive? (The Oilers, by their own admission, tended to get down on themselves when trailing last season.) When leading, how well was the team able to hold the leads?
Games when Trailing
Last season, the Oilers trailed their opposition almost half the time: 49.7%. This season, they’ve trailed much less: 36.1%. In particular, they spent less time trailing by 2+ goals, dropping from 20% to 12% of the time. Although their time trailing by 1 goal increased by 5% (19% to 24%), their time with the score tied increased from 36.5% to 42.9% (i.e., 4% more). So if you believe the Oilers are closer in more games this season, your belief is correct! Unfortunately, this hasn’t translated into more time leading–which is the same as last season (about 23%)–but one step at a time.
So they’ve spend less time trailing, but have they been better at pulling themselves back into games? When trailing by 2+ goals, no question! Last season when trailing by 2 goals, their 52.4% shot differential and 42% goal differential was near the bottom of the league. This season, their 58.9% shot differential is 11th best with an even stronger goal differential of 63%. When the Oilers are really down, they are not laying down. There’s more push back, much more than last season, or any season since 2007. In fact, from 2007-2015, the Oilers have the worst shot and goal differentials–yes, even worse than Buffalo–when trailing by 2 goals or more.
They’ve also improved when trailing by 1 goal, although not as much. Their shot differential has only improved by 0.6% to 53.5%, but their goal differential is much better at 50% (42% last season). The difference in goals is due to much better goaltending with save percentage improving to a decent 91.8% from a horrible 87.9% in 2014/15.
The Oilers are a better team when trailing! They are also spending more time tied in games. Both indicators of an improved team, especially the psychological element of not letting their confidence and effort drop when trailing. Ultimately, this motivation is within the players themselves, especially the leaders of the team, but McLellan’s influence is felt here too. In post-game interviews, I have heard him speak of observing the bench feeling down after a goal or two (e.g., shoulders drooped, heads down) and taking steps to motivate them, whether on the bench or between periods. Sounds like a coach who is attuned to the psychology of the team and has constructive ways to encourage the players.
Games when Tied & Leading
This is all well and good, but the team unfortunately has not improved when tied or leading. They give up leads this season as easily as they did last season. When leading, their shot differential of 42% ranks 20th and their goal differential of 33% ranks last. Both the team, who has allowed a boatload of shots (65 per hour; 25th in the league), and poor goaltending (89.1 save%), have lead to this awful goal differential.
Being in a tied game, which I noted above, is a situation the Oilers find themselves in more often. Yet, when tied, they have a rough time gaining a lead. Their shot differential is slightly worse this season than last–46.5% vs 47.7%–as is their goal differential (41% vs 43%). Goaltending cannot be blamed. When the score is tied, the save percentage is a very respectable 92.4%.
Indeed, on top of a poor shot differential, the team’s shot percentage is extremely low (5.2%; whereas 8% is more typical). To see if this low percentage is due, in part, to decreased shot quality, I looked more deeply into the numbers. Indeed, compared to last season, when the game is tied, their scoring chances dropped by 3 per hour. Most of that is due their decreased ability to generate high danger chances, which fell by 2 per hour.
I really don’t have an explanation for why the Oilers have not improved in tied games or when leading, except to repeat the obvious: The roster is weak, especially on defense and among the bottom-6 forwards. Although there’s been improved coaching, individual breakout seasons (i.e., Draisaitl, Davidson, McDavid, and Yakupov, especially with McDavid, although he is playing well without him) and more stable goaltending with Talbot, there are roster deficiencies.
Lander’s promise under coach Todd Nelson in 2014/15 has not developed under McLellan. Schultz has not flourished either. With Korpikoski on the ice, the team’s shot metrics are bad as last season’s Buffalo, which is to say monumentally bad. As I noted in a previous blog, Nurse is playing over his head. Fayne has struggled, although he seems to have found his way back onto the team recently and playing well. Still, he is a not top-2 defender.
Summary
We haven’t seen a full, healthy line-up all season, including long spells without some of our top-end players, yet we have seen improvement, especially when it comes to team defense. Players appear to be responding to the coaches, and vice versa. For instance, Fayne being put on waivers was a clear message that no contract is safe. Everyone is expected to pull their weight. Todd McLellan expects accountability from his players, and in turn, he has shown he is accountable to them when the team is under-performing.
We’re all hoping that General Manger, Peter Chiarelli, is able to accomplish two things before next season: (1) Create depth in the forward line-up (Kassian looks to be a promising piece) and (2) strengthen the blue-line with two legitimate top-2 defenders. This will likely happen during off-season with names like Travis Hamonic (my favourite), Dustin Byfuglien, Kevin Shattenkirk, and Sami Vatanen being thrown out there.
In the meantime, let’s enjoy the rest of the season without the anxiety of playoff expectations. We’ll have the pleasure of seeing our future top-2 centers, McDavid and Draisaitl, taking on their roles early. It will be a primer for the future.
Thanks for reading and please comment below. What do you think of the season so far?
Every so often, I debate a on-line friend on the issue of “intangibles” in hockey. The latest debate was inspired by Jonathan Willis’s article. I agreed with Willis’s argument that intangibles shouldn’t be used to explain things, or explain away things, that can otherwise be objectively measured. Willis wrote:
“I think intangibles matter. It’s just that all too often they’re used as a catch-all explanation for anything that happens on the ice even when a tangible explanation exists. Too often the false narratives arising from them lead to poor conclusions, like the idea that Nurse’s fight led the Oilers to a great first period. Too often they’ve been used as supports for ineffective players, and cudgels with which to club effective ones, as tools to further the predetermined opinion of a pundit.”
My friend challenged me with the question: “Surely to a certain degree you agree intangibles exist?”
[In response, I wrote:] It’s not that I don’t believe intangibles don’t exist; but they don’t exist in the way Willis argues that some people use them: As explanations for why something happened when there are more observable reasons for why it happened.
True, we cannot see into a player’s head to know what is motivating him at one time or another; his mood; his restfulness; whether he feels confident or not before the game; how the coach’s feedback is impacting him; how scoring or not scoring is affecting his confidence; and so forth. Yet, these “intangibles” are not intangible at all in psychology: We measure them all the time. It’s just that we don’t measure them in hockey.
Come think to think of it, I guess I’m saying intangibles–as they pertain to psychological explanations–don’t exist. Everything about a player or team is measurable; it’s just not practical or possible to measure everything in the context of a game.
[I then recalled some comments McLellan made after the recent Los Angeles game, in which he made psychological observations of his players.]
Observant and motivating coaches try to be sensitive to the psychology of his players. For instance, McLellan noted some slouched shoulders after LA scored 3 goals. He said he saw their play deteriorate after that happened, but then they recovered in the 3rd period. This kind of observation can be checked by looking at shot metrics.
After the 3rd goal, LA kept pressing until they took a penalty. Then the Oilers recovered a little and scored, but then LA took over again, leading to the 4th goal, then kept on rolling to the end of the period. So McLellan’s observation was spot on.
The Oilers came out gangbusters to start the 3rd, although LA was keeping pace. Still, the Oil scored. So McLellan noted that too, but he may have glossed over the rest of the period in his memory. After this initial push and goal, penalties happened. Then Edmonton was flat until the last few minutes, at which point it was too late.
So in this case, even an intangible–a coach’s observation of player confidence and effort–can be indirectly measured by the shot attempt battle.
It’s not always a happy marriage between analytics, intangibles, and psychology, but there are occasions–as above–when a psychological motivation (a supposed intangible) can be observed via the players’s disposition and then indirectly tested by the events on the ice.
The latest trade rumour became reality: This morning, the Edmonton Oilers traded goalie, Ben Scrivens, for Montreal Canadiens’ forward, Zack Kassian. Due to a strong training camp by Anders Nilsson, Scrivens had been demoted to the Oilers’ AHL-affiliate, the Bakersfield Condors. Zack Kassian has a checkered history, which includes on-ice antics, such as his slew foot on Eberle (Nov. 1, 2014), and a recent rehabilitation for alcohol abuse.
Quoting Wikipedia: “On October 4th, 2015, Kassian was involved in an accident on a non practice day, suffering a broken nose and fractured left foot. Kassian was not the driver of the car, however he was under the influence, which resulted in him being suspended without pay and sent to the NHL’s substance abuse program.
On December 15, 2015, after successfully completing the substance abuse program, the NHL announced Kassian’s return from suspension. However, only a few hours after the announcement, the Canadiens put him on waivers.”
That’s the downside of Kassian. On the upside, he is in prime at 24 years of age. Although his overall production isn’t spectacular–35 goals & 31 assists in 198 NHL games–more refined analysis of his even-strength performance suggests he has a strong scoring touch. The following two charts show his production, possession, and passing metrics.
Glossary for Passing Metrics
The HERO chart, which teases apart his production and possession metrics, suggests that on both metrics that he is a solid bottom-6 forward. What stands out, of course, is his goal-scoring rate, which is comparable to top-line forwards. His even-strength career scoring rate (i.e., shooting%) is 13.5%, which is above average even for a top-6 forward.
What about the influence of playing with Henrik and Daniel Sedin while playing in Vancouver? At even-strength, 10 of his 35 goals were scored with the Sedins in which he scored at an unsustainable rate of 22%. Without the Sedins, his career scoring rate is a more plausible 11.7%, which is still high-end for a bottom-6 forward.
However, if we look at his passing metrics (2nd graph), we see that he does not contribute all that much to offense. His Corsi (i.e., shot attempt) Contributions are on the low side, quite low in fact. Even accounting for time-on-ice by using ‘per 60 minute’ breakdowns, he’s not doing a lot to generate offense. But at the same time, if we look to the right of the graph and note his contributions to scoring chances, he looks very good; way above average. He doesn’t do a lot, but when he does, he’s heavy on quality chances.
One way to look at Kassian’s passing and shooting is that he favours quality over quantity. For a bottom-6 forward, I find it hard to complain about that. The more scoring depth for the Oilers, the better. Relative to the Oilers current bottom-6, Kassian is a definite upgrade. Moreover, as to the struggling power-play, I would definitely prefer seeing Kassian than Korpikoski.
Even if this trade doesn’t pan out, I don’t think the Oilers lose in this trade. But if he does become a strong possession and producing depth forward, then he is another valuable piece to the puzzle.
What are your thoughts on Kassian? I was skeptical given his history, but his metrics suggest this could be a good find for Chiarelli.
