Statistics, like any science, are functionally useless unless they can be tested. The 2013-14 Toronto Maple Leafs season may have been the single best test case so far for the first wave of hockey's advanced metrics—and it resulted in a rousing defeat for both the Leafs and those pundits stubbornly unwilling to look beyond the box scores.
The Leafs, whose playoff hopes effectively ended this weekend with a wet fart of a loss to Winnipeg, are not a good team. There is general agreement on that now, but the thing to take away is that it is the same not-good team that began the year 6-1, the same not-good team that as recently as Feb. 1 was 30-21-6 and seemingly sitting pretty for a playoff spot. Records change, outliers regress, but the fundamental underpinnings of the Leafs' performance have been woeful all season long, and if you knew where to look, you could see this collapse coming. And if you refused to even turn your head toward where everyone was telling you to look, you just might have a job in the Toronto media or the Leafs' front office.
Hockey's embrace of advanced statistics is roughly where baseball's was 15 years ago, meaning that what's on hand is incredibly basic—which makes it even more embarrassing for those who dismiss these measures as pointlessly complex nerdery. The most accepted advanced metrics have to do with possession, and really just quantify the "eye test" on things like positioning and puckhandling. Not even the most devoted traditionalist would dismiss the importance of controlling and shooting the puck, so giving the brush-off to metrics that measure just that is baffling.
Despite finishing fifth in the East in last year's lockout-shortened season, the Leafs had red flags everywhere. They finished at or near the bottom in all measures of possession. (I won't go into them here, but spend a little time with this explainer—possession metrics like Corsi and Fenwick pivot on your number of shots compared to your opponents'. If you don't have the puck, you can't shoot the puck. If you don't shoot the puck, you can't score goals. It really is that simple, and has a fantastic correlation with performance.) The Leafs also lead the league in PDO. (Another explainer, but essentially, a high shooting percentage—like a high BABIP in baseball—is unsustainable.)
All of which is to say that the stats—and not just the usual Chicken-Little pessimism of Leafs fans, honed by decades' worth of examples of the sky actually falling—indicated that Toronto got incredibly lucky last season, and unless drastic changes were made, that luck would not repeat and the Leafs would struggle. The changes that Leafs management made did not address the team's underlying issues. They sent possession players like Mikhail Grabovski and Clarke MacArthur packing, and brought in the likes of David Clarkson and Dave Bolland.
The summer was spent with battle lines being drawn. On one side, bloggers like stat evangelist Tyler Dellow and the folks at Pension Plan Puppets, all but screaming that GM Dave Nonis's roster and coach Randy Carlyle's systems were set up for failure. On the other side, the Leafs themselves, contemptuously dismissive of so-called "fancy stats," and their cheerleaders in the Toronto media, like TSN's Leafs mouthpiece Darren Dreger and the Sun's Steve Simmons.
This is not me being overdramatic. This was a season with a built-in black-and-white storyline, making it a referendum on who better grasps hockey—those who know how to interpret the numbers, or those actually running a team. The Leafs threw down the gauntlet early. Here's the GM, dismissing advanced stats as something not yet relevant:
"We're constantly trying to find solid uses for [our analytics budget," Nonis said. "The last six, seven years, we've had a significant dollar amount in our budget for analytics and most of those years we didn't use it. We couldn't find a system or a group we felt we could rely on to help us make reasonable decisions."
"People run with these stats like they're something we should pay attention to and make decisions on, and as of right now, very few of them are worth anything to us," he said at one point during the panel, blaming media and fans for overhyping the analytics currently available.
The Leafs came out of the gate strong, starting the season 6-1. Those with numbers on their side knew better than to trust small sample sizes, especially when the underlying metrics still predicted doom and gloom. Those who had defined themselves in opposition to what they perceived as threats to their enjoyment of the game had no such restraint. Simmons was far from the only pundit to gleefully break out the "mom's basement" clichés, but he was and is the loudest and the worst. A professional writer actually tweeted this:
Good thing the Leafs don't play in the CHL. The CORSI hockey league. They're doing just fine in NHL, though.
— steve simmons (@simmonssteve) October 30, 2013
The mindset of Simmons, and those determined not to expand their understanding of the sport, is best summed up by an email exchange during a Leafs swoon in December. SkinnyFish of Pension Plan Puppets sent Simmons a link to his October column, gloatingly headlined "Where are the critics of Nonis' off-season moves now?" Simmons responded, simply, "and what was leafs record when that was written?"
This worked almost as a caricature of sports punditry, with analysis starting and stopping at "Team X is good because they have a good record," and it played out in an entirely predictable fashion. Even as Toronto went on another run in January, the underlying numbers were still heralding disaster, with possession metrics indicating that any success could be chalked up to dumb luck. And what do you know? The Leafs' sky-high shooting percentage, which had carried them to the playoffs the previous season, was still unrealistically inflated. Finally, the bubble burst. Since Feb. 27, when the Leafs were still third in their division, Toronto has failed to score more than three goals in a game, despite not attempting measurably fewer shots. They were continuing to get outshot, but with their own shots finding the net at a league-average rate, they were exposed as a terrible hockey team.
In what can only be considered trolling, the Sun ran a story in late February claiming the "stats" indicated the Leafs were poised for a late playoff run. Those stats? The Leafs' post-Feb. 1 records from seasons dating back to 2006.
In what can only be considered ignorance, Toronto VP of hockey operations Dave Poulin had this to say in the midst of a losing streak:
Poulin "our shot differential over the last 8 games is better. So the analytics talk doesn't make a lot of sense"
— Hope_Smoke (@Hope_Smoke) March 31, 2014
Except, as explained here, that required a total misunderstanding of score effects, another basic concept. Teams that have a lead will turtle. The Leafs' shot differential was better only because they were trailing.
You know how the story ends. As the season progressed, the Leafs began playing more and more like the team that the advanced stats said they were. PDO came down to earth, while possession numbers remained in the cellar, with all of this reflected on the scoreboard. The cheerleaders and math-deniers are going down with the ship, never once acknowledging that there were vocal, informed voices who promised this exact outcome. These Maple Leafs were set up to regress, and we got to watch it happen in real-time.
This doesn't mean the Leafs' collapse had to happen. They could have snatched a handful of bounces along the way and made it to the postseason. But statistics aren't strictly predictive, and don't claim to be. Nothing can tell you which team will win or lose in any given game or season. They can only tell you how something happened, why it happened, and what's likely to happen next. The Leafs won because of unbelievable luck. They lost because they're bad at NHL-level hockey. There's no real excuse for anyone running an actual hockey team not to know this.
All of this is a huge win for hockey's burgeoning stat community, which doesn't want numbers to replace game analysis, only to augment it. But it can't be emphasized enough how basic possession stats are. While attempts at more comprehensive statistical ratings are in the offing (and judging from the pace of baseball sabermetrics, it will take a while to find true signals in the noise), there should be nothing controversial about stating that teams that control the puck are going to succeed. This isn't like waving away PECOTA projections; it's like those old men who dismissed the idea of measuring how often batters got on base as fancy math. It took a long time for even a simple and easily understandable hockey metric to win a resounding public affirmation, but, as we've seen in baseball, there's no going back.