Just ahead of playoffs season, ESPN today announced a new stat for the NBA called "Real Plus-Minus." It's envisioned as furthering the idea of traditional plus-minus, except, that's sort of already been furthered. So what's going on here?
The short answer is we have no idea what's here. The long answer, is we know exactly what a large but undefined portion of this is, since it already exists, but don't know what's new here. The sell is that Real Plus-Minus (RPM) tells you how much better a team played on offense and defense when a given player was on the floor, and how much that improvement was that individual player's doing. From the introductory post:
[The] metric isolates the unique plus-minus impact of each NBA player by adjusting for the effects of each teammate, opposing player and coach. ... The RPM model sifts through more than 230,000 possessions each NBA season to tease apart the "real" plus-minus effects attributable to each player, employing techniques similar to those used by scientific researchers when they need to model the effects of numerous variables at the same time.
One part infomercial, two parts bullshit; it's an easy elision to miss as you're racing past to see what the stat is for, but what it does isn't easy to glean from the post itself, or the presentation of the RPM stats on ESPN. For that, you really need to understand the stuff it's built on.
There is some novel work going on with RPM, but in the introduction, Steve Ilardi, formerly of the Phoenix Suns and the Kansas men's basketball team, references the new work by Jeremias Engelmann in the context of its immediate predecessors, Adjusted Plus-Minus and Regularized Adjusted Plus-Minus.
Here's Wayne Winston explaining what Advanced Plus-Minus does:
It reflects the impact of each player on his team's scoring margin after controlling for the strength of every teammate and every opponent during each minute he's on the court.
Adjusted +/- ratings indicate how many additional points are contributed to a team's scoring margin by a given player in comparison to the league-average player whose adjusted +/- value is zero over the span of a typical game. It is assumpted that in a typical game a team has 100 offensive and 100 defensive possessions. For example, if a +6.5 "adjusted +/-" player is on the floor with 4 average teammates, his team will average about 6.5 points better per 100 possessions than 5 average players would.
This makes sense, right? Points per 100 above or below replacement level. There are a lot of issues with APM, though. The numbers are high-variance year-over-year, roles and coaching systems change, and rotations feature some players paired together very often, and others very infrequently. This is one reason that a lot of people just gave up on the individual plus-minus stats, and instead went with lineup stats, which you can find on NBA.com, among other places. Knowing concretely that the Wolves' starters are punching weight with the Warriors' top unit, and actually better than the Pacers' and Blazers' starters, requires far fewer caveats than trying to dig out individual value.
The work on that front goes on, though. Here's Joe Still on RAPM:
In "Regularized Adjusted Plus-Minus" (RAPM), the goal is to provide more accurate results by employing a special technique called "ridge regression" (a.k.a. regularization). It significantly reduces standard errors in Adjusted Plus-Minus (APM).
Conventional adjusted plus-minus is shown to do a poor job of predicting the outcome of future games, particularly when fit on less than one season of data. Adding regularization greatly improves accuracy, and some player ratings change dramatically. The enhancement with the RAPM is a Bayesian technique in which the data is combined with a priori beliefs regarding reasonable ranges for the parameters in order to produce more accurate models.
All of this, by mission more than method, is what RPM is supposed to be doing. So what is new here? ESPN again:
RPM reflects enhancements to RAPM by Engelmann, among them the use of Bayesian priors, aging curves, score of the game and extensive out-of-sample testing to improve RPM's predictive accuracy.
Which sounds an awful lot like slapping a binding on someone else's science fair project and selling it as a textbook. RAPM is already used by hardcore NBA heads, so it's more than a little odd to see ESPN roll this out without explaining what it's doing differently. We will presumably hear a little more about what's gone into RPM at some point, maybe at next year's Sloan, maybe as the playoffs ramp up. But as NBA analysis gets more observational and, therefore, contextual, the need for this sort of reverse-engineered testing should fall away, at least a little bit. For now, though, this isn't a bad way at all to judge how important (or harmful) a player is to his team.
The example that ESPN used was of Taj Gibson and Jamal Crawford, two of the best bench players in the league. They are fine examples, with Gibson having much more impact defensively than Crawford does. Once you adjust for DeAndre, Barnes, and company handling the defense, Jamal doesn't look as great. RPM is no different conceptually than the other stats that try to do this; it just claims to do it better.
This, the thinking goes, should show you something like how valuable a player is to a team. So Reggie Jackson, the Thunder's backup point guard who often plays with the starters, can be contextualized as less important than them, but better or worse than others in a similar role. That relationship does, though, bump against an issue with this sort of analysis.
Just at a glance, the aforementioned player pairing problems—multicollinearity issues, to be specific—do seem to affect the ratings. Take Nick Collison as an example. He's sixth overall in RPM, which could lead you to believe that he is a secret cog in the Thunder machine. And yes, Nick Collison is great for what he is, but sliding in ahead of Steph Curry and Tim Duncan does raise a few questions. Just hazarding a guess, could Perk and his literally-worst-in-the-NBA -6.19 Offensive RPM (-3.19 overall) have anything to do with that? This type of analysis expressly tries to isolate a player from his context, but in extreme cases like this, things can get messy. One surmises that Collison often replacing the single most disastrous offensive player in the known cosmos has some kind of effect of his rating, which is the sort of thing that would show up the limitations of this sort of analysis. Then again, J Crossover is currently backing up Willie Green.
When ESPN Stats & Info rolls out a stat like this, it will typically be not when it's most useful to you, the fan, or other writers, but when it is best for its coverage. And frankly, the sixth men for the Bulls and Thunder having good seasons doesn't make for the most compelling TV.
Late in the season, though, when player value can turn into Most Valuable segments, that's a little entertaining. A lot like the Football Power Index and Championship Drives (and all the more interesting stats they comprise), a stat like RPM is most useful in the context of a debate. Awards season isn't a bad time for something like that.
Hopefully, RPM won't go the way of Total QBR, which is functionally a very good stat, but opaque in frustrating ways for outsiders looking to determine if it's actually worth using, and with an all-timer of an overreach on the sell, which was that it was the only stat you needed to look at to judge a quarterback. It wasn't, obviously, though it comprised many that were important. But the allure of having all football advanced metrics melted down into one bullet was too much. And now we do it again, with one stat to settle all of the He makes his team better! pillow fights on First Take.