From: Aaron Schatz
To: Josh Levin
After 10 years of writing stat analysis on the NFL, I think I've written about fourth-and-short decisions roughly eleventy-billion times. For 10 years, I've been writing that NFL coaches need to go for it more often on fourth down. For most of those 10 years, there have been a lot of other people also writing that NFL coaches need to go for it more often on fourth down. And Sunday in Atlanta, we all finally got to see the platonic ideal of mistaken, pusillanimous fourth-down decision making.
Up 28-27 and faced with fourth-and-1 on the Atlanta 45 with 1:44 left in the game, armed with two highly paid running backs and possibly the greatest short-yardage running quarterback in NFL history, and having gashed the Falcons that day for 5.7 yards per carry, Carolina Panthers head coach Ron Rivera punted. And there was a great disturbance in the Force, as if millions of stat analysts suddenly cried out in terror and were suddenly silenced.
We've been through a lot of controversial fourth-down decisions over the last few years. We had Bill Belichick's infamous fourth-and-2 against Indianapolis. We had Mike Smith's decision to go for it on fourth-and-1 in overtime against New Orleans. We had Jim Schwartz's decision to go for it on fourth-and-1 last week that ended up not actually being a decision to go for it.
Every time we hit one of these controversial fourth downs, conventional wisdom among the football commentariat will call for a punt. And pretty much every time, one of my compatriots like Brian Burke will run the numbers and find out that going for it was the right call. However, in a lot of these situations, it is only the "right call" by a small percentage of win probability. When Mike Smith had to make a decision on fourth-and-1 in that Falcons-Saints overtime, for example, the win-probability difference between punting and going for it was just five percentage points.
And so for the last couple years, I've been arguing that we—the football stat people—need to stop trying to make our argument with situations in which the advantage of going for it is small. We need to make our argument with obvious situations in which the advantage of going for it is huge. The typical old-fogey argument against all the research about fourth-down probabilities is that those stats don't take into account the actual strengths and weaknesses of the teams on the field. The research is based on league-wide trends and numbers. What if your team has a terrible offensive line? What if you have a great defense that is likely to stop the other team after you punt? In my opinion, these are valid arguments to make when the win-probability difference is small. Sometimes, the old fogeys are right: No, you don't go for it on fourth-and-1 from your own 30 if your running back is Jahvid Best and he's running behind the Miami offensive line (dead last in 2011, converting only 46 percent of runs in short yardage).
But here we have a situation in which a) the difference in win probability between punting and not punting is massive and b) the personnel involved screams "go for it." This analysis by ESPN Stats & Information suggests that the Panthers' chances of winning the game dropped by 26.1 percentage points when they decided to punt the ball, going from 83.5 to 57.4 percent. That's nuts.
But wait, it gets worse. As noted above, the Panthers have Cam Newton, who holds the NFL record for rushing touchdowns by a quarterback. How hard is it for Cam Newton to get fourth-and-1? Heck, he had just converted third-and-2 a few seconds before, although he ended up un-converting when he fumbled the ball. The chances of him converting the fourth down were a lot higher than the chances he would fumble again. And if you wanted to use Newton as a decoy? Well, in a league where more and more general managers are refusing to pay big money to running backs, the Panthers have not one but two backs signed to huge contract extensions worth almost $50 million combined. Is the blocking a problem? Well, center Ryan Kalil has made three straight Pro Bowls and was second-team All-Pro last year. Football Outsiders numbers had Carolina converting 71 percent of runs in short yardage in 2011, fourth in the NFL; that number includes runs with two yards to go, so their probability of converting with just one yard to go will be higher.
Oh, and what happens if you punt the ball? Well, Carolina would be handing the ball back to the current leading candidate for 2012 NFL MVP, Matt Ryan, and trying to stop him with a defense that was the worst in the league in 2011 and now starts a rookie fifth-round pick (Josh Norman) at cornerback and a free safety (Haruki Nakamura) who had never started a game until this season.
What in the name of Kealoha Pilares was Ron Rivera thinking here?
The old school will say that Rivera made the right decision because rookie Brad Nortman launched a spectacular punt. The Panthers downed it on the one-yard line; the only way the punt would have been better was if the Atlanta returner had fumbled it away. Because of that great punt, Rivera's decision didn't actually end up hurting Carolina's win probability—at least according to those league-wide statistical models. (Remember, personnel matters: Carolina was still stuck isolating Nakamura in deep coverage against Roddy White, one of the five best receivers in the league.)
But you don't judge decisions based on results; you judge them based on process. When Rivera made the decision, he didn't have a magical crystal ball that told him Brad Nortman was going to land the punt on the one-yard line. Nortman easily could have put the ball in the end zone for a touchback—just like he did when punting to New Orleans in a similar situation near the end of Carolina's Week 2 win.
And not to put too fine a point on it, but Atlanta won anyway. The decision to punt led to pretty much the best possible result you could hope for, and the Panthers still lost. If you want to judge based on results instead of process, you have to be honest and admit that it didn't work. The process sucked, and the results sucked.
This is it. This is the decision we need to hold up when we're screaming about fourth downs; not Mike Smith's fourth-and-1, not Bill Belichick's fourth-and-2. This play. Cam Newton is now our poster child for bad fourth-down decision-making. General managers need to bring up this play in every interview with every head coach prospect from now on. If the job applicant would not trust his 6-foot-5, 245-pound quarterback to ice the game with an 80 percent probability of succeeding, he shouldn't be a head coach in the National Football League.