What Do NFL Scouts Think They Know About Quarterbacks?

Illustration for article titled What Do NFL Scouts Think They Know About Quarterbacks?

We're doing a season-long NFL roundtable with our friends at Slate. Check back here each week as a rotating cast of football watchers discusses the weekend's key plays, coaching decisions, and traumatic brain injuries.


« Previous entry | Next entry »

From: Luke O'Brien
To: Jeremy Stahl, Robert Weintraub

Before I address your Andrew Luck question, Jeremy, allow me to second your irrational distaste for Norv Turner. The man is a hack in a headset. Although widely considered one of the best play-callers in the business, he's a professional mediocrity. San Diegans anxiously await his firing. I watched Turner for seven seasons in Washington as he grimaced along the sidelines in his windbreaker and his television hair. Did you know the Redskins have won only two playoff games since 1993? Shockingly, Turner was responsible for one of them.

What bothers me most about Turner is that I can't find anything specific in him to really detest. Norv seems like the milquetoast type you might encounter sipping nutmegged cocoa in a ski lodge. I almost feel bad when his "explosive" pro-set offense—modeled on the schemes of Don Coryell, the San Diego coach during the Dan Fouts era—explodes all over his face, as it does several times a season. Regrettably, there's no better way to evaluate coaches beyond a win-loss record.

Evaluating San Diego quarterback Philip Rivers is a different matter. Rivers was weighed and measured in every dimension long before he took a pro snap. He was hailed as a potential star largely because of his accuracy in college at North Carolina State, where he broke the NCAA record for consecutive games started. Despite being overshadowed in the 2004 draft by No. 1 pick Eli Manning, Rivers has thrived individually in the NFL, making it to three Pro Bowls.


To the naked eye and the distant observer, Andrew Luck seems to have everything necessary for similar, if not greater, success in the NFL: height, arm strength, touch, pocket awareness, accuracy, scrambling ability, intelligence, leadership, etc., blah, ad infinitum, blah. How many tools is that? Eight? The whole drawer? That wasn't enough for Luck's former Stanford coach, Jim Harbaugh, who coined a new tool for his quarterback when Luck was just a freshman: memory talent. Luck, apparently, never makes the same mistake twice. That's one mistake away from God, if I'm counting correctly. All these hosannahs leave me no doubt that Luck must also ooze that ineffable quality distilled from pixie dust and rainbow truffles that we refer to in these parts as "poise."

Poise be damned, though. Success at the highest level is tough to predict, even with all the fancy calipers in use today. Take the Wonderlic test. Jim Kelly and Dan Marino both scored 15 on it. Matt Leinart registered a 35 and routinely proves himself an idiot. At this point, we should all be deathly sick of hearing about the Wonderlic and how brilliant Harvard men ace it, only to be discriminated against in the NFL—too smart for their own good, those fellows. Problems with authority, them.


This is laughable. Being good at taking tests is not evidence of an untamed spirit. The Harvard of today is a place for rebels the way the Metropolitan Club is a place for dirty hipsters. And if Ryan Fitzpatrick really were smarter than Einstein, you'd think he'd be able to hold the Buffalo offense in his head without having to refer so often to a flip-top wristband that looks like it was purloined from the Predator set.

More important than Wonderlics for predicting future success in the NFL, it turns out, are the attributes Rivers possessed going into the draft. Namely, he'd amassed a ton of experience and had an excellent completion percentage. I keep referring to Football Outsiders because they've got the best publicly available nerd-analysis out there. (Just accept the love, Rob.) Football Outsiders introduced the world to the Lewin Career Forecast that you reference, Jeremy. And the Lewin Career Forecast v2.0 tells me that the key variable in predicting a college quarterback's performance in the NFL is probably the simplest one: career college games started. Without being more of a nerd, I can't be sure that the statistic is a measure of talent or a measure of playing enough games to be effectively scouted or both. (Someone help me out, please.)


There's a reason, of course, the Lewin Career Forecast is in version 2.0: It's not perfect. Even advanced nerd analysis fails to account for Matt Leinart, who made plenty of starts for USC and had a fine completion percentage but nevertheless sucks professional bird. Chad Henne, Brian Brohm, and Pat White have similar profiles. And let no scout speak of the 2007 draft. JaMarcus Russell and Brady Quinn were the first round picks. Kevin Kolb, John Beck, and Drew Stanton were taken in the second round. That's not a bust or two. That's a busted class.

As the game evolves, so must our predictive abilities. The quants love Colt McCoy, who played for Texas for half a decade, and hate Cam Newton, who only had 14 starts for Auburn but has a decent chance to break all the rookie passing records in the NFL this season. The successful Joe Flacco didn't start many college games. Same goes for Aaron Rodgers, who wasn't recruited out of high school and played at a junior college before Cal stumbled upon him. Rodgers looks like a superhero in the NFL. Maybe it has to do with him riding pine in Green Bay behind a gnarled lecher who sniped at him, an experience we can safely call an intangible. Or maybe Rodgers is just really good. As a junior in college, he completed 23 consecutive passes against the top-ranked team in the country. In 2005, he was drafted in the first round.


All of which suggests to me that the key variable is really the number of passes thrown, not the number of college games started. I spoke to Gil Brandt, the chief scout for the Cowboys from 1960-1989 and the NFL's "personnel guru" for the last eight years. As Brandt explained it to me, the most significant transformation in youth football over the past decade or so has been the rise of seven-on-seven football, an elite travel system similar to AAU basketball. Young quarterbacks spend much more time throwing the ball today than they did even ten years ago.

With increasingly polished passers entering college to be evaluated by increasingly astute scouts, physical ability is less of a variable in gauging NFL success. The scouts have made a science of knowing which on-field attributes they want in quarterbacks. I listed most of them above with Andrew Luck. They're easy enough to spot. "There's 10 percent really great players," Brandt says. "Housewives can tell you who those are. There's 10 percent really bad players, and housewives can tell you who those are, too. Then there's 80 percent that look a lot alike."


Finding the gems in that 80 percent is the trick. And to do it, scouts look more than ever to how players perform off the field, according to Brandt. He's talking about all those mushy values like work ethic, leadership, character, and so on that tend to get mocked by the quants because they're hard to turn into numbers. But the "breakthough" in scouting over the past few years, Brandt says, is the intensive psychological test administered to players that makes the Wonderlic look like tic-tac-toe.

The Cowboys have been psych testing quarterbacks since 1960, giving the team a wealth of data that is paying dividends in the modern era. Brandt read me a snippet from JaMarcus Russell's test results: "Slow mental quickness and problems in focusing, indicating a high probability of learning difficulties and execution problems on the field." It's not as if Russell's propensity to implode wasn't documented. The Raiders just didn't care.


Dreamy Andrew Luck's psychometrics are still a mystery, as far as I know. If he becomes the NFL star he's supposed to, a great many people will want to take credit for predicting it. And if he flames out, well, we can blame it on the universe or alcohol or sleep apnea. Or even bad scouting.