Baseball is a strange market, forever correcting itself in strange ways. For a long time, the most obviously central ability for a hitter—getting on base—was overlooked, until suddenly it wasn't, and it occurred to teams that there were probably all sorts of other valuable skills they were neglecting. In the years since Michael Lewis wrote about how Billy Beane's Oakland A's got an edge in an unfair game, all sorts of inefficiencies in the market have been noted and accounted for. The most glaring, though, is still right there, waiting to be taken advantage of. It doesn't have to do with who the players are, but rather with who picks them.
Most observers take it as given that the market for MLB front office employees—general managers, scouting directors, analysts, and so on—reflects a rational equilibrium for an industry in which aspiring workers vastly outnumber available jobs. In my senior thesis, "If You Build It: Rethinking the Market for Major League Baseball Front Office Personnel," I challenge this assumption and offer literally game-changing estimates for the potential values of the people who put the teams together.
The short version is that the value of an elite general manager to his or her team is on par with (if not higher than) that of an MVP-caliber player. Despite this, white-collar team employees make significantly less than top players. That means that a dollar spent on front office talent will go much further than a dollar spent on a free agent—and that the market for non-player employees as it is currently conceived is irrational.
In recognition of the fact that my full paper is longer and more detailed than most people would care to read, here's an abridged summary of what I did, what I found, and what it means.
Background
Major League Baseball teams are willing to spend enormous sums of money to win more games. For the 2013 season, I calculate the cost of a free agent win to have been just over $7 million—i.e., an average general manager who wanted to make his team one win better by signing established players should have expected to pay out $7 million in salary. (You can find my explanation of how that relates to teams' individual demands for wins here.) The price of a win passed $6 million in 2007 and has vacillated within that range for several years; given the influx of new television revenue, we may end up observing a substantial spike in the market in 2014.
Yet despite the high values they place on making themselves marginally better, MLB teams are significantly stingier when it comes to hiring the people who put the rosters together. To my knowledge, the highest-paid non-uniformed MLB team employee whose job directly pertains to baseball is Chicago Cubs President of Baseball Operations Theo Epstein, who averages $3.7 million in annual salary under his current contract—or approximately the cost of half of a win on the free agent market. This implies that there is not a single MLB front office employee who is worth more than half of a win to his or her team per year. And that's to say nothing of the lower-level scouts, analysts, and player developers who often make fractions of what they could get in other industries and whose salaries suggest that each contributes less than one one-hundredth of a win's worth of value to his or her team.
The typical explanation for why non-player employees' salaries lag so far behind those of players is that the supply of job candidates is both higher than the demand for employees and quite inelastic—i.e., those who dream of working in baseball aren't dissuaded by low wages. "Teams always have the advantage when hiring," player agent Joshua Kusnick writes, "because so many people are willing to work for next to nothing just to get their foot in the door." (I offer a more thorough explanation of the MLB labor market and the assumptions on which it is based here.)
Though this conception of the market has intuitive appeal, it is based on several key assumptions that seem highly questionable. The most important assumption for rationalizing this supply-driven model of the MLB labor market is that it doesn't much matter whom a team hires for a given position from the pool of serious candidates. The best scout or statistical analyst may be willing to work for a fraction of the value he or she provides to his or her team, but in a competitive market he or she wouldn't need to. If one team is paying 10 cents on the dollar for its best front office employees, other teams should be willing to hire elite baseball operations talent for 11 cents on the dollar; eventually, the market would correct itself. (Think about what the Moneyball Oakland Athletics teams did in investing in undervalued players with good plate discipline, and how the market came around so that the league values on-base percentage more accurately today.)
If there exists significant heterogeneity in MLB front office employees' skills and values to their teams, then the current manifestation of the labor market is irrational. So I set out to test just that.
What I Did
In any industry, the ability to accurately evaluate individual employees' skills and values is hampered by a lack of causal data. Researchers can look at how a teacher's students improved on standardized tests after being in their classrooms, for example, but they cannot conclusively determine causality without observing the instructor's specific actions (or lack thereof) in dealing with his or her pupils. Baseball is the same way, largely because no team wants its internal processes and proprietary information disseminated across the league.
Yet in looking at MLB general managers, there is one category of actions that can be isolated, quantified, and attributed to those responsible for them: player transactions. Every free agent signing or trade is a decision that the GM (or someone whom he authorized to act in his place) made based on information he (or people he chose to hire and listen to) obtained, leading to a quantifiable outcome. I use these as the bases for challenging the notion of value homogeneity among front office personnel by estimating the variation in player-investing ability at the GM level.
After well over 100 hours of research, data entry, and editing, I compiled a database of every free agent signing and trade from November 1995 (i.e., since the end of the last strike-affected season) through September 2013 that met my qualifications for usability. I matched each player involved in each transaction with his salary and the value of his production (based on FanGraphs' typical wins above replacement model for hitters and its RA9-WAR model for pitchers and my estimates of the price of a win) for each season between when the deal occurred and when he next reached free agency. I calculated each team's return on investment from each transaction in terms of the production it received and/or the salary it shed relative to the salary it took on and/or the player(s) it traded away.
I then ran random effects regressions on each set of data with GMs as the independent variables (including organization-specific controls for free agent signings but not for trades) used to predict transaction ROI. Rather than treating each general manager as an isolated unit, random effects modeling treats the whole population of GMs as related and estimates a quasi-normal distribution for the population at large. The variance statistics of this distribution are the core of this analysis.
(See the full paper for more details on the nature of the data, my qualifications for which transactions were included, my calculations of transaction ROI, and why I built the random effects models as I did.)
What I Found
The random effects model for free agent signings estimates that a single standard deviation of player-investing ability corresponds to a 40-percent difference in a team's return on free agent investments. I calculate that the average team spent $47 million on players working under contracts they signed as free agents or extensions signed after landing free agent contracts in 2013. Forty percent of $47 million is just under $19 million in surplus value, so at $7 million per win, an additional standard deviation of free agent-investing ability at the GM level leads to the equivalent of 2.7 extra wins per team per year to the average team. (Of course, a new GM cannot completely overhaul his or her roster immediately. These numbers can be interpreted as either the annual added value a GM would provide once he or she has been in place long enough to fully shape the roster, or as the total non-discounted long-term value added from player transactions per year on the job.)
For trades, the random effects model estimates that the variation in trade-investing ability is proportionately smaller, with one standard deviation of improvement at the GM level leading to an additional 31 percent of return on investment. Yet because making a trade involves investing both salary and players, I calculate that the average team invested a combined $111 million in post-trade salaries and player production for the 2013 season. That pegs a single standard deviation of trade-investing ability as worth $34 million per year to the average team—the equivalent of acquiring an extra 4.9 wins.
When a team loses enough confidence in its GM that it decides to shake things up, a one-standard-deviation improvement in roster-building ability is probably a conservative estimate for the upgrade the team seeks to achieve. If it manifested itself in both free agent signings and trades (there was no significant correlation between individual GMs' estimated abilities in the two categories of transactions, but that can be somewhat explained by the noisiness of the individual estimated effects), such an improvement would correspond to an additional $53 million in returns on the average team's transaction investments—or, 7.6 annual added wins. More broadly, the range in annual value among even the serious candidates for a given vacant GM position could plausibly exceed $100 million.
The graph below gives a sense of what this sort of value means for a team. Given the cost of a win and the amount of salary and player production teams invested in free agent signings and trades in 2013, I looked at how much different levels of improvement in player-investing ability at the GM level, as compared to the league average, would be worth each year to the average team. Adding a GM who was one standard deviation above the mean at both trading and free agency would, essentially, be like adding a strong MVP candidate. Adding a GM that was exceptional in one of these fields—two standard deviations over the mean—would be an enormous boost as well, although trading ability is more valuable than the free agency role.
It should be noted that these models treat everyone from the assistant general manager to the advance scouting intern as endogenous to the GM. Part of a GM's value to his or her team is knowing whom to hire and listen to and how to run the office, but these numbers are inherent overestimates insofar as they see a GM's ultimate responsibility for his or her employees' actions as the ability to work alone. On the other hand, signing free agents and making trades are but two facets of a GM's job; if there is comparable variation in GM skill at other aspects of running a team, my numbers could be significant underestimates.
My models are designed to estimate the variation in player-investment skill among the entire population of GMs rather than evaluate specific GMs, and for a variety of reasons I don't put much faith in any of the individual results. Yet there is a pretty clear correlation between how an individual GM ranks in my models relative to his peers and his reputation in and around the game, which suggests that the models are onto something. (To avoid the individual results being taken out of context I do not list them here, but they can be found in the appendices at the end of my paper.)
What This Means
Even several months after getting my first round of results, that a single standard deviation of player-investing ability at the GM level could be worth over $50 million a year blows my mind. But even if you want to shade my variance statistics down to a fraction of the numbers I present, they provide what I believe to be a sharp rebuke to the notion that there exists no significant variation in value among MLB front office personnel. And with that, the prevailing conception of the league non-player labor market falls apart.
My best estimate for the value of a single standard deviation of player-investing ability at the GM level is $53 million a year. The market says the maximum value of the head of baseball operations department is less than $4 million a year. Unless I'm way, way off, there's a massive inefficiency in the market for elite general managers waiting to be taken advantage of.
The rejection of the market status quo need not stop at the top level. If a special GM could be worth nine figures compared to whomever his or her team would hire instead, surely an assistant GM or scouting director could be worth eight. An elite scout or statistical analyst may provide millions in value to his or her team each year. And even if the best intern in the league were worth but a fraction of a win above replacement over the course of the apprenticeship, a team could give him or her a salary in the hundreds of thousands and still realize a favorable return on its investment.
The trouble is determining who among the massive group of current and aspiring MLB team employees is worth what, and at this point I cannot offer a concrete quantitative solution. But is that a true obstacle? A team shouldn't need a major study to get a sense of how valuable its own employees are; many of the best-reputed lower-level employees are known as such throughout the league. You don't need to consult WAR and wRC+ to know that Mike Trout is a special player, and you don't need a complex model to know that Billy Beane is a good GM.
A team owner's first reaction to these results might be satisfaction: If he or she is paying the GM far less than he's worth, that's a great investment. But in a rational and competitive market, such an inefficiency shouldn't last. If the range in value between the best and worst GMs in baseball is on par with—or possibly greater than—that between the best and worst players in baseball, smart teams will start bidding up the prices for the best-reputed executives in the league, so long as their wages are still substantially lower than their presumed values. The same goes for lower-level employees. Unless a team is certain that the market will never change (an assumption that carries major collusive implications), its best move given this information would be to be the first to offer marginally above-market salaries to elite front office personnel and to take advantage of this massive market inefficiency before the rest of the league catches on.
The typical explanation for why MLB front office personnel are paid comparatively poorly boils down to their supposed replaceability. But in light of these findings, a team declining to seriously and actively compete for the best baseball operations talent would be like it saying there's no reason to sign Robinson Cano because there's another second baseman in Triple-A who would be happy to take the job. Even among the massive oversupply of potential candidates for front office jobs, special individuals are worth larger investments.
In the current MLB non-player labor market, a dollar spent on front office talent will go significantly further than a dollar spent directly on the field. Ironically, this means that the so-called "next Moneyball" is in fact the undervaluation of those who are looking for the "next Moneyball." The first to acknowledge, act on, and take advantage of this massive market inefficiency will hit one out of the park.
Lewie Pollis is a recent graduate of Brown University. He has contributed to several publications, including ESPN Insider and Baseball Prospectus. He interned for an MLB team in 2013 and will begin another such internship later this month.