More on Quarterbacks

 

A few more thoughts on quarterbacks:

 

There are two separate issues with respect to quarterbacks. The first is whether, historically, NFL teams have done a good job of predicting which college quarterbacks will succeed in the pros. Dave Berri and Rob Simmons’ paper in the Journal of Productivity Analysis (that I relied on in the essay “Most Likely to Succeed” in my new book “What The Dog Saw”) proves pretty convincingly, I think, that the answer is no. One of the best parts of that paper is how Berri and Simmons demonstrate how much NFL teams tend to irrationally over-weight “combine” variables like speed, height and Wonderlic score.

There’s a second wonderful paper on this general subject by Cade Massey and Richard Thaler—Thaler being, of course, one of the leading lights in behavioral economics—called “The Loser’s Curse.” The argument of the Thaler-Massey paper goes something like this (and I encourage anyone who is interested in sports to read the whole thing, because I can’t do it justice here). By looking at the trades that NFL teams make, we can estimate the “market value” of a draft pick. And what we find is that teams place a very high value on high first round picks. The first pick in the draft, they write, has historically been valued as much as “the 10^th and 11^th picks combined, and as much as the sum of the last four picks in the first round.” Then Thaler and Massey calculate the true value of draft picks, using what they call “surplus value.” The key here is that all NFL teams operate under a strict salary cap. So a player’s real worth to a team is the extent to which his performance exceeds the average performance of someone making his salary. And what do they find? That market value and surplus value are radically out of sync: that teams irrationally over-weight the importance of high first round picks. In fact, according to their analysis, the most useful draft picks are in the second round, not the first: that’s where surplus values tend to be highest. Hence the title of the paper: “The Loser’s Curse.” The NFL rewards its weakest teams by giving them the highest draft picks—but those picks are actually not the most valuable picks in the draft.

    It is important to note here that we are talking about relative value. Personnel decisions in the NFL have clear opportunity costs: if you pay $15 million for a quarterback who only gives you $10 million of value, then you hve $5 million less to pay for a good linebacker. As they write: “To be clear, the player taken with the first pick does have the highest expected performance . . . but he also has the highest salary, and in terms of performance per dollar, is less valuable than players taken in the second round.”

What Massey and Thaler are saying, in essence, is that NFL general managers are not rational decision-makers. That’s why I think its so useful in this particular discussion. Those who believe that draft position is a good predictor of quarterback performance are essentially voting for the good judgment of the people who make draft decisions. And what Berri and Simmons in particular—and Massey and Thaler in general—remind us is that that kind of blind faith in the likes of Matt Millen and Al Davis simply isn’t justified. And, by the way, why should that fallibility come as a surprise? We’ve known for a long time that it is not easy to making decisions under conditions of extreme uncertainty. Here is Massey and Thaler from their conclusion:

 

Numerous studies find, for example, that physicians, among the most educated professionals in our society, make diagnoses that display overconfidence and violate Bayes’ rule. The point, of course, is that physicians are experts at medicine, not necessarily probabilistic reasoning. And it should not be surprising that when faced with difficult problems, such as inferring the probability that a patient has cancer from a given test, physicians will be prone to the same types of errors that subjects display in the laboratory. Such findings reveal only that physicians are human.

Our modest claim in this paper is that the owners and managers of National Football League teams are also human, and that market forces have not been strong enough to overcome these human failings. The task of picking players, as we have described here, is an extremely difficult one . . . Teams must first make predictions about the future performance of (frequently) immature young men. Then they must make judgments about their own abilities: how much confidence should the team have in its forecasting skills? As we detailed in section 2, human nature conspires to make it extremely difficult to avoid overconfidence in this task.

   

   This brings up the second question. Is it possible to ever accurately predict which college quarterbacks will succeed in the pros? Both the Thaler analysis and the Berri analysis hold out the real possibility that teams can be a lot smarter than they currently are. The New England Patriots clearly have taken some of Thaler’s lessons to heart, for example. There has also been a real effort by the folks over at Pro Football Outsiders to come up with a more useful algorithm for making quarterback selections. David Lewin’s “career forecast” zeroes in on career college starts and career college completion percentage as the best predictors of professional performance. I took the position in my essay “Most Likely to Succeed” that I didn’t think that quarterbacking (as opposed to other positions on the field) was predictable in this sense—that there is so much noise in the data, and so much variability between the college and professional games—that attempts at rationalizing draft day decisions have real limits. I’m still of that inclination. I’m willing to be convinced, though. I’d love to see more statistically-minded people weigh in on the Lewin analysis, and I’d also like to have a better handle over how the recent innovations in college offenses—particularly the use of ever more aggressive spread formations—affects the accuracy of that algorithm.

 

Comments

[Cliff Cline]

You can't cover all angles in any short piece - I get it.

A weakness of the assertions that I have read here is the apparent disregard for the economic engine which requires inputs as well.

For instance, does a quarterback taken first who is from the NFL city he is drafted into have more value to the franchise than a statistically identical one from out of town. Will he fill more seats, create a better vibe, provide an "it" factor the whole team benefits from and produce a better home record due to said "it" factor.

How measureable is this kind of parameter? I am sure someone can find a way. Kind of like valuing a stock, some irrational gut feelings play into many "professional" opinions.

[Mark]

How appropriate is 'Blink' to Tim Tebow just getting drafted in the first round. Cover his face and name and this guy never gets drafted based on his skill set. People are blinded by this guy's personality and the team he played for. I really don't see him doing anything in the NFL

[Pablo]

What would be a good idea, in my opinion, is to see what draft position gives the highest value with the lowest standard deviation. I think this stat would be a bit difficult to come up with, however, if you found that the most valuable draft position was in between the 2nd and 3rd round, what I would do on a yearly basis is, trade away my first round pick for multiple picks in that range.

This way, your likelihood of getting a player who will add value to the team, while keeping the salary of the team under control, would be much higher, and in turn give the team more freedom to get a free agent that would put them into the playoffs, and super bowl.

Lastly I want to say, I think the most important part of this is that, general managers aren't always looking for the best player. In my opinion, they are probably looking for the best player who will put people in seats and sell jerseys. However, even that thinking can be calculated. I think a team could do a cost benefit analysis of a player vs. getting a free agent in the same position, who is already established.

GM's are very irrational, and I think that the whole combine/workout for the NFL has too much emphasis.

I believe that like different economies around the world, every player has a TFP or some A in production functiong Y=AKL, were K=athletic ability L=Wonderlic score.

Most players will fall under a specific range but they will fall all over the draft. In this case, its this A that differentiates who is a star and who isn't. If there is a way for GM's to calculate this A of football players, and see what player is most productive, holding K and L constant, I think we'd have people making smarter decisions.

[propecia online]

Adrenaline is the main ingredient for many people regardless of race, color, age, religion practice these extreme sports.

[stephen patt]

then you hve $5 million less to
s/b
then you have $5 million less to

it is not easy to making decisions
s/b
it is not easy to make decisions

[Robert]

Malcolm,

I love your writing and am a big fan. In this particular instance I cannot help but think, however, that you may be oversimplifying a situation in a way that leads to a misleading conclusion.

First of all, the entire calculation of a players worth versus performance seems to be based upon an expectation of a linear correlation instead of other options such as an exponential relationship to pay versus performance. A player who is the best in the world maybe worth 1000 times or more what the 10th best player is worth if the difference means winning the championship.

Secondly the entire analysis overlooks the fact that Quarterbacks who are drafted into the NFL frequently undergo an extensive training apprenticeship process before they begin to start in the games that can last months or even a year or more. The player's eventual success could be significantly influnced by how competent teams are in their role as trainers.

In other words the quarterback is drafted for their potential, but their success on the field is in part due to how well the team cultivates and shapes this potential even further.

And finally, predicting success in aggregate is dicey because some coaches do this whole thing better than others, and the success of a particular quarterback may have to do with the style of offense the Coach is building. Joe Montana was not the number one draft pick, but he was the one Bill Walsh wanted because he recognized in Montana the charactersitics that would enable him to effectively run the offense Walsh was creating. Had Montana instead gone elsewhere he might have been a washout if his particular talents did not mesh with the offense.

Thanks again for a stimulating and thought provoking piece.

Robert

[Jordan AJF 8]

It is a really good go through for me, Must admit that you are one of the best bloggers I ever saw.Thanks for posting this informative write-up.

[Retro Jordans]

The only thing we can handle is to cherish what we have now, our career, dream, friends and lover.

[mbt shoes]

"Love is not a thing to understand.
Love is not a thing to feel.
Love is not a thing to give and receive.
Love is a thing only to become
And eternally be. ."

[Jordan 1]

As with much else in the world today, the pace of progress will depend on the US –the insufficient but still indispensable power.

[Elroy Hirsch]

Tim Tebow surpasses traditional conceptualizations and quibbling over such mundane matters as salary and draft pick scuttlebut.

The legendary Tim Tebow will just give whatever money he gets from football to charity for the betterment of orphans and mankind in general.

[Daniel Foley Jr.]

I read the article about Quarterbacks and couldn't help but feel like maybe football could benefit from a minor league system. I know it might be a bit unruly and that college is supposed to be the "minor league" for the pros, but, at least for the quarterback position, it might be beneficial. Look at baseball. At any one time, MLB teams have something like 15-20 outfielders in their system, about 75-ish pitchers, and at least 6 to 8 players auditioning for every other position. By the time a guy makes it to the big leagues in baseball, he's a lot less likely to flop.

[Mike Breidenbach]

Interesting story. I wonder if the NBA is better than the NFL in terms of predicting the eventual success of first rounders? I know that for every Sam Bowie there is a Michael Jordan or Hakeem Olajuwon, right? My gut says the percentage of success seem higher to me in the NBA draft, I wonder if statistics support my guess?

[detox marijuana]

arterbacks:
 
There are two separate issues with respect to quarterbacks. The first is whether, historically, NFL teams have done a good job of predicting which college quarterbacks will succeed in the pros. Dave Berri and Rob Simmons’ paper in the Journal of Productivity Analysis (th

[winter boots]

Much like Bean in Oakland, they adopted this type of reasoning before anyone else ever began contemplation.

[mesa doctor]

So, the best coaches overall are massively fat, followed by overweight, then normal and the worst coaches are fit. Perhaps the extra resentment some players may feel when yelled at by fatter coaches is somehow channeled into a focused anger that is then taken out on the field during games, leading to higher quality play.

[asics shoes]

these gleans for resource recycling played an effective role; 2 it is, they recycling waste resources http://www.asicsusashoes.com/

[mbt shoes]

with low cost and makes some to classify rubbish as raw material enterprise can produce lower prices of http://www.mbtusasale.com/

[ugg australia]

products. But, this kind of spontaneous loose organization gleans army harm also exist. In response, http://www.uggsaleaustralia.com/

[mztllmo]

lUqVFe vhxiodrxzklp, [url=http://ibyigkfpcroy.com/]ibyigkfpcroy[/url], [link=http://sqjrgyneutyp.com/]sqjrgyneutyp[/link], http://mymmwhnaltae.com/

[Mark]

I'm your biggest fan in Washington, DC!

I am reading your books now and watching all of your guest speaker events online.

Let your fans know what we can do to support your efforts!

[federal criminal lawyer]

Im very close in being convinced, you sure know what you are saying.

[trollbeads bracelets]

Image expecting a characteristic similar to this! Not too long ago whilst I used to following per yearning songs because of college I assumed to help ourselves, "I want to disclose in this thanks to my own HS amigos across my own Facebook or twitter page", and these days It's possible to. Wanted upgrade! In search somewhat more is intergrated into the future.

[lkj]n]

very gooooooooooood

http://nb-des.com/vb

[Coach Factory Store]

So fun article is! I agree the idea!