I wrote a piece for the The New Yorker a few weeks ago about a group of people who have created a neural network that predicts (or tries to predict) the box office of movies from their scripts. (It's not up on my site yet, but will be soon).
The piece drew all kinds of interesting responses, a handful of which pointed out obvious imperfections in the system. Those criticisms were entirely accurate. But they were also, I think, in some way beside the point, because no decision rule or algorithm or prediction system is ever perfect. The test of these kinds of decision aids is simply whether--in most cases for most people--they improve the quality of decision-making. They can't be perfect. But they can be good.
In "Blink," for instance, I wrote about the use of a decision tree at Cook County Hospital in Chicago to help diagnose chest pain. Lee Goldman, the physican who devised the chest pain decision rule, says very clearly that he thinks that there are individual doctors here and there who can make better decisions without it. But nonetheless Goldman's work has saved lots and lot of lives and millions and miillions of dollars because it improves the quality of the average decision.
Is the average movie executive better off with a neural network for analyzing scripts than without it? My guess is yes. That's why I wrote the piece. I think that one of the most important changes we're going to see in lots of professions over the next few years is the emergence of tools that close the gap between the middle and the top--that allow the decision-making who is merely competent to avoid his errors to be reach the level of good.
I think the same perspective should be applied to the basketball algorithms I've been writing about. It is easy to point out the ways in which either Hollinger's system or Berri's system fail to completely reflect the reality of what happens on the basketball court. But of course they are imperfect: neither Berri or Hollinger would ever claim that they are not. The issue is--are we better off using them to assist decision-making that we are making entirely judgements about basketball players using conventional metrics? Here I think the answer is a resounding yes. (Keep in mind that I live in New York City and have had to watch Mr. Thomas bungled his way toward disaster. I would think that.)
And the reason that lots of smart people, like Berri and Hollinger and others, spend so much time arguing back and forth about different variations on these algorithms, is that every little tweak raises the quality of decision-making in the middle part of the curve just a little bit higher. That's a pretty noble goal.
That said, here are the latest updates on the Hollinger-Berri back and forth. And remember. I don't think this is a question of one of them being wrong and the other right. They are both right. It's just that one of them may be a little more right than the other.
Here we go. First Hollinger's response, courtesy of truehoop.com, (an excellent site by the way.)
And then. Berri's response.