What surprises me most about this story isn't that it happened, but that people still are surprised it works. A data-driven approach will always be better than picking people by how good they "look", etc. Human scouts are biased, make mistakes, and miss plays. Statistics don't lie. The one catch is of course how you analyze those statistics makes all the difference.
NBA media and fans are much much more resistant to empirical stuff than baseball. I think it's because the appeal of the game is more about emotions and hype than baseball. There's this feeling that things can't be captured in the numbers, but what they use instead is so inferior for judging players. I mean the Knicks acquired Carmelo Anthony at the cost of all the efficient players on the team. All of the stats guys were very skeptical the media blared on and on about how they would be contenders and they were terrible until this undrafted rookie came along.
Another example, what I've found interesting reading some of the NBA statisticians blogs is this argument about "clutch". Most fans and media insist that Kobe Bryant, a player who shoots a lot of shots at the end of games and makes spectacular shots sometimes is the best closer in the game. Meanwhile almost all statistical evidence points to Chris Paul, a player who runs boring routine plays down the stretch, as by far the best closer in the league in terms of actually winning close games.
Until the NBA stats can predict winners like baseball, the public will always go by the "look." The teams themselves however, are almost all hiring statisticians now.
Baseball is easier to predict from statistics when compared to most other sports. Most plays in baseball occur in isolation (relatively).
For example: pitcher vs. batter -- each pitch is essentially a new, repeatable experiment.
In basketball, I would guess that there are very few repeatable experiments. (Different players, strategies, locations, etc). It is a much more fluid game and therefore determining an indicator of success is much more difficult.
Agreed. Also keep in mind baseball probably keeps track of more stats than any other sport, making it easier to spot potential.
I once read an interview with a Major League scout who said he relied heavily on the numbers. He basically said, "We can get any stats, on any of these guys, at any minute, and find out exactly where they are, and if they have the numbers for the big leagues."
Exactly. The key difference is how advanced the statistics that matter are in the different sports. Only very advanced stats will really tell the story in basketball whereas it is much easier to see in baseball with just baseline stats. Most laymen take that to mean that watching games is the only way to tell when, in reality, they just need to get the right stats and formulas.
At the core, stats tell a story of what a player has done in the past. Projections predict what a player might do based on what comparable players have done in the past. Statistics don't tell the future. They are just one tool of the scouting process.
Plus the models are always evolving. What was considered an indicator of talent 20 years ago might be inferior to the indicator 10 years ago that is actually inferior to what is in vogue today.
Statistics don't lie, but what story do you think they actually tell?
According to 'Thinking Fast and Slow' which I'm currently reading expert predictors of performance are generally worse than even quite simple statistical models. This being due to all the flaws in human decision making - if it's a sunny day, if the player 'looks the part' if the scout happens to view them on a day when they're feeling lucky. It's a very good book, but it suggests that in all fields of prediction you should actually rely on combinations of statistical metrics over expert judgment.
I would expect a complete statistical approach to work great for a sport like baseball where a team is exactly the sum of its players (i.e. most play is individual), but basketball it seems like basketball has so many other factors because you need all 5 players to work together to be effective. A weak player can bring everyone else down and a strong one inflate everyone else's stats. But maybe that just makes the statistics that matter more complex.
They have been getting much more complex. For example many stat geeks have been trying to make one that takes the +/- stats (how well the team does when the player is on the floor in terms of point difference per 100 possessions) and adjusting it based on the +/- of the teammates and opponents that have shared the floor with them. It's definitely more challenging than baseball, but there are a lot of interesting things to try.
>"A weak player can bring everyone else down and a strong one inflate everyone else's stats. But maybe that just makes the statistics that matter more complex"
That's exactly it. People often argue that something or other isn't accounted for, but the statistical methods are sophisticated.
Besides, it's not about being perfect, it's about getting an edge.
Not surprised that most experts(NBA & DI school scouts) don't embrace this because it is easier and safer to go with the typical archetype as opposed to raw stats. Its "human nature" safe in the sense most people will not question if you go with the established archetype. It also devalues the current process of scouting.
An apt quote from the movie Moneyball:
"I know you've taken it in the teeth out there, but the first guy through the wall. It always gets bloody, always. It's the threat and not just the way of doing business, but in their minds it's threatening the game. But really what it's threatening is their livelihoods, it's threatening their jobs, it's threatening the way that they do things. And every time that happens, whether it's the government or a way of doing business or whatever it is, the people are holding the reins, have their hands on the switch. They will bet you're crazy. I mean, anybody who's not building a team right and rebuilding it using your model, they're dinosaurs. They'll be sittin' on their ass on the sofa in October, watch the Boston Red Sox win the world series."
Erm, as I understand it most NBA organizations have significant statistics/economics departments. I don't think the practice of going by looks or things of that nature is dominant any more except for sports pundits trying to rationalize events after they have happened.
Yes, some NBA clubs have started to spend money on quant/stats but I wouldn't say there is a majority cultural buy-in particularly in regards to scouting. Mostly it is in the form of a stat guy assistance coach who will recommend how to play opponents.
It is gaining popularity. Boston Celtics are one of the pioneers. The Nets even Bought a sports video/stats startup recently. Time will tell if talent evaluation will also change as Lin has been a paradigm-destroyer of the old way of scouting.
"Sometimes the convincing force is just time itself and the human toll it takes, Kuhn said, using a quote from Max Planck: "a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it."[1]"
While it's true that sport scouting has traditionally under-relied on data, I wouldn't be so quick write the entire profession off entirely in favor of spreadsheets. Statistics can certainly be made to lie (cf. the book, "How to lie with statistics"), and there are a lot of aspects of human performance that are difficult to quantify. Performance under pressure is the first thing that comes to mind--reliably sinking a buzzer-beating three-pointer is a harder and more valuable skill to possess than just being a good shooter from beyond the line. You could try to quantify "performance under pressure", but a good scout will pick that up intuitively and probably do a better job of it. As always, the truth lay somewhere in the middle; a combination of both data and human input seems to be the best course.
The baseball equivalent, clutch hitting, has been kicked around in sabermetric circles for almost as long as Bill James has been writing of baseball's flawed stats and biases of pro scouts.
The "common wisdom" of sabermetrics (if you can call it such a thing) holds that clutch hitting doesn't exist, but many including James, aren't quite ready to completely write it off.
The common wisdom isn't that clutch hitting doesn't exist. Hindsight reveals that some people perform better in "pressure situations". The issue is that it's unpredictable and seems to be unrepeatable, and hence not applicable to models.
Your example is a good one: name someone who has a "skill" at draining game-winning buzzer-beaters.
it is true stats don't lie. but there is no replacement for the human judgment of personality and dedication. if it was all about stats, allen iverson would have lead the 76ers to multiple championships. THAT'S the importance of personality! thats the importance of showing up for practice. :)
Actually, a point guard led team winning a championship is rare. It's the reason people watch Rose[Bulls] with suspect. The last time it was done was with Piston during the Bad Boy's era[late 80s, early 90s]. So, although individual accolades accompany Iverson, I doubt anyone would predict him to win multiple championships without significant help[well except the media].
How good are the Oakland A's? Not very. So how successful is the data driven approach really? Isn't accepting a single case like this in the NBA as any kind of proof unscientific?
How good are they? You could try watching the movie,or better yet, read the book. They did well.
Unfortunately for the As (and fortunately for those of us who embrace the statistical analysis of sports) other teams caught up and surpassed their approach. But it's still going on. In every sport. And there is much evidence that it works.
But they spent a lot less money to play pretty well, whereas other teams spent a ton of money to play about the same. Only one team can be the best, but many can be very good.
