You Are Here > Basketball-Reference.com > BBR Blog > NBA and College Basketball Analysis

SITE NEWS: We are moving all of our site and company news into a single blog for Sports-Reference.com. We'll tag all Basketball-Reference content, so you can quickly and easily find the content you want.

Also, our existing Basketball-Reference blog rss feed will be redirected to the new site's feed.

Basketball-Reference.com // Sports Reference

For more from Neil, check out his new work at BasketballProspectus.com.

The 2008-09 All-APBRmetrics Team

Posted by Neil Paine on May 15, 2009

When the league announced the All-NBA Teams this week, there were relatively few surprises. For those who haven't seen them already, the selections looked like this:

2008-09
FIRST TEAM SECOND TEAM THIRD TEAM
Dirk Nowitzki, Dallas
LeBron James, Cleveland
Dwight Howard, Orlando
Kobe Bryant, LA Lakers
Dwyane Wade, Miami
Paul Pierce, Boston
Tim Duncan, San Antonio
Yao Ming, Houston
Brandon Roy, Portland
Chris Paul, New Orleans
Carmelo Anthony, Denver
Pau Gasol, LA Lakers
Shaquille O'Neal, Phoenix
Chauncey Billups, Denver
Tony Parker, San Antonio

I thought it would be interesting, though, to see how the analytic community would have picked the teams, assuming we filled out our ballots according to our respective metrics. The rules for voting would be the same as those for the real panel of sportswriters that vote on the teams: five points for a first team vote, three points for a second team vote, and one point for a third team vote. Positions are defined by our database here at Basketball-Reference (unless we absolutely need to consider a F at C); each "team" will consist of two forwards, one center, and two guards. And each metric's ranking is according to its own preference -- for instance, PER and adjusted +/- prefer to rank players on a per-minute/possession basis, while Win Shares rank on the basis of cumulative value over the 0-win margin. Here are the sources I'm using:

Win Shares (Justin Kubatko)
PER (John Hollinger)
Adjusted +/- (Dan Rosenbaum/Aaron Barzilai)
Statistical +/- (Dan Rosenbaum/Neil Paine)
eWins (Mike Goodman)
Wins Produced (David Berri)
Composite Score (Jon Nichols)
Tendex (Dave Heeren/Doug Steele)
WARP (Kevin Pelton)
Roland Rating (Roland Beech)

(If for some reason you have a rating system and I forgot you, drop me a line and I will update the voting to reflect your results.)

Here are the results of the "voting":
(UPDATE: Added WARP & Roland Ratings -- thanks, Kevin & Roland!)

Player Pos WS PER APM SPM eW WP CS Tndx WARP RR Total
LeBron James F 5 5 5 5 5 5 5 5 5 5 50
Chris Paul G 5 5 5 5 5 5 5 5 5 5 50
Dwyane Wade G 5 5 5 5 5 5 5 5 5 5 50
Dwight Howard C 5 5 5 5 5 5 5 5 5 45
Yao Ming C 3 3 5 3 1 3 3 3 3 27
Kobe Bryant G 3 3 1 3 3 1 3 3 3 3 26
Tim Duncan F/C 1 5 5 3 1 1 5 1 22
Dirk Nowitzki F 3 3 5 5 1 3 20
Pau Gasol F/C 5 1 3 1 3 1 3 3 20
Brandon Roy G 3 3 1 3 1 1 1 3 3 19
Chris Bosh F/C 1 1 1 3 3 3 12
Kevin Garnett F 5 5 10
Jason Kidd G 3 3 3 1 10
Lamar Odom F 3 3 3 9
Gerald Wallace F 3 1 3 1 8
Rashard Lewis F 5 1 1 7
Troy Murphy F 1 5 6
Ray Allen G 1 3 1 5
Rajon Rondo G 1 3 1 5
Shaquille O'Neal C 1 3 1 5
David Lee F/C 1 3 1 5
Andre Iguodala G 3 1 4
Danny Granger F 3 1 4
Al Jefferson F 3 1 4
Paul Pierce F 3 1 4
Deron Williams G 3 3
Marcus Camby F/C 3 3
Tony Parker G 1 1 1 3
Nene F 3 3
Andrei Kirilenko F 1 1 2
Steve Nash G 1 1
Jameer Nelson G 1 1
Antawn Jamison F 1 1
Chauncey Billups G 1 1
Manu Ginobili G 1 1
Rasheed Wallace F 1 1
Ime Udoka F 1 1
Chris Andersen F/C 1 1
David West F 1 1

First Team:
G Chris Paul, Hornets
G Dwyane Wade, Heat
F LeBron James, Cavs
F Tim Duncan, Spurs
C Dwight Howard, Magic
Second Team:
G Kobe Bryant, Lakers
G Brandon Roy, Blazers
F Dirk Nowitzki, Mavs
F Pau Gasol, Lakers
C Yao Ming, Rockets
Third Team:
G Jason Kidd, Mavs
G (tie) Ray Allen & Rajon Rondo, Celtics
F Lamar Odom, Lakers
F Kevin Garnett, Celtics
C Chris Bosh, Raptors

There were not a lot of differences from the "official" team at the top (the most obvious one being Chris Paul's unfortunate snub to 2nd-Team status because humanity would never hear the end of it if Bryant wasn't a 1st-Teamer, & they couldn't justify sending Wade down either). But as you move down, the teams become more and more divergent -- the metrics don't consider Pierce, Parker, Shaq, or Billups to be All-NBA material, and Carmelo Anthony (sometimes regarded as one of the league's most overrated players in APBRmetric circles) didn't garner a single "vote". So it's apparent that we're not all on the same page here.

But at least the two groups of voters can agree that D-Wade, LeBron, and Dwight Howard are 3 of the most elite players in the league. And almost every metric I came across listed James as the game's best/most valuable performer this season, which also matches the voters' official MVP decision. We may disagree on a lot of things, but there's no difference of opinion on who the absolute cream of the crop has been this season: he's the guy who wears #23 and plays his home games in Ohio.

ShareThis

60 Responses to “The 2008-09 All-APBRmetrics Team”

  1. Ed The Sports Fan Says:

    I love this, even though i'm a big Melo fan and he didn't even get a sniff on this list, it goes to show what "real" productivity looks like. There's no way in life Chris Paul shouldn't be 1st team anyway, get real.

    -Ed.

  2. Jason J Says:

    The first team is even closer than you mention. Gasol's only got 1 "vote" more than Dirk in the metric team, so the team could easily be off by nothing more than the perception that Kobe has to be the greatest today since his move-set is reminiscent of the player who most consider the greatest of all time. BTW what would have happened if Kobe was taken as a forward? Does that slide him up or down the scale?

    The other interesting point that comes out of this is just how divergent these metrics can be. After the top 11 vote-getters there's not a single player that more than 3 metrics can agree on. A few players like Garnett, Rashard, and Troy Murphy got first team votes from one metric without another first or second team vote from any other metric.

  3. Neil Paine Says:

    It is interesting how everyone agrees who the very best players are, and then you get a crazy divergence in opinion the further down the list you go. And if you're running a team, the analytic edge comes the lower you go on that list. Any moron knows LeBron James is the best SF in the game, but what happens when you have to make a call between two seemingly average SFs, and various metrics are giving you different answers about who is better? It may not seem important, but those "little" decisions add up to big successes or failures in the long run.

    And yes, Kobe would have made 1st Team had I considered him a forward. But he's not a forward. And neither are Wade or CP3.

  4. Tsunami Says:

    Love it.

    These players are the best players because they contribute the most to WINNING.

  5. S Vu Says:

    Why can we consider players F/C and choose which one fits better and not players to be SG/SF and choose which one fits better

    Some years Tracy McGrady is a SF, some years he's a SG, but does his game really change?

  6. haushinkaa2 Says:

    RE 3.:

    It is NOT interesting how everyone agrees on the best players and it gets divergence.

    These fancy statistics were created and accepted only because they matched popular perception on the top players. If I came up with a system that rated lebron an average player, who's gonna endorse it?

    The fact that they get divergent shows that either
    a) the signal to noise ratio of the data used is not high enough to differentiate good from bad players
    b) most (if not all) these measures are totally unmeaningful. (the accurate part is the top players by design).

    I personally believe in both a) and b). It is difficult if not impossible to evaluate a team just from (possibly fancy) box scores. There are really phony measures in this list you compiled.

  7. Brandon Says:

    @tsunami: according to this no Nuggets should have been on the All-NBA team yet 2 Mavericks should be. Please explain how it equates to winning?????

  8. Anon Says:

    @ haushinkaa2,
    Yeah, you're right. Neil and these guys were fooling us all along, trying to come up with these fancy-schmancy formulas that will make LeBron look good on purpose. Truth of the matter is that they're all part of the "Let's make LeBron the new face of the NBA!" conspiracy. Somewhere, the evil David Stern is smiling right now...haha, give me a break.

    You're definitely in the wrong place if you're going to come here to make that kind of fallacious argument.

  9. haushinkaa2 Says:

    @ Anon

    I think you're misinterpreting my point. Firstly, it's not meant to bash lebron. By any measure, lebron is (among) the best in nba right now. You don't need formulas to tell you that. Secondly, I didn't say neil and the rest are fooling us. I do believe they, at least in principle, are trying to apply what they believe is true.

    My point is that, despite their intentions, some if not most of them, are not measuring anything valuable. When you come up with a system to rank players, you're only inclined to believe it if the elite players are ranked high. So it is no coincidence that the various systems agree on the top players. Again, just in case my point is still not clear, this is because the system that don't would have been abandoned and not included in this meta-analysis.

    In effect, each system calibrates by the top players and extrapolate to the rest of the nba population. And it's not surprising at all that these extrapolations don't match further down the line.

  10. Justin Kubatko Says:

    Haushinkaa2 wrote:

    In effect, each system calibrates by the top players and extrapolate to the rest of the nba population. And it’s not surprising at all that these extrapolations don’t match further down the line.

    Speaking only for Win Shares (since I created the system), I can say the statement above is absolutely false.

  11. Neil Paine Says:

    I understand what you're saying, haushinkaa, but I can assure you that I have not left off any metrics because they don't agree with the others (I included Wins Produced, didn't I?). I added 2 more metrics this afternoon and I would encourage anyone who has a system that I've overlooked or am unaware of to send me a link, and I'll include it as well.

    Regarding the suggestion that the systems are engineered intentionally to have the best players by conventional wisdom come out on top, I can say with 100% certainty that this is not the case for Win Shares, Adjusted +/-, or Statistical +/-, because those are the 3 I know inside and out (in terms of how they are calculated, and also where the inspiration came from). APM is the ultimate "unbiased" stat, using no boxscore stats, and SPM is an OLS regression I ran myself a few weeks ago on the results of APM since 2003. Win Shares uses Dean Oliver's offensive and defensive ratings, which are based on probability theory and are outlined in Basketball on Paper. I believe WARP uses the same dataset, but I'm not completely sure on that.

    I am sure that Wins Produced, while flawed for other reasons, is not engineered to match the public's preconceived notions of value. Prof. Berri's work is based on regression analysis at the team level, and he almost takes a perverse pride in the way it flouts conventional wisdom. Now, I think he's misapplied those team regressions at the individual level, but that's an entirely different argument.

    Tendex is basically NBA Efficiency, the simplest formula with a weight of 1 for every category, so it hasn't really been rigged at all. It's horribly flawed, but not for the reason you mention. Composite score is a meta-rating that incorporates multiple metrics, as is Roland Rating. I have no idea if they've been rigged, but if their component metrics haven't been, then there's a good chance they haven't been either.

    I cannot speak to PER or eWins, because I don't know specifically how they were created. I don't think PER is rigged to match preconceived notions of value, but I can't say for sure. And next to nothing is known about the process behind eWins, so I have no idea whether what you're suggesting is true of it or not.

    Besides, this was supposed to be a fun alternative to the writers' voting process, not a rigorous player ranking. And I resent the suggestion that I've left off systems that are divergent at the top. I have welcomed anyone with a stat-based ranking to send their results to me, and I continue to ask readers to do that.

  12. haushinkaa2 Says:

    @ Neil

    Thanks for your informative response. I'm actually a statistics phd student, specializing in machine learning aka high dimensional regression/classification. Your response is comforting since I've more than once recommended your site to those who ask me statistical questions about basketball despite not having looked through your site extensively.

    I apologize for implying that you endorsed these systems without merit. You clearly took the effort to learn and understand them. I also apologize (esp to Justin) for lumping all the systems together with one which I know for sure is phony (guess which one).

    That being said, I think my point is still slightly misinterpreted. Even without any engineering (of any individual system), there is still the selection factor (as in natural selection like evolution). If a system is too "perverse", it wouldn't be popular. And despite your efforts to factor in all such systems, there is bound to be more popular systems included. Just to be clear, I am not suggesting *you* left off systems that are divergent at the top, but the effect is still there, either by the creators abandoning them, or they never getting noticed.

    I think what we really disagree on is why they diverge. Correct me if I'm wrong, but you seem to think that the different systems are capturing different (real) factors, and hence it'd be interesting to see where they diverge (eg. is troy murphy really good?). I, on the other hand, feel that they diverge because the data fed into the systems is too noisy, so they can't tell players apart unless one is "clearly" superior to the other (eg. duh of course lebron is good).

  13. haushinkaa2 Says:

    and for what it's worth, if I had to build a model from scratch, it'd be based on +/- with a crapload of adjustments.

  14. Neil Paine Says:

    Thanks for responding thoughtfully (sometimes we get trolls around these parts; I'm glad you're not one of them). And I do agree that in an exercise like this you may very well see unintentional selection bias, in that systems with more "off the wall" results will never even make it to my inbox. I don't know how you correct for that, though... It's probably not possible to find every formula that has ever been devised and include them (my experience is that most look like Tendex anyway -- they assign each category a default weight of 1).

    Now, as for why we see so much divergence everywhere but the top... Well, every system but APM is really working from the same dataset -- the only reason for the differences is different weighting for the various box score categories (give or take a few team-based adjustments for defense). So it makes sense that players who excel in multiple categories (cough, LeBron) show up as the best, because at that point it doesn't matter what the weighting is for each cateory, they're good in all of them. But when you leave the elite who do well in many categories and get into more one-dimensional players, that's where the divergence comes -- some systems favor scorers, some like rebounders and efficient shooters, some like assist men, thieves, and flyswatters, etc.

    LeBron, Howard, Wade, and CP3 are good at just about everything the box score tracks, and that's why they're universally agreed upon as the best. But after them, you see the different weights at work, and things diverge. I argued that this could provide some team an edge because (theoretically) there is some set of weights that best approximates the reality of basketball. The closer you get to it -- and the further away some of your opponents are from it -- the greater your opportunity is for arbitrage.

  15. Anon Says:

    @ haushinkaa2,

    In the middle of typing a lengthy response it turns out that Neil and Justin just HAD to beat me to it. Haha, but I think their responses (especially Neil's) capture my sentiments exactly.

    But while I certainly sympathize with what you're saying - namely, how do statisticians make sure that they're not just loading their formulas so they can appease to their preconceived notions - I think you have to keep in mind that these ranking systems diverge because they often place different subjective WEIGHT upon the various aspects of the game in their analysis, such as field-goal percentage vs. assists for example. Or, even if you're not dealing with box scores such as in APM, you still have to base that system upon a principle that you believe is the most important factor in a basketball game ("Is my team scoring more points than the opponent when I'm on the floor?") People have differing opinions on how these factors ultimately affect the outcome of games, and in turn it will lead to differences in the rankings. In this sense, APBRmetrics (as well as statistical analysis in baseball and in other sports) is definitely an art as well as a science, and is precisely the reason why there's no one "be-all, end-all" system or ranking. Just some food for thought.

  16. Kevin Says:

    Haushinkaa2,

    I understand your point, but it's sort of a "Duh" point.

    Of course a system wouldn't be accepted if it determined Troy Murphy was the best player, and LeBron James was No. 20. That's because any system has to pass a "smell test." That system would have to really prove its mettle and go through rigourous tests from the rest of the statistical analysis world.

    Of course different systems are going to disagree on who the 13th best or 21st or 50th best player is. Not because there is too much noise. But because those things are harder to determine to begin with. More players are close to the same ability when trying to determine the 13th best player.

    But if you look at the actual numbers prodcued by these metrics, they basically admit that such things are hard to determine. For example with PER, players 10 through 18 (Yao through Jamison) have PERs of 22.6 to 20.6. The players in that range are basically the same -- a coin-flip away from each other. That's how talented distribution works. I'm sure players 50 through 58 are even closer together than 10 through 18. There are more average players in any league than All-Star level players.

  17. Nat Says:

    Troy Murphy kind of sticks out like a sore thumb with that single 1st team vote, doesn't he? Another reference point in the: "Is Berri overcounting defensive rebounding when he assigns credit to individuals in WS?" question.

    Murphy's defensive rebounding rate this year was a staggering 32%. Good for #10 all-time since offensive/defensive rebounding is available for high minute (2000+MP) players. He trails only Swen Nater (3 times), Dennis Rodman (4 times), Ben Wallace, and Will Walton. Offensive rebounding? Not so much. His 6.4% offensive rebounding was good enough for #36 for high minute players this season.

    Murphy was the designated defensive rebounder for the team this year. Those could have been Foster's...or Hibbert's...or Nestrovic's...but he was the guy who was designated to get the rebounds contested between teammates this year to start the offense.

    His DRB% - ORB% is 25.6%. The only player with a higher disparity was Bill Walton in 1977/78. Only Marcus Camby in one year comes within a point of this kind of difference.

    With Walton, you can argue that he was one of the best outlet passing bigs in history, so it made since for defensive boards contested between teammates to go to him. Murphy was more or less a default choice.

  18. biggles Says:

    (I think this is connected to Haushinkaa2's notion of "noisy", though I'm uncertain of his definition.)

    Here are two questions:

    (1) How do we measure how good Joe Rookie is?
    (2) Given a metric, how accurately can we measure Joe Rookie's goodness?

    There are lots of answers to (1), but with many systems, we're going to have a very wide confidence interval for Joe's goodness, even with a full year of data. That's why we often ask a different question:

    (3) How do we measure how good Joe Rookie was this season?

    Once a metric is chosen, this can usually be measured much more precisely.

    I think that most systems are attempting to find a good answer for (3), which may or may not lead to a good answer for (1).

  19. haushinkaa2 Says:

    @Neil

    Ha, thanks for responding kindly. I was responding thoughtlessly to begin with. I'm glad we've come to terms though. I agree wholeheartedly with your last reply. And in view of that, do you still think it's interesting that they agree on the top but not down the list?

    @Kevin

    When I say the data is too noisy, I mean relative to the signal. You are in fact making my point. The middle pack of nba players are so close in ability, that box scores or whatever measure you like is unlikely to be able to tell them apart. (Though it is unfortunate you chose PER to illustrate your point.)

  20. Kevin Says:

    Sure, it is "unlikely to be able to tell them apart."

    But every metric is trying to get closer and closer to the truth than we were yesterday, even if we are not at the whole truth yet.

    Why is it unfortunate that I picked PER?

  21. Kevin Says:

    I know that I was making your point. I was just sort of saying your point is sort of a fact of life, not necessarily a criticism of the metrics. You could say the same thing about any stat or metric in any sport. One player averages 25 points, another 22. They are really close as far as point production.

  22. Carlos Says:

    I think you guys are defending "metrics" against an attack that never took place.

    Neil Pane pointed out that it's interesting that the First Team voted on by people matches the First Team selected by the metrics. haushinkaa2 says that's not surprising; any system that didn't have Kobe, LeBron, Wade, and DHoward as its top players would be tossed out or never published. That's a fair point. There's nothing very controversial here, he' right. Even when numbers are involved, human biases and patterns have a huge influence. That's something a casual fan getting into the NBA stats scene needs to appreciate.

  23. Tsunami Says:

    Sure there isn't a hole lot of controversy - that's because Manu Ginobili didn't play this year.

    When he's healthy, he comes very close to Kobe in a lot of these categories - way too close for Kobe Nation.

    As far as results diverging between the metrics when you get near the middle of the pack, that's obvious since there is a much bigger pool of average players than elite players.

    I'd like to see more work done with defensive metrics - that is the area where nobody has any sense of reality.

  24. An Onimous Says:

    Why does everyone keep saying that there's no divergence between the statistical models and the human perception at the top? The humans say that Carmelo Anthony was one of the 6 best forwards in the league. The statistical models wound up assigning at least one point to 22 different Fs (and F/Cs)... and Anthony wasn't one of them. Chauncey Billups was a popular fixture on NBA MVP ballots, meaning people were saying with a perfectly straight face that Billups was one of the five most valuable players in the entire league. The statistical models assigned him one single point- which tied him with teammate Chris Andersen, and left them both behind teammate Nene Hilario. The human perception tends to be that Chris Paul, Tony Parker, and Deron Williams- in one order or another- are without question the three best PGs in the league, at least during the regular season; the statistical models assigned 50 points to Paul, and 3 each to Parker and Williams (behind Brandon Roy, regular season Rajon Rondo, and the artist formerly known as Jason Kidd).

    Under any definition of "the top" that includes more than 8 total players, I would say that the statistical models *DO* diverge greatly from popular perception at "the top". Maybe not at the TIPPY-TOP, but they do diverge at the top. The aforementioned Kidd over Parker and Williams, for instance. Or Lamar Odom over any number of more widely acclaimed forwards, starting with Carmelo, Pierce, Granger, and West. Everyone might agree with #1 at every position, but the divergence starts as soon as when you start discussing who the second best player at any given position might be.

    I would actually argue that the reason everyone agrees on the very very tippy-top has nothing to do with natural selection and everything to do with overwhelming dominance over one's peers. Can someone come up with some way to weight any data outputs from this past season that *DOESN'T* result with LeBron, Dwight, Dwayne, or CP3 coming out on top? Unless you're doing something ludicrous like counting turnovers as a positive or penalizing a player for minutes played, there's really no way. Those guys were among the league leaders in so many different categories (both in terms of box score and in terms of +/-) that any weighting is going to recognize their value. You don't see systems without those guys at the top, not because of any sort of "smell test", but because such a system would be incredibly difficult to devise. You'd pretty much have to start out with a stated goal of creating a statistical metric that DOESN'T rank those guys as elite.

  25. An Onimous Says:

    Oh, and as for Carmelo Anthony... general perception is that he's a lot better than most of his peers. The statistical perception is that he's nothing more than an average player in terms of what he brings to his team. Personally, I think both sides are right. I think that Carmelo is one of the most talented pure scorers in the entire league, but I think that that talent doesn't translate into dramatically improved scoring for his team. It seems like a paradox, but it makes sense if you think about it. Think of a tough, low-reward shot- say a contested 18-footer. Imagine that a league-average player might shoot 35% on such a shot, but Carmelo is such a great scorer that he shoots 45%. Of course, since Carmelo KNOWS he's such a good scorer, he's more likely to jack up that 45% jumper, whereas the lesser scorer would either work his way into a lower-risk shot, or would back up into a higher-reward shot, or would give up the ball and let his teammates attempt to do it, instead. Carmelo could shoot better than a league-average forward on every single shot in the book, but thanks to Simpson's Paradox, the league-average forward could shoot a higher percentage overall, and draw more fouls and get more assists, to boot, making the league-average forward more valuable to the team.

    I think that Carmelo has given lots of reasons to believe that he's better than his regular-season stats indicate. For one, his performance on the Olympic team. For another, his status as the best "clutch" scorer in the NBA (in "the clutch", often the ONLY shots are low-percentage shots, which lets his natural advantage on such shots show through the noise). For another, his dramatic increase in efficiency so far in the playoffs. I agree that Carmelo is overrated, but I also agree that he's supremely talented, and I believe that sooner or later he's going to put it all together and start making contributions more commensurate to his abilities.

  26. Derek Says:

    Thanks for this. Even without the stats, you'd be hard pressed to explain why Chris Bosh isn't a top 15 player in the NBA.

  27. Neil Paine Says:

    Great comments, An Onimous. Obviously we were focusing on the fact that LeBron, CP3, Wade, & Howard are essentially unanimous 1st-Team selections here, but after that group (the "tippy-top", I like that) is where the divergence begins. And like you said, it's the weighting: those 4 are so good in so many categories that it's basically impossible to find a set of weights that doesn't have them ranked as the best in basketball. It's only when you get down into players who aren't so dominant/multi-dimensional that you see the different weightings begin to influence the results for each metric. That's why it's interesting to see who emerges as the "next-best" in each metric, because it sheds light on what categories that particular metric values.

    I also agree about Melo, although it appears to me that a lot of metrics do see him as above-average, but just not elite. And I think it's probably true that some players' skill sets and/or playing styles (Kobe, Melo, etc.) lend themselves to somewhat lower-percentage play across the entire game, but the ability to maintain that percentage in situations where the rest of the league's % goes down dramatically -- the end of the shot clock, isolations, clutch shots -- gives them an edge that the stats don't pick up on fully. At the same time, though, you have to consider that you wouldn't necessarily need their crunch-time heroics if they had been as good as somebody like LeBron in the entire game leading up to the clutch situations.

  28. Zandungeo Says:

    Great post Neil!

    Look at what Berri had to say about this:

    "Okay, I am overwhelmed by sarcasm (be forewarned):

    Yes Westy, you have hit on the essence of how science should work. We simply take a vote to see which model is the best. And in the spirit of democracy, everyone gets to vote. You don’t understand standard errors… you still get to vote. Don’t get residuals… you still get to vote. Don’t get statistical analysis at all…. hey, you still get to vote (and by the way, I have seen many people who claim to do statistical work on-line who fit all three of these descriptions).

    And all votes count the same. Whichever model gets the most votes, that is clearly the best.

    This is what I do when I get a paper to review form a journal. I take it to my family and friends, regardless of whether they have studied economics or not. If they think the paper is good, I recommend publication. If not, I say it should be rejected.

    Any other process would be unfair. Why should anyone think we should base our evaluations on the actual evidence ? That is just elitism at its worse."

    What a jerk!

  29. Justin Kubatko Says:

    Zandungeo wrote:

    Look at what Berri had to say about this:

    I'm not sure those comments were in response to this post. I have posted a comment on David's blog asking for clarification.

  30. Zandungeo Says:

    Thats fair I guess I could have misinterpreted it.

  31. Justin Kubatko Says:

    Berri's comments on the WoW blog were directed toward another commenter, not this blog post, so let's drop this angle. Thanks.

  32. rusty Says:

    plz explain the nuggets, model-developers. cause if the players aren't responsible for our 54 + 8 wins so far, and i can assure you george karl is not responsible for them, where do they come from? The altitude?

  33. Neil Paine Says:

    Just because only a few systems considered Denver players to be among the Top 3 in the league at their positions doesn't mean they think the Nuggets are populated with a bunch of bums. By my estimation, Denver has 6, maybe even 7 above-average players -- it's just that none of them crack the Top 3 at their position.

  34. Justin Kubatko Says:

    Rusty, the Nuggets players as a whole had 50.7 Win Shares, which is close to their actual total of 54 wins (and very close to the 50.4 expected wins based on their points scored/allowed). Chauncey Billups led the team with 10.2 WS, the 11th-best total in the league and the 2nd-best among point guards.

  35. Gordon Says:

    It's funny that nobody mentioned that 73% of Chris Paul's WS were the offensive kind (12.8 vs 4.7) yet he made first team all-defense. His offensive and defensive WS were almost identical to his 2008 numbers.

  36. Gordon Says:

    Michael Jordan's best PER was 31.7 at age 24. LeBron's best PER to date was this season, 31.7 at age 24.

  37. Zandungeo Says:

    If you look at both across their first 6 seasons, Lebron has a 26.2 PER and Jordans is 29.9.

  38. Gordon Says:

    Yes, but look at Jordan's season #1-4 versus LeBron's season #3-6 (aged 21-24 for both) and they are about equal.

  39. Anon Says:

    I just have a quick question, Jason (or Neil as well), does Win Shares calculate how many wins a player is worth to HIS team, or to any ONE team in the league? Just wanted some clarification - thanks in advance!

  40. Neil Paine Says:

    His team. Ideally you'd like it to carry over from team to team, and in most cases there's a pretty strong relationship between performance from year to year, but since we have to use team defense in the formula for DRtg, there will inevitably be discrepancies when players switch between teams at extreme ends of the defensive spectrum (Ray Allen 2008, for instance).

  41. Dave Says:

    Neil, You mention the difference between the cumulative metrics like Win Share and per-minute Production Metrics like PER. If you are going to make some season-long comparison giving equal weighting to the metrics you need to address this problem.

    Is it reasonable to consider Jefferson's PER on a very brief season? Conversely, do we require players to play in as many games as possible for them to be considered for Season long honours (i.e. for cumulative metrics like win share). Personally I think that anyone playing less than 50 games in the season cannot be considered, but would also like to think a player is allowed a minor injury (say miss 5 - 7 games) and that won't punish them (oh but it will for wins produced etc).

    One solution would be to convert the cumulative metrics to a per game / minute basis - and since most of the PER-type metrics have minimum criteria it seems reasonable to establish a minimum criteria for length of season played too.

  42. Neil Paine Says:

    I relied on each metric's own preferred playing-time cutoff; for PER I believe it's 500 minutes, for SPM I chose 2000 minutes, for APM it's 1,155.67 minutes, etc. Remember, we're trying to leave each system to its own devices here and have them represent "voters", so it's really the creator's decision to have either judge on a cumulative or a per-minute basis.

  43. Mountain Says:

    The APBR 1st -3rd teams by position got 84% of the total points available.

    4 of the top 5 perimeter players were leading assist men on their team but not "officially" at PG. A pretty successful way to go if you got a guy who can fill the role.

    LeBron is the only player on these lists bigger than 6'6 but less than 6'9. And yet a lot of league is playing guys who fall in between. The composite player, potential fulfilled to highest level only by LeBron? On the All-NBA lists only Anthony adds to this mid-sized player count.

  44. Mountain Says:

    Correlation of metric point assignment and the APBR average:

    WS PER APM SPM eW WP CS Tndx WARP RR
    0.816 0.838 0.680 0.773 0.869 0.469 0.712 0.780 0.861 0.786

    eWins is most consistent followed by WARP and PER.

    Wins Produced by far the most different. Adjusted Plus Minus second.

  45. Mountain Says:

    Ah formatting...

    try this
    WS 0.816
    PER 0.838
    APM 0.680
    SPM 0.773
    eW 0.869
    WP 0.469
    CS 0.712
    Tndx 0.780
    WARP 0.861
    RR 0.786

  46. Anon Says:

    @ Neil,
    Thanks again for your earlier response about win shares.

    @ Mountain,
    Thanks for the data work. Interesting results...it would be nice to see how consistent the different metrics are over a longer time period than just one season, at least for the ones that are calculated from the box score stats. Surprised to see PER do pretty well for this season, I've always been a little skeptical about it. But of course, I'm still learning more about all of these ranking systems and the different methodologies that they employ in their analysis.

  47. Mountain Says:

    Thanks for saying thanks anon. It is a good thing to sprinkle around.

    Yes a longer study would yield a more solid basis for spotting trends or determining meaning.

    For the heck of it I compared the correlation of Wins Produced to all other metrics for this list. Surprising and interesting to me the highest correlation was with Roland Rating at a very high .898. Lowest correlation was with statistical at just .342 and not as surprising given the very different values for defensive rebounds (and missed shots). WP and adjusted correlate at a middling .602.

  48. Mountain Says:

    Classical adjusted and statistical plus minus despite being from the same source and sharing some assumptions are actually the least consistent pair with adjusted at .293. Composite Score also surprising to be second least consistent though its complexity might be the cause. Most consistent with adjusted? A dead heat between PER and Ewins at .723.

    Most consistent to PER? Composite score at .921. SPM second. Least? WinScore at .353. Followed real close and surprisingly by Tendex. They are not interchangeable.

  49. Mountain Says:

    FYI these numbers are based on correlation for the entire list. The top 15 probably would tell a different story.

  50. Mountain Says:

    Use just the top 15 and WARP is most consistent to APBR average and WP pulls up into second. Roland ratings slips into least consistent.

    But as anon said, probably need a longer study. And of course the meaning or value of consistency to this 10 part average is the eye of the beholder since it is dominated by box-score based metrics of various flavors and twists.

  51. Mountain Says:

    P.S. Not directly on this exact topic but I wondered how Dave Berri's Wins Produced for teams would have done for predicting the playoffs and why he choose to go with efficiency differential ... , home court advantage, and any relevant injuries" instead. If Wins Produced is the product of the best academic research why not use it? For the first two rounds it would have been the same except it would have taken the Spurs in the first round and the Celtics except for the injury factor and I can understand switching for that reason. But if Wins Produced does exactly what efficiency differential, home court advantage, and any relevant injuries does and the author of Wins Produced used them for his playoff predictions rather than his own model was that academically sound model really that important or better than the rest of the stuff out there and better than the stuff other people without a PhD level formal statistical training are using? Just asking.

  52. Mountain Says:

    Dave Berri at his blog:

    "So this contest really is not a “test” of anyone’s ability to evaluate teams. Again, this is because a) I think we essentially have the same evaluation and b) the playoffs are simply not designed to test that evaluation. Of course if I get this test right, then we will forget everything I just said and conclude that I really do know something :)"

    Ok so this doesn't matter. Or it might.

    The regular season gave a fuller test and several versions of Wins Produced compiled by another person not the author faired poorly compared to the rest of the metric and non-metric based competition that was won by Bill Simmons. But this doesn't matter. Or it might. I'll leave it there.

  53. gdth Says:

    My $.02 about ratings at the top agreeing, while ratings lower down don't..

    Think horseracing..in a 12 horse field, easy to pick out 2-3 favorites (good in may categories), almost impossible to distinguish between 8th-12th best horses...

    Hold true in mamy (most?) competitive endeavors..Another example, world-championship chess: 4-5 WC contenders (Anand/Topolav/Kraminik/Carlsen)...but #50-100 by FIDE rating, virtually interchangeable...

    No matter what the 'ruler'/scale, outliers at the very top readily identifiable... down below, lost in the crowd...

  54. Anon Says:

    @ Mountain,

    Man, you're really breaking out the calculator today :) All great info that you provided in your posts.

    But just to add to my suggestion of a longer study, I think perhaps a five-year timespan could provide a rough gauge on the consistency of these metrics, and how they correlate with each other. I say five because its probably the most practical given the accessibility in the resources for analysis...it might be able to include non-box score stats such as APM, depending on how far back the data necessary for its computation has been available. Then again, I don't know if the data for systems like WARP and Roland Rating have been available for that long either.

  55. Mountain Says:

    It would be interesting to see which regular season metrics are most consistent with playoff performance for the top guys or broader. Some insiders scorn the attempt to compute / refine very or all inclusive metrics but it still seems worth checking this.

  56. Zach Morris Says:

    Just to add to Haushinkaa2's original critique and Neil's (excellent) response, I believe the Roland Ratings are a composite of an adjusted plus/minus rating and a modified per rating(or at least some sort of box score based metric) because when Roland was creating the system the composite ratings passed the smell test better than either of the individual measures. So at least in regards to that specific metric Haushinkaa2's complaint may be valid.

    Of course the flip side of the Roland Ratings passing the smell test better than other systems is that the rankings seem better at first glance. Even players who are ranked higher in the Roland Rankings than the conventional wisdom would suggest (Nene, Odom, Thaddeus Young, Brendan Haywood etc.) make sense to people who follow the game closely and these players generally do well in the other metrics (it is a composite of per and adjusted +/- after all).

    So from a completely non intellectual perspective its hard for me to consider the Roland Rankings invalid or useless. Which might be Haushinkaa's original point.

    Hope I didn't steer the conversation back in the wrong direction.

  57. Ben Says:

    EWA seems like the best way to use PER for all NBA teams.

  58. merl Says:

    Why does Yao come first for APM and Dwight comes first for SPM yet neither player registers on the other stat? Makes you question plus/minus....?

  59. photographer Says:

    Hello there, I discovered your web site by means of Google at the same time as searching for a comparable topic, your site came up, it appears to be like great. I've bookmarked it in my google bookmarks.

  60. internapoli city Says:

    Thanks, I have recently been looking for info about this topic for a long time and yours is the best I've came upon so far. But, what in regards to the conclusion? Are you sure in regards to the supply?