16 Responses to “Which Remaining Team Plays Its Best Lineups the Most?”

1. Jason J Says:
   May 26th, 2010 at 1:17 pm

   So if Rondo-Ray-Pierce-Garnett-Perkins is great, and Rondo-Ray-Pierce-Garnett-Wallace is terrible... ... ... hmmm...

2. Neil Paine Says:
   May 26th, 2010 at 1:34 pm

   Yeah... Sheed.

3. P Middy Says:
   May 26th, 2010 at 2:32 pm

   By terrible, you must mean Sheedtastic.

4. Adam Says:
   May 26th, 2010 at 8:44 pm

   Sheed!

5. mookie Says:
   May 27th, 2010 at 10:08 am

   Most interesting thing I've read all week.

6. Mike in Denton Says:
   May 27th, 2010 at 12:26 pm

   Yeah, BUT... these numbers can't tell you how a certain lineup will do against a particular opposing lineup. For example, maybe a certain lineup has a great net rating because the players on that lineup only play against the scrubs of the other team, and they do really well. So you decide, "heck, I'm playing my bench against the other guys' starters because of their net rating!" Then your guys get creamed and you wonder what happened.

   Matchups still matter. Even though Shaq is a liability and his lineups for Cleveland in the playoffs would probably score poorly on this scale, did you see him in the Chicago series draw three consecutive fouls on the undersized Miller and Noah without even taking a shot? That's a matchup problem.

   The strategy of the coaches' to play who they play against other teams' lineups is the X-factor which limits the value of these statistics. I think they are still interesting, but not to be taken without a lump of salt.

7. Neil Paine Says:
   May 27th, 2010 at 12:43 pm

   Adjusted lineup +/- takes into account the strength of the opposing lineup (i.e., whether it's starters or bench players), so that criticism isn't really valid. But I agree that specific matchups matter, just not as much as most people think.

8. Mike in Denton Says:
   May 27th, 2010 at 1:29 pm

   I don't want to misunderstand you, but I think the criticism is still valid. The source material (BasketballValue.com and Rosenbaum's article) describes the approach as using regression to statistically simulate how the player did against the competition they played against with the teammates they played with. So it's using tracked statistical information as its source data. While the adjusted plus/minus is definitely (in my opinion) better than raw plus/minus, I wouldn't say it definitively "takes into account the strength of the opposing lineup", although it does do a better job than other attempts I've seen used.

   So to just throw out a wild example, suppose Phoenix' hypothetical D lineup has never played against the Lakers' hypothetical B lineup. Then there is no data involving this particular matchup in the results. The data is being simulated through estimations. How well do statistical simulations emulate real world examples? To my knowledge nobody has yet found the means to ensure a good fit a priori -- we say it's good if the results are close, and say we messed up if they are way off.

   Maybe a better way to describe the adjusted plus/minus would be to say that it "adjusts for level of competition and level of teammates". How well does it do that? I'd be interested to see a comparison of the Laker bench lineups vs. the Sun bench lineups in the first two games of the playoffs and the regular season. According to my cursory inspection of the stat tables, the Suns bench should have killed the Laker bench, but the opposite happened in the first two games. Then, it seems to have flipped in the last two series as the Suns' bench killed the Laker bench. And it seems like the differences have been by wide margin, not just a little bit.

   I think the best way to use these stats would be to compare lineups within a team, assuming the coaches are taking into account the matchups, to find out which groups are doing well and which ones were not, just in case the coaching staff was having delusions regarding certain lineups.

   We're probably on the same page though regarding specific matchups. I'm just trying to warn against playing certain lineups just for the net rating regardless of the competition, and you're trying to steer me away from disregarding them completely. I do believe this data says something, I'm just not exactly sure what yet. :)

9. AHL Says:
   May 27th, 2010 at 1:40 pm

   I don't think I like your assessment of the Suns using Grant Hill + Jared Dudley against the Lakers. That defensive combo worked great against other teams, but because of the size of Odom/Gasol/Bynum, it's tough enough just having one of those guys out there. One guy was always getting wasted guarding Artest or something unnecessarily.

   In fact, in Game 4's zone extravaganza, those two were never on the floor together. Same (minus garbage time) with Game 3's win. I think the Last 10 Games link you posted only had the San Antonio and Portland series data skewing the results, because aside from some garbage time in Game 2 that combo hasn't really proved anything against the Lakers this postseason. More important is having two of Amare/Lopez/Frye/Amundson on the floor at all times.

   (this post brought to you with the use of http://www.popcornmachine.net/).

10. Neil Paine Says:
    May 27th, 2010 at 1:50 pm

    There is a significant standard error attached to certain lineups if the sample on which the regression was run was not large, that's for certain. So a low-minute lineup will have a larger range of possible "true" skill levels than a high-minute lineup; we're more confident about the true ability of the Celtics starting 5 than the Suns' bench, for instance.

    But I think you may be misinterpreting what I mean when I say it takes into account the opposing lineup strength -- I mean that the raw net +/- of a lineup is autmomatically adjusted based on the opposing lineup during each of its "shifts", so a lineup that faced nothing but bench players would not have an APM as good as its net +/- would have you believe. In other words, the level of competition adjustment is built into the regression.

    Even if two lineups have never faced, we can predict their performance against each other based on how they've each performed vs. common opponents -- it's essentially like those college football rating systems that determine the relative strengths of Alabama and USC even though they've never faced each other, because they have common opponents, or their opponents have common opponents, etc. Every unit is connected to every other unit somehow; the units that are more well-connected have smaller standard errors because we're more confident about their "true" skill level.

11. Mike in Denton Says:
    May 27th, 2010 at 2:37 pm

    Thanks, Neil. Has anyone actually done that kind of prediction yet? I mean, take two lineups that have never faced (or even have faced), predict what the results will be beforehand, then analyzed the results? I'm not seeing anything like that in the articles linked, but I haven't looked exhaustively.

12. Neil Paine Says:
    May 27th, 2010 at 2:48 pm

    Joe Sill actually presented a paper on the topic at the MIT Conference a few months ago, where he made some big breakthroughs to out-of-sample predictive validity by regularizing APM via a technique called ridge regression. Here's his site:

    http://www.hoopnumbers.com/allAnalysisView?analysis=RAPM&discussion=True

13. Neil Paine Says:
    May 27th, 2010 at 2:49 pm

    Here's the paper:

    http://sloansportsconference.com/uploadfiles/pdf/joeSillSloanSportsPaperWithLogo.pdf

14. Mike in Denton Says:
    May 27th, 2010 at 4:47 pm

    Interesting. I see that Joe's R-squared result from his test was around 16% which almost doubles those of standard linear regression, and is worthy of publication. Still, the results seem to indicate there are a lot of unaccounted-for factors that contribute to the performance, would you agree?

    I guess I'm still of the mindset that while these stats are definitely interesting, their best use is not to determine what lineup should be played regardless of the opposing lineup. I think the matchups plays a bigger part than what is showing up in the stat tables.

15. Neil Paine Says:
    May 27th, 2010 at 5:16 pm

    Well, that would be true if the APM metric was perfectly capturing true skill and you still had a low r-squared value. As it stands now, that value is not only saying "you can't predict future results from +/- alone", but also "+/- is not capturing true skill enough right now". The error bars on individual +/- scores are very large -- we can only say with 68% confidence that Dwight Howard is making the Magic better by between 9 and 25 points per 100 possessions. So Howard could conceivably be the best player ever, or the 5th best center in the league. And he's a high-minute player... imagine the ranges on bench players!

    However, this discussion isn't really relevant to lineups, because 5-man units have a much lower standard error in their +/- estimates. Joe Sill's work was with predicting out of sample 5-man performance from individual +/- ratings, but the 5-man scores I listed above derive from data on that lineup as a unit, indivisible into smaller parts. Which is why right now, you can say with a lot more confidence that "Lineup X is better than Lineup Y" than "Player X is better than Player Y" using +/-.

16. Joseph Says:
    August 17th, 2010 at 6:51 pm

    This is very interesting. I was very interested in the Suns bench because of their performance this season. This shouldn't be surprising since many of the commentators were branding the bench as the best in the league, saying, "they've outperformed the starters plenty of times" on numerous occasions.

    I do think the opposing lineup matters. While the Phoenix bench may be better than the starters, at times, I don't think many benches in the NBA are ever better than the Suns. If the Suns bench is playing a weak bench, they're going to blow them away.

    It'll be interesting to see if we see Dragic and Nash on the floor at the same time more often next season.

    By the way, love the site and the work you've put into i.