Thursday, September 18, 2008

The Arbitrarian: Operationalizing interestingness

David Sparks is the Arbitrarian. His stats column runs weekly here on Hardwood Paroxysm.

Which teams were the most interesting last year, and which might be the most interesting this coming year? Obviously, there are innumerable ways to conceptualize "interestingness" in basketball--amount of interpersonal drama, perhaps, or exciting style of play--but today I'm going to present a series of different takes on interestingness, and apply these measures to the NBA. There's no way we can cover all possible understandings of what makes a basketball team interesting, but hopefully I can offer several reasonable-sounding operationalizations...

Predicting game outcomes

First, since several of the following definitions of interestingness rely on it, I want to define a very simplistic and abstracted way of predicting game outcomes. First, imagine that you have two coins, each of which has a 50% chance of landing on heads, and a 50% chance of landing on tails. What are the odds of the various possible outcomes from flipping both simultaneously?

This is pretty straightforward, where we have two coins, A and B, and each can land Heads up or Tails up:

p(A=H&B=H) = 0.5*0.5 = 0.25
p(A=H&B=T) = 0.5*0.5 = 0.25
p(A=T&B=H) = 0.5*0.5 = 0.25
p(A=T&B=T) = 0.5*0.5 = 0.25

So, each H/T combination of two-coin outcomes has an equal chance of occurring, 25%.

What if A has a 60% chance of landing on Heads, and B has a 40% chance? Probabilities are somewhat different:

p(H,H) = 0.6*0.4 = 0.24
p(H,T) = 0.6*0.6 = 0.36
p(T,H) = 0.4*0.4 = 0.16
p(T,T) = 0.4*0.6 = 0.24

Here the probabilities are not all the same. Now, imagine that we say that a coin landing on Heads is the winner and a coin landing on Tails is the loser, and if they both land with the same face up, the two coins tie. Using the 60/40 probabilities above, we know that there is a 36% chance that A wins, a 16% chance that B wins, and a 48% chance that they tie. (Make sure this makes sense to you.)

Now, imagine that we disregard the ties (i.e. if they tie, we make them re-flip until there is no tie). What is the probability that A wins? Just take the probability from above (36%) and divide it by the universe of allowed outcomes (p(A wins) + p(B wins) = (36% + 16%) = 52%). You find 0.36/0.52 = 0.6923077. In other words, if you pair a 60% winning coin against a 40% winning coin, there is about a 69% chance that A lands on heads and B lands on tails, if you re-flip any ties.

How does this apply to basketball? Well, take Denver and Chicago from the 07-08 season. Denver's winning percentage was about 60% and Chicago's was about 40%. Assume that their winning percent is analogous to the weights assigned the coins above, that is, it's the odds that they'll win on any given night. Just as above, if we know that on one particular night one of these two teams won and the other lost, the chance that it was Denver that did the winning is about 69%, and the odds that it was Chicago who won are about 31%.

Now let's pretend that a head-to-head game is just like the coin tossing face-off above--that is, it doesn't matter how well the two teams match up, or who's injured, or who's on a hot streak, or who's defense will smother the other team's best scorer, or what have you. Imagine if, on game night, the owners of the two teams met at center court and flipped coins weighted according to their season-long winning percentage, and re-flipped ties... that's how I would predict the chances of each team winning.

Now, in the Denver v. Chicago case, both teams have a chance of winning (69% and 31%, respectively), but Denver is more likely to win, because their odds are greater than 50%. So, if you had to predict a single game, you'd go with Denver. However, if the teams met 1,000 times, you would expect Denver to win roughly 690 times, and Chicago to win about 310 times.

And that's how I'll be constructing win probabilities for the remainder of the post. As long as neither team is undefeated or has literally no wins, there is always at least a slim chance that the underdog team can win. Applying this algorithm to a Boston v. Miami game in the 07-08 season, you'd get a 94.852% chance of Boston winning, meaning that it's highly unlikely, but not impossible for Miami to have won that matchup.

Interestingness as unpredictability

If we know each team's winning percentage, we can come up with predictions of a winner for each game in a season. If, for each game, we call the team with a greater than 50% chance of winning (this is, essentially, the team with the better win%) the predicted winner, when comparing these predictions to actual 2007-08 regular season outcomes, we find that we predict correctly 69.7% of the time, which is, at least, better than half, and probably better than many of ESPN's experts.¹

One possible definition of interestingness is unpredictability--that is, if we know the outcome of the game before hand, the game itself is likely to be less interesting (this is why people who miss the live broadcast, but plan to watch it later on TiVo, don't want to be told the score). So, which teams were the most and least predictable?

UTA2008 0.610
SAC2008 0.622
ATL2008 0.634
DAL2008 0.634
IND2008 0.634
NJN2008 0.634
PHI2008 0.646
WAS2008 0.659
CLE2008 0.671
POR2008 0.671
CHA2008 0.683
CHI2008 0.683
HOU2008 0.683
DEN2008 0.695
GSW2008 0.695
MIL2008 0.695
NYK2008 0.695
ORL2008 0.695
MIN2008 0.707
NOH2008 0.707
LAL2008 0.720
DET2008 0.732
LAC2008 0.732
SAS2008 0.732
TOR2008 0.732
PHO2008 0.744
SEA2008 0.756
MEM2008 0.780
BOS2008 0.805
MIA2008 0.817

Note that Miami, having the worst record in the league, would have been predicted to win none of its games, while Boston was predicted to win all of its games, because it always had the better record in its matchups. Miami game outcomes were correctly predicted 81.7% of the time, which is (1-win%), and Boston outcomes were correctly predicted 80.5% of the time, which was their winning percentage. In general, it's easier to predict teams with more extreme winning percentages, because they are more (or less) likely to face teams with worse records. Close-to-.500 teams are the hardest to predict correctly (in general). Thus, it is instructive to contrast predictability with record, as I do in the graph below:

The huge outlier is Toronto, oddly enough. Despite their exactly 0.500 record, Toronto's game outcomes were correctly predictable almost 3/4 of the time--surprisingly high. The biggest outlier at the other end of the spectrum is Utah, who despite having one of the best records in the league (which should lead to ease of prediction), were the least easy to correctly pick.

Interestingness as upsets

Unpredictability just means that a team lost games it "should have" won, and won games it shouldn't have. However, for a fan, losing games that should be won adds more to angst than interest. Which teams did the best relative to their opposition--making their fans happy by winning games they were predicted to win, and upsetting opponents who should have beaten them?

To determine this, for each game played by each team, I estimate the team's probability of winning using the above methodology. Then, depending on the actual outcome, I assign a binary 0/1 value for that game if they lost/won. To compute an "upset factor," I subtract predicted probability of winning from the binary lost/won variable.

Thus, if a team has a 72% chance of winning (it is substantially better than its opponent), and wins, the upset factor for that game is (1-0.72) = 0.28. Had they lost, the upset factor would be (0 - 0.72) = -0.72. A team with very little chance of defeating it's opponent, say 6% (like Miami's odds against Boston), would get 0.94 if they won, but just -0.06 if they lost. Thus teams are rewarded for winning (and punished for losing), but proportionately to their projected odds of winning.

Over the course of the season, the best teams will beat most of their opponents, and so should generally have positive cumulative upset factor sums. The worst teams will lose more often, and so should generally have negative cumulative upset factors. However, some teams will defy their probabilities, and outperform (or underperform) expectations, and thus a bad team which manages to be an occasional "Giant Killer" may have a season-sum upset index that defies its record. How does this look for 07-08?

IND2008 -1.481
NYK2008 -1.389
MIL2008 -1.351
ATL2008 -1.343
MIA2008 -1.207
CHA2008 -1.116
NJN2008 -1.030
CHI2008 -0.897
MEM2008 -0.786
MIN2008 -0.734
TOR2008 -0.610
SEA2008 -0.554
WAS2008 -0.495
PHI2008 -0.429
LAC2008 -0.379
ORL2008 -0.213
BOS2008 -0.005
DET2008 0.000
CLE2008 0.035
SAC2008 0.410
POR2008 0.902
DEN2008 0.903
GSW2008 0.964
UTA2008 1.293
DAL2008 1.375
PHO2008 1.464
HOU2008 1.551
NOH2008 1.615
LAL2008 1.654
SAS2008 1.851

As you can see, the "most upsetting" teams are some of the league's best, which in this context means that they beat teams they were expected to, and did not lose much to teams they should have beaten. In this respect, the Celtics are at a disadvantage, since based on their record, they should not have lost to anyone, and so every loss counts heavily against them.

One possible interpretation (and I stress "possible") of these numbers is that the Spurs actually played 1.851 wins better than their record of 56 wins would indicate, given their opposition. The Celtics' and Pistons' actual records fairly accurately capture their ability given their opposition, and the Pacers' 36 and 46 record is actually about a game-and-a-half too good, given how they lost to teams they should have defeated, and failed to upset many better teams.

Interestingness as potential

One final means of defining interest as we head into the 08-09 season, is potential. Every player, at any given time, has a certain level of productivity, and this level of productivity varies in generally predictable ways: usually it takes several years in the league to climb to peak productivity, which is maintained for several more years, before a decline sets in. Typically, players are their most productive in the middle of their careers--rarely do they peak in their rookie year, and even more rarely do they leave the league at the top of their game.

It is possible, then, to think of a player's potential as their current productivity, given their age or experience in the league. Extremely valuable players, if they are very young, have more "potential" than extremely valuable players in their late-20s. This is part of the reason there is always so much excitement about rookies and rising stars--any amount of success they find early on is likely only to increase as they come into their prime.

At the other end of the spectrum, players in the middle of their careers, who have still not managed to become highly valuable, have very little potential. Of course, all of this varies. Tim Duncan, even at this relatively late age, is still likely to be valuable in the near future, even if he doesn't necessarily improve. However, a General Manager might be more inclined to sign Chris Paul to a long-term contract than Jason Kidd, even if the two had been equally productive last year--Paul just has more potential, given the success he has found, and given his age.

Thus, we can estimate, for every player, some index of potential, essentially by dividing value (measured in MVP) by age. (Technically, I divide MVP/age at the per-game level, and multiply by the minutes-weighted mean age in the league (nearly 27), and then multiply this by 82, to estimate the trend of that player's value.) When applied to the 07-08 season, we find the following estimates of potential:

(I've also thrown in the top-500 best-potential seasons from my dataset, which only includes 1986-2008, and so misses out on some really excellent rookie seasons. Apparently LeBron has lots of potential.)

Now, incorporating all the offseason moves, and using a magical formula that lets me convert MVP to team wins (Pythagorean 5.25), here are my projections (based only on this estimate of potential) for team success (in wins) at some future time:

MEM 13.8
SAC 21.7
NJN 23.1
DEN 23.6
PHO 27.7
GSW 28.0
MIL 28.6
OKC 28.9
CHI 31.6
MIA 33.3
TOR 33.5
POR 35.1
SAS 35.9
NYK 38.0
LAC 38.2
ORL 38.4
DET 38.8
CHA 46.7
BOS 48.0
ATL 48.7
MIN 50.7
WAS 52.2
CLE 53.8
PHI 56.0
IND 56.6
UTA 57.1
NOH 58.5
DAL 58.5
HOU 61.8
LAL 63.1

Notice all the hedging I did--it's unclear whether these estimates should apply to next season, or several seasons down the road. I doubt, for example, that Phoenix and San Antonio will fall so far in 08-09, but you could imagine that, playing with these same rosters four or five years from now, the then-senior citizens on those teams would not fare so well. Also note that this doesn't include anyone with no NBA stats--meaning that I haven't incorporated the doubtless boon brought by Oden, Rose, Beasley, et al. That said, I can see the Lakers, Hornets, Rockets, Jazz, and 76ers being very interesting in the near future, and so this may not be all crazy.


I'd be very interested to hear if you like these conceptualizations and measures of interestingness, and especially if you think the measure of Potential has any merit at all. How would you measure interesting, if you had to use statistics? Does your impression of teams on the rise and teams on the decline mesh with the team success projections listed above? Let me know in the comments.

¹ Keep in mind that this prediction methodology is extremely simplified. It doesn't take home court advantage into account, nor any interaction effects between the two teams. Obviously, adding in both of these would make the model more accurate, but if I had the time and ability to predict outcomes perfectly, I wouldn't be sharing that knowledge with you, I would be gambling. So, please accept this approximation for the abstraction that it is.

Add to Technorati Favorites