Jumping ahead of the story

This was supposed to be a post about Steve Levitt's paper comparing poker and baseball. In a sense it still is, because I have a couple of reservations about the test Levitt did.

The reservation I have which I haven't tested is to compare the WSOP results to NBA results. I think that's an interesting test as teams tend to not change quality that much from year to year. I haven't found the information in an quick format to run calculations on. Once I do get them together, I will post the results.

The other reservation I had was with the population size that Levitt tested. For poker players, he identified 720 players as being high skill that participated in the 2010 WSOP. We know that the full population of WSOP players is at a minimum 7,319 players, because that is the number who played in the main event. So Levitt's test of poker players defines the population of high skill players as being < 10% of the population. But when he makes the same cut for baseball, he defines the equivalent high skill as making the playoffs. The problem with this is that he's defining high skill in baseball as being 26.67% of the population.

That difference is huge. I realize there are restraints given the number of teams in MLB, but it still strikes me as there has to be a better definition to be used. So I decided that a slightly more fair definition would be to only consider the four teams that made the LCS. This would set the cutoff at being 13.33%, which strikes me as being closer to equivalent for my liking.

So off I went to run the calculations. And to be honest, I was expecting to find that teams which made the LCS won 56% or 57% of games against teams that did not make the LCS, and I would write something pithy about cosnidering definitions when doing a test and Levitt might not have proven what he thought he did.

Then I saw the results - Teams that made the LCS won 54.46% of games against teams which did not make the LCS. Compare this to the playoff/non-playoff cutoff, which sees playoff teams win 55.72% of their games against non-playoff teams.

Quite honestly, I don't know what to make of this. There is something about teams that lose in the divisional round of the playoffs - LCS teams only win 44.07% of their games against teams that lost in the divisional round. But LCS teams fare better against teams that did not make the playoffs when compared to teams that lose in the divisional round.

I am definitely going to get more data on this - I want to know if this is just a three year blip on the radar, or if there's an actual trend in place. Also, I want to know exactly what is going on here - is there some sort of pattern that can be found regarding divisional round losers? Is using World Series competitors any different for determining skill vs. non skill? Is there something else I'm missing?

The bigger issue here is that I had this post practically written in my head, and then I went to get the numbers to back up my thoughts only to find that maybe assuming what was going to happen was a bad idea. So even if in getting into this data I find absolutely nothing conclusive, I will be able to take away something - it's always better to ask a question, gather your data, then answer the question than to ask a question, answer a question and then gather your data.