Since I’ve started building a forecast model for Idol, I’ve been terribly interested in Dialidol, a service that measures votes and busy signals during American Idol voting, and gives a raw score. Their main touted statistic is that they have never flubbed a prediction of the finale: since Season 4 (when they started), they have correctly predicted the winner of the contest every time.

However, if dialidol was a perfect forecast, there would never be a need for anything else, and it’s actually quite far from a perfect forecast. I can think of lots of reasons for this, but the main problem with dialidol is, of course, that it’s got sampling problems. It only measures a small number of people using the software to register votes by calling on a land line. Cell phone calls, texts, and now internet votes, are never counted. Moreover, the fewer people watch Idol (and its ratings have been dropping), the worse that sampling gets due to simply numbers.

The first, and by far the most important question, is how often does Dialidol accurately forecast the person going home: Read the rest of this entry »