Top 8: Experts vs Idol Analytics

The Zap2It American Idol Exit Poll has published this week’s picks. Here they are alongside my model.

Why are the experts so confident about Paul? Nobody would lay any money on Haley? And they say it’s six times more likely that Paul goes than Stefano? This is weird. If somebody were to offer me 4:1 odds on Stefano, I’d be on that in a heartbeat. Of course, the people polled only get one pick, and they don’t collaborate. Still, that’s very certain. They are also notably sure that Jacob won’t go. I just don’t see the wisdom here. Again, 4 or 5 to 1 on Jacob, I’d take that bet.

Top 8 Forecast

Here’s a bit more detail than I previously printed. The name, gender is printed, followed by the preliminary approval rating from WNTS, the Dialidol score (also preliminary), the average of the 3 previous rounds WNTS scores, and finally the projected probability of going home.

       Contestant Sex WNTS.Rating Dialidol.score PriorAverage  Probability
1   Paul McDonald   M          20          11.83        51.66   0.14932005
3 Stefano Langone   M          58          12.09        36.66   0.12445903
7      Jacob Lusk   M          43          16.56        56.00   0.07198350
5    Casey Abrams   M          39          10.86        58.66   0.06979012
4 Scotty McCreery   M          45          22.93        58.33   0.06134782
6  Haley Reinhart   F          46          11.77        58.00   0.06083877
2   Lauren Alaina   F          63          14.32        69.66   0.02243664
8    James Durbin   M          71          13.98        74.00   0.01471944

Paul McDonald is forecast to go home, with Stefano as a close second. Dialidol predictions at this stage in the contest are pretty bad, and they don’t predict anything anyway—nearly all the scores are the same. Gender doesn’t factor basically at all. What does factor in is the average of the three previous rounds score (from WNTS) and the preliminary results for this night (shown above)

This was a dismal night, to be honest. While this season has been on average 4 points better than any other season, tonight was its nadir, and was below the finals average, historically speaking.

James Durbin should be safe (what Dialidol says notwithstanding). Jacob, Casey, Scotty, and Haley are all running almost even. If I had to guess, I would put my money on Haley to round out the bottom 3, but ultimately be safe on the strength of the previous two weeks, which were popular and kept her out of the bottom 3.

Top 8 Pre-game odds

For the first week, gender makeup no longer is a statistically significant factor. To good significance, the mean approval rating from the past 3 weeks is. The elimination probabilities are

Name Elimination Probability
Jacob 0.07
Haley 0.06
Casey 0.06
Lauren 0.03
James 0.02
Scotty 0.06
Stefano 0.20
Paul 0.08

The numbers are all low because the average prior scores for this group is much higher than previous seasons. This really is a very talented group. The probabilities don’t add up to 1 as a result.

Nevertheless, the model thinks Stefano is more than twice as likely as his nearest competitor, Paul, to go home. He needs to really kill it if he wants to stay in the game.

Top 9 pre-game odds

Let’s get right to it

Name Probability of elimination
Haley 0.218
Lauren 0.126
Pia 0.125
Paul 0.124
Stefano 0.097
Casey 0.093
Scotty 0.092
Jacob 0.079
James 0.052

Now that we’re in the Top 9, the odds of being eliminated as a man get marginally bigger. The problem here is that the data from previous seasons is lopsided towards women; therefore, I’ve based current projections only on seasons with 3 or 4 women only left in the Top 9. This week now incorporates average prior approval ratings into the mix, taking the average of the last three weeks.

Conventional wisdom is that Pia Toscano needs to step up her game, and I don’t see any reason why this is wrong. Despite having only 3 women, they are still at a huge disadvantage here, and her performances lately have been perceived as boring. Her song choice this week, River Deep – Mountain High, is in my opinion a terrible choice. It is a marginally well-known song, and has a limited but disastrous track record on the show. She runs the risk of being forgettable very easily; if you’re rooting for her, it could come down to performance order, so hope for the pimp spot.

Last week notwithstanding, Haley Reinhart is still in big trouble. She has to beat Paul and Stefano by about 10 points to not be considered likely to be in the bottom 3. Lauren, Pia, and Paul are all about equally likely to go, and so their performances will sort them out. The only people I consider bulletproof are Jacob Lusk and James Durbin, though Jacob is certainly singing against type this week, as it’s Rock and Roll and he is more of a soul/R&B guy.

Top 11 post-game redux: assessing the model

The model was somewhat vindicated last night after a disastrous first week. I made significant improvements to the way it considered the data, and it correctly predicted the bottom 3. As I said, Naima was a sure thing, with Paul and Thia in a dead-heat. In the end, Thia was eliminated. What’s interesting to me is just how far off Dialidol was:

The bottom 6 according to Dialidol

On the one hand, Dialidol got 2/3 of the bottom 3. However, it significantly overestimated Naima, which my model had by a factor of 4 the most likely to be eliminated. I also get no sense that Lauren was possibly at risk, but Dialidol ranked her as fourth worst. Does anybody really believe that?

The WNTS approval ratings were quite different: Continue reading