How the model saw past Top 3 rounds

Tomorrow night I’ll post a forecast on the Top 3, and you may want to know how the model that the forecast is based on “saw” the past Top 3 rounds. Indeed, the model was potentially misleading in this year’s Top 4 (though the Top 3 Tracker was pretty good all year) by seeing Candice as the most likely. Of course, for all we know she was second-to-last, but it is peculiar that the person it saw as the most likely was safe and the person it saw as least likely was eliminated. In any case, take this historical information and make of the model’s prediction what you like.

Season 5

S5ContestantsSeason 5 was a three way tie. The vote was 33.68% versus 33.26% versus 33.06%. Elliott Yamin had the lowest, and was eliminated, while Taylor Hicks (presumably, but not necessarily, first) and Katharine McPhee advanced to the finals. The model assigned the following not-safe probabilities:

Taylor Hicks 17%
Katharine McPhee 40%
Elliott Yamin 43%

It was right in ranking Yamin as most likely, but it was overly confident of Taylor Hicks. Then again, winning by a small margin still counts as winning, so perhaps it wasn’t.

Season 6


Season 6’s top 3 was Jordin Sparks, Blake Lewis, and Melinda Doolittle. Though I don’t believe we were told how the voting went, the result was Doolittle going home. The assigned probabilities by the model were

Jordin Sparks 34%
Blake Lewis 50%
Melinda Doolittle 17%

Absolutely shocking result, no way to disguise it, and the ranking was quite wrong. However, such a result should eventually happen if there really is a 17% chance. That’s why the ranking is not the only question. Nonetheless, if you see such a prediction here tomorrow, let this be a reminder that unexpected things do happen.

Season 7

s7contestantsBetween David Cook, David Archuleta, and Syesha Mercado, the call was not hard. Syesha was a shocking inclusion in the Top 3, who had multiple times been in the bottom 3. The model’s assigned probabilities were

David Cook 21%
David Archuleta 28%
Syesha Mercado 51%

Mercado was almost twice as likely to go home as the other two, and she did indeed go home that night.

Season 8

s8contestantsThe all-male season 8 had Danny Gokey, Kris Allen, and Adam Lambert. None was the clear winner going into this round, though Gokey had never been in the bottom 3. Gokey ended up going home, in what I can only guess was a close vote. The assigned probabilities were

Kris Allen 31%
Adam Lambert 17%
Danny Gokey 52%

Again, Gokey was rated by the model as the most likely by a large margin.

Season 9


Season 9’s top 3 was Casey James (a bit of a dark horse for the Top 3), Crystal Bowersox, and Lee DeWyze. Bowersox was the favorite as far as I can tell, though DeWyze took the title. Again, no voting info was given, but it was James who was sent home. The probabilities were

Lee DeWyze 17%
Crystal Bowersox 27%
Casey James 57%

Again, the model had no problem picking the lowest vote-getter. James was more than a factor of 2 above his nearest competitor.

Season 10


The country year featured Scotty McCreery (the odds on favorite from the beginning) Lauren Alaina and Haley Reinhart. The women had both found themselves in the bottom group (Haley several times). The probabilities were

Scotty McCreery 24%
Lauren Alaina 49%
Haley Reinhart 31%

The model thought Haley was more likely to make it, but she was eliminated. The margin was not huge, but neither was it very small. Clear ranking error (though, again, 31% is not a small chance).

Season 11


Last year had another dark horse, Joshua Ledet, in the Top 3, alongside Jessica Sanchez (who had to be saved) and eventual winner Phil Phillips. Phil was obviously going through. Between Joshua and Jessica, I would have flipped a coin. Here’s the model’s impression

Phil Phillips 18%
Jessica Sanchez 32%
Joshua Ledet 50%

It was Joshua who went home, which was in line with the model’s ranking. Again, I don’t know that Jessica had a huge lead, but at 18 points that was the most likely outcome


Season Correct ranking Margin Assessment
5 Yes +3 Lucky guess
6 No -33 Huge miss
7 Yes +23 Easy call
8 Yes +21 Easy call
9 Yes +30 Easy call
10 No -18 Medium miss
11 Yes +18 Easy call

5 of 7 calls were correct in the Top 3, for a 71% accuracy. It’s nothing to write home about, but it’s not bad either. At least one of those (season 5) was dumb luck.

Now, I’ve said it before but it bears repeating: none of these was a prediction. Tomorrow will be the very first Top 3 prediction this model has made, and all of the above results have been incorporated into the model’s fitting parameters. However, the model is usually right in the Top 3.

Bookmark the permalink.

Comments are closed.

Comments are closed