Reuben

Author's details

Name: Reuben Gann
Date registered: March 18, 2011

Latest posts

  1. Closing thoughts — May 17, 2013
  2. Top 2 prediction (updated) — May 15, 2013
  3. Prediction coming soon, but Candice is already way ahead — May 15, 2013
  4. How the model saw previous finales — May 15, 2013
  5. A brief introduction to statistical modeling — May 14, 2013

Most commented posts

  1. Jessica Sanchez vs. Phil Phillips — 24 comments
  2. Top 3 final prediction — 18 comments
  3. America loves White Guys With Guitars — 17 comments
  4. Too close to call — 12 comments
  5. Jessica may be closing the gap — 12 comments

Author's posts listings

May 17 2013

Closing thoughts

As I said at the beginning of the year, my sole aim this season was to build a predictive model worth a damn, just to see if it could be done. In the end, the model was 93% accurate in calling people safe (including winner Candice Glover tonight) and picked the person eliminated 78% of the time (89% of those eliminated were ranked either first or second most likely to go, FWIW). That’s pretty decent.

The site found an audience, as well, which I suppose I’m happy about. The site was the top Google search for “Idol predictions” as of yesterday, and about 6000 unique visitors showed up to see the (thankfully correct) call of Candice as winner. About 3000 people found their way to the site weekly, on average, during the finals.

S12Sitestats

I would note that if you wrote a message to me in the comments, I likely did not see it, since reading comments on the interwebz makes me at least 25% less happy (often more). Not being rude. Jessica does read them, but if you want to drop me a line, my Twitter is the best way.

It’s worth asking, was this a good season of American Idol? The best singer did win, and the Top 3 were good singers. This was most definitely not the case in the past 2 years. There were a few moments that stand out in my mind: Janelle doing “I Will”, Candice in several of her outings (“Straight Up”, “Don’t Make Me Over”, and “I (Who Have Nothing) last night) all gave me chills. I liked Nikki Minaj as a judge a lot, and am very disappointed that she will not be staying on. Angie started pretty strong for me, though she clearly began to drop off. So, yes, it was a pretty good year.

There’s been a ton of digital ink spilled about what Idol needs to do to revive its ratings, and what I’ve read of it didn’t seem right at all. The bottom line is that that the show existed for many years as the only real game in town. Sure, there were little shows like Nashville Star, but nothing substantial. But now there’s The Voice. There’s X Factor. Why wouldn’t we think the ratings would go down? So, anyway, if you want my dumb opinion, to go along with the rest of the internet’s dumb opinions, it’s this: if you want ratings back, stop cannibalizing yourself, cancel X Factor, and get Simon Cowell back on the show. Anything short of that, such as firing Nigel, changing the themes around, reducing the auditions rounds, is rearranging deck chairs on the Titanic.

Since I’m not invested in (and, in reality, have never seen) any of the other shows that sites like this cover after Idol ends, I do not maintain this site between seasons, so if you are a regular reader, sorry. I do maintain my personal blog IdleAnalytics, all year, and I tweet a fair amount, should you want to follow such things. Jessica can be found on Twitter as well as her Tumblr.

Thanks for reading.

May 15 2013

Top 2 prediction (updated)

Name Song WNTS
(avg.)
DialIdol VoteFair Not-safe
Probability
Kree Harrison Angel /
All Cried Out /
Up To The Mountain
59.667 9.287 42 0.758
Candice Glover Chasing Pavements /
I Am Beautiful /
I (Who Have Nothing)
64 14.735 58 0.242

Updated 10:54 AM EST. Candice has expanded her lead on Dialidol, and is now called by that service as well, outside the margin of error. Dialidol tends to be cautious when it calls a contest, so this is looking better for Candice.

Important!

The methodology for the finals model is described here. The model is 87% accurate on ranking within a margin of error of +/- 3%. Probabilities being what they are, somebody with a not-safe probability of just 0.25 will be in the bottom 3 one out of four times. Please do not comment that the numbers are wrong. They are probabilities, not certainties or even claims. Do not gamble based on these numbers.

The model predicts Candice Glover will win season 12 of American Idol.

Candice is currently sweeping all major indices, and while the numbers may flit a little this way and that, unless Kree can actually switch one of them, she is unlikely to win. Mostly, though, this comes down to Dialidol, and that measurement shows Candice winning by a large margin. Without some kind of west coast surge, there’s no way I can see to do it.

Of course, Kree could win, but it would be a major upset. Candice is about 3 times likelier to take the title, at least as far as the model can tell. Note that until the model is wrong about a finale, there is some uncertainty as to its ratings (as perverse as that may sound).

You can see how the model rated past finale rounds to see if they match your gut about how those years ran. To me they fit pretty good. And based on what I saw, and based on the huge amount of high-rated performances (as rated by WNTS) that Candice has had, she looks more like a winner than Kree does (from a numerical point of view).

It’s not in the bag, but it looks good for Candice fans.


Reuben is the statistician and lead writer at IdolAnalytics. Follow him on Twitter. His personal site (run all year round) is IdleAnalytics
.

May 15 2013

Prediction coming soon, but Candice is already way ahead

The official model call will be posted as soon as WNTS ratings are posted, but those don’t matter all that much at this stage. Candice is handily beating Kree on both Dialidol and Votefair, and that by itself is enough to put her at a 75% chance to win, versus only 25% for Kree. If WNTS ratings come in like I think they will, it will be a clean sweep. Check back for the official numbers after Dialidol equilibrates a little, but I have Candice at about 1:3 odds at present.

May 15 2013

How the model saw previous finales

Just as before, here is some context for the prediction that will be published later tonight. Here’s how the model rated previous finales.

Season 5

s5finaleconts

Read the rest of this entry »

May 14 2013

A brief introduction to statistical modeling

Here I talk a lot about “the model” as if it’s a living, breathing thing. But there isn’t anything mysterious about a model, it’s a totally understandable thing based on math, data, and a bunch of assumptions.

When you think about a “model” used in regular life, it’s something like a scale model of a building. Why is such a thing useful? After all, the model of a building isn’t functional in any of the same ways as the building itself. Nobody can live in it, go inside it. The model is useful, though, to a group of architects trying to lay out a neighborhood, or to a plumber deciding where to run his pipes. That is, a model is purposeful: it only has to be like the real system in a way that you get the information you need.

When you build a numerical model, you’re building a description of something. Consider the first thing most physics students learn: the path a ball takes when you toss it into the air.

Read the rest of this entry »

May 13 2013

Programming note

Just a quick note to people who follow the site—we had to make some big technical changes last week because the site was getting hammered (the site actually went down on Thursday for about an hour … sorry about that). This included removing the Liveblog feature and the Polls to allow for supercaching. My crack technical team (Jessica) may get either or both working again by Wednesday, but if she doesn’t, that’s why they aren’t there.

May 13 2013

Season 12 model accuracy assessment roundup

As we head into the finale, it’s time to look at how predictable the season has been, particularly with regard to the three prediction models on this site. The first is the semi-finals model, predicting who would be in the finals. The second is the finals model, a week-to-week guess as to who would be in the bottom group, safe, and/or eliminated. Finally, the Top 3 Tracker, which tried to guess who the Top 3 would be.

Read the rest of this entry »

May 09 2013

Top 3 post-game

Sometimes models go against your gut because they’re wrong, but sometimes the reverse is true. As hard as it was to believe, the data pointed to Angie’s departure. Though she was hugely popular on Votefair, that index isn’t nearly as reliable as Dialidol in this round, which is why Dialidol factors heavily into the model’s predictions in the Top 3.

The model’s ranking accuracy in the Top 3 rises to 75% (6 out of 8 correct). As I said in the Top 3 Tracker methodology, it’s quite hard to predict what happens in the Top 3 early on, even though it’s pretty simple to determine who the Top 3 will be. All year the Top 3 Tracker had Angie, Candice, and Kree as the most likely Top 3. Angie always appeared on top.

The closest historical analog to this situation was Danny Gokey. Like Angie, he appeared to be headed for the finale in the early rounds. But his support suddenly flagged in the Top 3, and out he went. Angie’s Dialidol numbers had been low for weeks, indicating that while she may be popular, she didn’t draw rabid voting in the way Kree and Candice did.

However, if you want to credit one thing for sinking Angie, it was perhaps the judges and the documentary-length home visit clips of the week. Kree’s was heart rending and Candice’s uplifting. The judges lavished praise on Candice (not that she necessarily didn’t deserved it) in a way that they did not for Angie.

I’ll have an overview of the model’s accuracy this year coming within the week, but it’s been a pretty decent run.

May 09 2013

Top 3 prediction

Name Song WNTS
(avg.)
DialIdol VoteFair Not-safe
Probability
Angie Miller Sorry Seems to be the Hardest Word/
Try/Maybe
57.333 3.799 45 0.382
Candice Glover One/Next To Me/
Somewhere
64.333 6.581 34 0.309
Kree Harrison Perfect/
Here Comes Goodbye/
Better Dig Two
49 8.664 20 0.309

Updated 11:35 AM EST. No change in ranking.

Update: NB: If you read the post below you’ll see that this year will be the second closest Top 3 in the model’s history. The only one more competitive was Elliott Yamin vs Katharine McPhee. Quite close contest this year.

Important!

The methodology for the finals model is described here. The model is 87% accurate on ranking within a margin of error of +/- 3%. Probabilities being what they are, somebody with a not-safe probability of just 0.25 will be in the bottom 3 one out of four times. Please do not comment that the numbers are wrong. They are probabilities, not certainties or even claims. Do not gamble based on these numbers.

Names in green are predicted safe. Names in red are the most likely bottom 3. Names in yellow are undecided. Any person not in green is not considered safe by the model. The most probable eliminated contestant is Angie, though none of these would be shocking.

DialIdol had a miserable failure last week, which is unusual. Typically, that index becomes more accurate as time goes by, and by the Top 4 one shouldn’t take its pronouncement lightly. Given that, Angie seems to be in trouble. As to this (official) projection, Angie had a poor Dialidol score and was in the middle on WNTS, and is thus the most likely to be eliminated based on historical factors. I would note that these numbers are fairly close to one another, and so this represents something close to a three-way tie.

However, if you are skeptical as to this call (as am I), then perhaps you will be more interested in the following “Dialidol is full of crap” projection:

Name Song WNTS
(avg.)
DialIdol VoteFair Not-safe
Probability
Kree Harrison Perfect/
Here Comes Goodbye/
Better Dig Two
49 8.664 21 0.504
Candice Glover One/Next To Me/Somewhere 64.333 6.581 32 0.313
Angie Miller Sorry Seems to be the Hardest Word/
Try/Maybe
57.333 3.799 47 0.183

This is obviously not the official model call, but it may match your gut more than the official one does, as it certainly does mine. The Top 3 Tracker had Kree in third place all year with the exception of one week. Given the accuracy of the T3T, I tend to think Kree is heading home.

Nonetheless, the official call is Angie as of this writing.


Reuben is the statistician and lead writer at IdolAnalytics. Follow him on Twitter. His personal site (run all year round) is IdleAnalytics
.

May 08 2013

How the model saw past Top 3 rounds

Tomorrow night I’ll post a forecast on the Top 3, and you may want to know how the model that the forecast is based on “saw” the past Top 3 rounds. Indeed, the model was potentially misleading in this year’s Top 4 (though the Top 3 Tracker was pretty good all year) by seeing Candice as the most likely. Of course, for all we know she was second-to-last, but it is peculiar that the person it saw as the most likely was safe and the person it saw as least likely was eliminated. In any case, take this historical information and make of the model’s prediction what you like. Read the rest of this entry »

Older posts «