Effect of auditions exposure on American Idol, part 3

This is part 3 of 3 of the series. Part 1 describes how the data was collected and how total screen time influences how far a given contestant gets in the contest. Part 2 looks at how initial audition affects the contest. Part 3 looks at how pre-exposure influences voting in the semi-finals.

The semi-final rounds

Once the contestants from the general auditions are winnowed down into 30-40 people, the Top 10/12/13 are chosen by a sometimes-brutal “semi-final” round or a series of such rounds. (I put scare quotes because I don’t get why it’s named this. There is no “regular season”, no quarterfinals.) The rules of this have varied wildly:

Season # of semifinal
1 1 The top 30 were divided into 3 groups of 10, one performing each week for three consecutive weeks. 3 people from each round were chosen by the viewing audience votes. Of those eliminated, 5 were chosen to compete in a wildcard round, where the judges chose 1 to round out the top 10.
2 1 4 groups of 8 people each performed over 4 consecutive weeks, and 2 were chosen each week by the viewing audience votes. A wildcard round with 9 people (not all of whom performed in the first 4 groups) performed, and 4 were selected to complete the top 12. 3 were chosen by the judges (one apiece) and 1 by audience vote.
3 1 Same as Season 2, except the wildcard round included only 8 people.
4 3 This year started with 12 men and 12 women. Each group sang on the same week on two consecutive nights. 2 men and 2 women were eliminated by audience vote. This repeated with 10 men and 10 women, then 8 men and 8 women, each a week apart. This left 6 men and 6 women, which comprised the Top 12. No wildcard round was held.
5 3 Same as Season 4.
6 3 Same as Season 4.
7 3 Same as Season 4.
8 1 3 groups of 12 (6 men and 6 women) sang, and 3 were selected by audience votes (without regard to gender). A wildcard round was held the same week as the third group, and 4 were selected by judges on the spot to make a Top 13.
9 3 Same as Season 4.
10 1 The Top 12 men and Top 12 women performed on consecutive days, with 5 each chosen by audience vote. A wildcard was held during the results show, and 3 were chosen by judges to make a Top 13.

I confess I don’t quite understand why the system was changed in Seasons 8 and 10, after a successful formula was implemented in Seasons 4-7 and Season 9. That system, with 3 rounds, seems to me the most fair. Perhaps not coincidentally, Seasons 4-7 were the best years for the show (though Season 9 was not).

Despite these differences, every season has a round that is the first performance by any given contestant. We can compare all of these rounds and see how pre-exposure affects the chances of being eliminated during this first voting round. Since all we know is the outcome of the voting (which is categorical), and since we’re interested in at least one variable that is continuous (the approval rating) the regression that makes sense is the logistical regression. Considering only approval rating as scored by WhatNotToSing and pre-exposure time (in seconds), we get a regression that fits quite nicely. To visualize this fit along one dimension, let’s look at how elimination probability changes with quality for different amounts of pre-exposure time:

Fitted elimination probability during the first round of voting as a function of approval rating, for different amounts of pre-exposure (see legend) in seconds. Dots are actual outcomes from the contest, with dots at 1.0 representing people who were eliminated and 0.0 representing people who were safe.

How to read this chart: Dots show actual results from the show, where a point at 1.0 on the vertical scale was a contestant that was eliminated and 0.0 was a contestant who was safe (in a sense, the probability that someone who was eliminated will be eliminated is 1!). The curves represent the probability of elimination as a function of the quality of the performance (on a scale of 0-100). The black line is for 0 seconds of pre-exposure, the red for 200 seconds of pre-exposure, etc. According to this, with a terrible performance (rating = 0) and no screen time during the auditions, there is a 90% chance of being eliminated in the first round. With 1000 seconds of pre-exposure and a great performance the elimination probability is practically zero. So at least we have that.

However, with a terrible performance, you can see that pre-exposure can pick up the slack quite a lot. Someone with as much pre-exposure as Scotty McCreery (> 1000 s) has only a 50% chance of elimination, compared to 90% for someone who got no screen time. The effect is even more pronounced for mediocre performances, around 50 on the WNTS scale, shown here in tabular form for easy reading:

Pre-exposure (seconds) Elimination Probability (percent)
assuming approval rating of 50
0 53.2
200 42.7
400 32.9
600 24.2
800 17.4
1000 12.1

For an average performance, getting about 10 minutes of screen time in the audition rounds cuts your chances of getting kicked off by half.

We can also look at this fit along the dimension of pre-exposure screen time for differing qualities of performance:

Probability of being eliminated in the first round of voting as a function of how much pre-exposure a contestant got in the audition/Hollywood rounds. Lines are shown for different performance quality (zero to 100 percent).

Having a high level of pre-exposure reduces the probability of being eliminated in the first round by half nearly regardless of how good your performance was.

Going back to the question of initial audition, we can see how the odds of being eliminated in the first round are affected by whether or not the initial audition was shown. The fit here is also good:

The probabilty of being eliminated in the first voting round as a function of approval rating, for contestants who did and did not have their initial audition shown.

As you can see, this one feature alone doesn’t dispose a contestant to be eliminated as prominently as total pre-exposure time does. Of course, if someone had his or her audition shown, then his total pre-exposure time is at least 60 seconds, often more, so this is entirely expected.

Correlation or causation?

What someone could say to all this is that the producers might choose the people who are the best at getting people to vote for them, and give them the most screen time during the audition rounds. As such, it isn’t that the pre-exposure time is influencing the votes, but that in fact the relationship is the other way around: the voting patterns affect who the producers choose to show. There could, also, be something else that causes both of these things.

This is the great problem of science, of course, which is to show that there is no common cause to the two things under consideration. Sadly, I am not in a position to carry out a controlled experiment in which two identical contests with identical people is carried out, but allocating screen time in an entirely different way. I can, however, think of one way to figure this out, though I do not have the data available to test it. If one could determine how many of the voters watched the different audition episodes (by, say, looking at the Nielson Ratings), and weigh the screen time accordingly, then you’ve removed the possibility of the producers’ prescience being a factor, and could show that the pre-exposure time directly influences who people vote for. Only the good people at Fox television have access to such data.

If you were asking me to wager, I would put my money on the hypothesis that the exposure does influence the voting. But, it actually doesn’t matter. While watching the semi-final rounds, if you find yourself wondering whether someone you like is going to get through, what you had better think to is whether that person was shown much in the initial auditions and in Hollywood week. If not, chances are you’re going to be disappointed.

Bookmark the permalink.

Comments are closed.

Comments are closed