For those just hitting the site (now defunct), here is a compendium of the knowledge I’ve derived from American Idol. Much of the code used in making the models is found on Github, as is the data in the database.

First, the models:

I am also particularly proud of the Statistical Snapshot of American Radio (methodology here, see all posts here), a large data-mining effort to quantify what songs are predominantly played on radio by format, time-period. I built some fairly sophisticated data visualization tools for these, also on Github, along with all the data.

On more Idol related issues:

It wouldn’t be a website in the 2010s without some listicles:

Finally, I spent many hours compiling video retrospectives of the first three seasons. You can watch them all here.

Thoughts on Idol’s ending

This is the penultimate post that will appear on this website (I will liveblog the show’s finale tomorrow). I haven’t followed this season at all, but I would reckon that the finale will try to bring together the show’s history into some kind of retrospective.

Yesterday I read the story of how Idol began from the man at Fox who was mostly responsible for it. He goes by the moniker Masked Scheduler on Twitter, though for some time it’s known that he is Preston Beckman, former scheduler at Fox and NBC before that. The story is worth a read. Beckman, for instance, had to come up with the idea of a results show that was separate from the performance show due to the fact that the US has many time zones, but Britain has only one:

Since Idol was a competition show based on viewer voting they suddenly realized that the US has four time zones (five if you include Hawaii) and that Idol could not replicate the British formula of voting and then returning to the air later in the evening to report the results. Mike and I had to explain the difference [to the producers] between network and affiliate time and all the issues involved with that. We played around with different solutions such as excluding the West Coast from participating in the voting. Finally, I threw out the idea of a second show that would air the night following the voting.

At its peak, Fox had The Death Star as 3 full hours of primetime in its weekly schedule. I don’t envy the person who tries to fill it.

What most interested me, though, was the explanation of what I would call the end of the golden era, when Fox changed the format and brought in a fourth judge

For me Season 8 was the pivotal year for American Idol. The ratings were starting to decline so there was a feeling among the powers that be that we needed to shake things up…and we did. We eliminated the boy/girl round of 24 and went back to a final 36 where groups of 12 contestants performed over three weeks. We added a fourth judge in Kara Dioguardi thus telling the world Paula Abdul’s days were numbered. After years of resistance on our part we added the judge’s save to the show. We negotiated with the producers as to how deep into the show the save could be used and we made sure that they stuck to it. Finally, the auditions, which were the highest rated part of the show, were taking on a nasty tone and the ratings were reflecting it.

I think this was a mistake for many reasons. For one thing, the ecosystem that had come up around Idol depended on many things, and discussions of Paula’s lucidity were part of the fun. The producers ought to have done anything they could to keep the original judging panel for as long as possible. Dioguardi rarely added anything to the discussion. For instance, I once went through and tabulated how the judges rated the performances for each show on a scale from 1 (being bad) to 3 (being great). Looking only at season 9, see the table below

Randy Kara Paula Simon
Average score 2.095 2.183 2.449 1.929
Standard deviation 0.863 0.833 0.753 0.856

Paula gave the highest reviews overall (no surprise there), and a small standard deviation. But Kara, while overall giving more negative reviews, had a standard deviation lower than Simon or Randy. So why have her there? Paula at least was interesting. And don’t get me started on Ellen Degeneres, who gave an overall average 2.317 and standard deviation 0.779. Ellen was a combination of reviews that were as uniformly positive as Paula’s but without any other redeeming qualities (such as forgetting how many times the person had sung).

Changing judges was trivial, though, compared to possibly the biggest mistake Idol ever made: combining men and women into the same semifinal rounds. As I’ve said in this space many times (and is borne out by the data), women face a significant disadvantage overall, and tended to be eliminated at a high rate in semifinals. The superstars and semi-stars that have come out of Idol have all been women (save for Chris Daughtry). Jordin Sparks and Carry Underwood both took time to mature, and the semi-final round rules kept them in and gave them that opportunity. By eliminating this, they shifted the contest to one that was advantageous to one subgroup of people: people who were white, people who played guitars, and people who were male.

The judges’ save similarly took out part of the fun of the show, because it muted the effect of VoteForTheWorst, a site dedicated to keeping terrible singers in the contest for as long as possible.  The judges’ save would never have allowed  Gina Glocksen to be eliminated while Sanjaya stayed in Season 6.

I also missed the theme nights, which forced the contestants to perform out of their comfort zone. Almost nothing brought me the kind of glee I got from seeing a supposed hard-rock guy have to pick an Andrew Lloyd Webber tune and perform it hilariously on stage.

For his part, Beckman ascribes the main part of the decline to be due to a critical mistake:

Was there a moment when this all could have ended differently? Yes, it was when we delayed the launch of X Factor by a year after we planned the transition of Cowell from Idol to X Factor. That gave NBC the opening to introduce The Voice that was more akin to X Factor than to American Idol. This also kept Cowell off the air for over a year. That was not the plan. Once Idol was not THE singing competition but one of three well….

I think this analysis is probably at least partly true. But allowing Cowell to leave at all seems to be the main issue. The X-Factor was never popular, and was canceled after 3 seasons, but in the meantime the audience had two shows being run on Fox (though not concurrently). I don’t think that diversifying in a market that was likely already at saturation was a very good idea.

What seems clear to me is that the show could still do some good, but needs to go away for probably a decade. Idol for years has been drawing at least partially from cast-offs from other shows (like The Voice), and has seen a disturbing trend of recruiting contestants who already have released records and have a fan-base (Jax comes to mind as someone who had an organizational effort on Twitter from day 1). The thing that the producers of Idol perhaps forgot was that good singing was always only part of the equation: those of us who relished the show loved the portions where there was bad singing, we loved the stupid disco theme, we loved the judges bickering, and we even liked the injustice of the Worsters spoilers.

The ironic thing is that I can point out very memorable moments from even recent years (particularly season 12), but the totality of it is severely nerfed from Idol’s heyday. Every reality show eventually declines and dies, and Idol had a good long run. But it’s long past time to end it. Maybe if things lie fallow for a number of years, there will be a new generation of undiscovered singers who need such a show. As it stands, I think that field is infertile. Season 12 might have been a fitting end, if I had to choose.

They nays have it: American Idol cancelled after next year


Entertainment Weekly is out with the report this morning:

Fox is officially closing the curtain on American Idol after its upcoming 15th season, the network announced Monday.

American Idol XV will have the same judges as the last two cycles—Jennifer Lopez, Keith Urban and Harry Connick, Jr.— along with the show’s host, Ryan Seacrest, who has been with the series from the beginning.

This shouldn’t be a shock to anyone. American Idol’s ratings have cratered due to a number of reasons. First is the competition from other shows that do the same thing, some would say better [NB: I have never watched The Voice or America’s Got Talent]. Second is the waning of the early 21st century reality show ecosystem, as cable and network dramas have eliminated the empty rerun season that Idol was originally envisioned to fill. The show has also become much too expensive to run: it requires a full-time auditorium, the salaries of three star judges, recording studio time, plus the audition tour. Fox reported before this year started that American Idol was no longer turning a profit.

I have to say, I’m puzzled as to what Fox thought would change this year. If the show was already unprofitable, was this the team to turn it around? Could anyone turn it around at this point?

Fox has also signaled that the final season will be a nostalgia tour, saying “the final season will pay tribute to the show’s previous years.” That’s an admission in and of itself: the previous years were just better. The judges were harsh and not constantly fawning over the contestants. They steered the competition, rather than letting a pure democracy rule. But that all ended the day Simon Cowell left the show on an ill-fated attempt to bring the X-Factor to the US, a show that flopped spectacularly. The judges panel never recovered.

It remains to be seen whether the other shows of this ilk will go by the wayside as well. Many commenters at TvbytheNumbers predict that all of the singing reality-shows are on their way out, and maybe they’re right. It is possible that at some point you exhaust the nation of all its talented, undiscovered artists. This is particularly true when there are a number of new avenues for distributing demos, such as Youtube, which, I will remind the reader, didn’t even exist before about the 4th season of American Idol. Indeed, several singers in recent years were from other shows, and cut Idol contestants have appeared on the other shows as well. It’s all beginning to look like bottom-of-the-barrel.

So that’s it. The Death Star goes out not with a bang, but with a nostalgic whimper. We have one more year of Seacrest et al, and then finis. I myself am hoping for a return of Brian Dunkleman before the end.