Let’s Overanalyze Netflix’s New Viewing Metrics

SNL’s Squid Game parody. Photo-Illustration: Vulture; Photos by NBC

Netflix is such a big force in the culture that it’s rarely not making headlines in some way. But even by that standard, the streaming giant has been dominating news cycles to an almost ridiculous degree the past few weeks.

There was its massive dominance at the Emmys last month, where it not only crushed rival HBO but tied Nixon-era CBS for the most wins of any platform in TV history.

That was soon followed by the shocking success of Squid Game, which came out of nowhere (at least to clueless American audiences) to become a global sensation worthy of SNL parody just a month after launch. (And Nielsen now confirms how big it is: The show’s first full week snagged a massive 1.9 billion minutes of streaming in the States, an 800% jump from opening weekend. Needless to say, it was the No. 1 streaming show in America.)

There was also the far less welcome controversy generated by the latest stand-up special from Dave Chappelle, whose decision to use his platform to question trans people’s identity prompted a walkout of Netflix employees and a clumsy reaction by execs. Things got to the point that late Tuesday, co-CEO Ted Sarandos had to do a series of press interviews admitting he “screwed up” his handling of the situation.

And then there was the other big Netflix story of the week: The company’s third-quarter earnings. The news here was once again positive, as the streamer added another 4.4 million net global subscribers (double projections) and said it’s making more money from each subscription, too.

I’ll get back to the latest developments on Chappelle, but Netflix also used its quarterly report card to make some news on the data front. Rather than just offering a glimpse at viewership numbers through its earnings letters, or whenever it feels like bragging about something on Twitter, the streamer promised to “release title metrics more regularly” so that “members and the industry can better measure success in the streaming world.” It also said it would now reveal how many hours of viewership shows and movies subscribers consume rather than simply saying how many accounts sample something. Theoretically, these could be big developments, signaling the start of a new era of transparency for Netflix. In reality? Well …

While it is a good thing the streamer seems to be making an effort to be a bit more open about who’s watching what on the service, Tuesday’s announcement sent mixed messages about just how much things are going to change. Consider:

➽ The words “more regularly” in the aforementioned Netflix statement are maddeningly vague. If that phrase ends up meaning weekly or bi-weekly reports, then yes, the company will have taken a big step toward letting outsiders figure out how stuff is performing on the service. A monthly release would also be a step up from the status quo (numbers at least every three months, usually with one or two random data tweets in between) but not exactly a game changer. Anything less frequent would be a distinction without a difference. I emailed Netflix PR Wednesday for clarity; as of now, the company said it isn’t offering any more details on how its new plan will work.

➽ Number of hours consumed is absolutely a more valuable metric than the previous “accounts who watched for two minutes” number. It does away with all those doubts folks understandably had about people sampling something, bailing, and yet still being counted as a “viewer.” But this new data point has its own set of problems, not the least of which is that it ends up comparing apples and oranges — or in this case, comedies and dramas. A half-hour comedy such as Emily in Paris now automatically starts out at a disadvantage versus an hourlong drama. Things get even jankier when you throw in differences in how many episodes a series produces per season. For example, a five-episode limited series has to work twice as hard to tally up as many hours viewed as a standard ten-episode drama. And something like The Chair — a half-hour dramedy with just six episodes — is now going to have to be really, really popular to land on any future top ten lists. This is exactly the rationale Netflix execs have given in the past when asked why they weren’t releasing time-spent-watching data, by the way.

➽ In explaining its rationale for the change in measurement, Netlix correctly said this new yardstick is more in line with what Nielsen has been releasing (total minutes consumed) and that it does a better job reflecting audience passion for a title. After all, subscribers who really, really love Never Have I Ever or Lupin may watch an episode two or three times. That doesn’t show up when you’re only looking at which accounts clicked play, nor does it factor in folks who grew bored and tuned out a couple episodes into a show. All of this is true, but hours spent watching also doesn’t take into account members who like to take their time watching a series (not everyone binges in a weekend!) and thus may not have all their viewing show up in a report that measures the first four weeks a show is on the platform. It also diminishes the impact of subscribers who begin watching a show toward the end of a measurement period.

Despite these issues, I don’t think Netflix’s latest evolution on data reporting should be written off as meaningless. For one thing — and this is a pretty important point — we still don’t know all the details of these changes. While the wording of the letter sure makes it seem as if we’re never going to know how many member accounts sampled a title — i.e., if the new metric replaces the current one — maybe the streamer will surprise us and decide to release both, the way it did in some sample lists released in the shareholder letter. If so, it would sort of be huge: We’d be able to see if a half-hour show or a comedy special actually was generating interest among members but simply had too short a running time to crack a top ten. If Netflix wants reporters and the entertainment industry to take it seriously when it says it wants us to be able to “better measure success,” it should report more data and not simply different versions of it.

Another way Netflix could shock us all is if it started releasing hours viewed for all original shows. That would go a long way toward ensuring that comedies and series with smaller episode counts don’t get erased simply because their shorter run times make it difficult to generate as many hours of viewing as something that takes twice as long to watch. But if it won’t go that far, Netflix could at least decide to expand beyond reporting just on series and movies and publish regular lists of its top-performing comedies and specials, too, so we’d at least know how titles with similar lengths are doing. I would think creators of Netflix’s half-hour series would insist on this. If you’re Mindy Kaling, for example, do you want season three of Never Have I Ever being negatively compared to season four of You simply because your half-hour comedy simply has fewer hours with which to generate audience consumption?

Look, Netflix is never going to satisfy the reporters, agents, and producers who want to be able to quantify streaming audiences the same way we’ve measured TV consumption the past seven or so decades. There are some who are convinced the data it has been putting out the last year or two is b.s., and they’re not going to be swayed by a new flavor of it. (For the record, I am not a Netflix Numbers Truther. While the company clearly choreographs data dumps to paint its programming in the most positive light, I don’t think it simply makes stuff up — if only because the stats Netflix puts out end up in the company’s earnings reports, which means that if the data were fabricated, execs could end up going to jail over it.) But if Netflix is going to play the numbers game, it should at least commit to putting out data points that have real meaning and allow folks to make accurate comparisons among different programs. Even Squid Game has rules.

And Then There’s the Chappelle Stuff

Netflix employees and allies stage a protest. Photo-Illustration: Vulture; Photos by Al Seib/Los Angeles Times via Getty Images

As expected, trans activists and their allies within Netflix used a rally and walkout Wednesday to draw attention to their anger over both Chappelle’s special and the company’s actions surrounding its release. I wrote a more detailed look at what was at stake in a special extra edition of Buffering sent out Tuesday (you can read it here if it’s not in your inbox), and Vulture covered the events at the protest, too.

I have no clue what happens next in this ongoing story. Sarandos, having initially focused his energies on defending Chappelle, is now working overtime to assure Netflix staffers he also understands why the special has made them so uneasy. It’s not for me to say whether or not he’s said or done enough, but it seems clear to me that he is at least trying to rebuild whatever trust was broken between some staffers and the company.

AMC+ Makes a Date With Movie Premieres

Bruce Willis in Apex. Photo-Illustration: Vulture; Photos by RLJE

It’s been a big couple of weeks for feature films streaming on subscription services and opening in theaters the same day. HBO Max, which has been doing the shared windows thing with Warner Bros. movies since last December, debuted The Many Saints of Newark on October 1 and gets the new Dune today at 6 p.m. ET. Peacock shot to number one in Apple’s App Store over the weekend just as Halloween Kills landed on the service (and simultaneously slayed at the box office). And now, the smaller-but-growing AMC+ is looking to get in on the day-and-date action, too — part of a bigger effort to bulk up its movie offering.

Buffering has learned that the AMC Networks–owned platform has worked out a deal with sister company RLJE Films to bring the new Bruce Willis action film Apex to AMC+ on Friday, November 12, the same day the picture will open in theaters and be available via standard VOD purchase. The movie is one of four titles AMC+ will debut in November under the banner of “Friday Movie Streamieres” (hey, it’s better than “Peacocktober”) as the service attempts to expand its series-heavy lineup with new or nearly new theatrical films. The full slate:

 No Man of God (November 5): Elijah Wood plays an FBI analyst who, in 1980, develops a complex relationship with the imprisoned Ted Bundy (Luke Kirby), hoping to better understand the inner workings of the famed serial killer.

 Apex (November 12): Willis is an ex-cop who’s landed in jail for something he didn’t do and is now trying to win his freedom by competing in a Squid Game–like competition where six sick folks pay big bucks to hunt humans like Willis. (The film was shot last year.)

 Prisoners of the Ghostland (November 19): Nicolas Cage does his villain thing in this Sundance flick, playing an evil bank robber who is let out of jail in order to track down the missing granddaughter (Sofia Boutella) of a rich tyrant (Bill Moseley.) The catch: Cage’s character is locked in a leather suit set to self-destruct if he doesn’t complete his mission within five days. The movie will also stream on AMC-owned Shudder (which, by the way, really is a must-stream destination during Halloween season.)

 South of Heaven (November 26): Ted Lasso himself, Jason Sudeikis, goes a bit darker, playing a paroled convict whose desire to give his dying girlfriend (Evangeline Lilly) a memorable final year gets derailed when he ends up running afoul of a crime boss.

While Apex is the only day-and-date release on the AMC+ November calendar, the other movies were all released to theaters relatively recently. South of Heaven, for example, opened in limited release just two weeks ago; the others opened later this summer. And while none of them burned up the box office or made a big splash in the culture, in some ways that’s an advantage for AMC+: For most consumers, these films will essentially be AMC+ originals with well-known actors (Willis, Sudekis, Cage, Wood) as stars. Rising streamers such as AMC+ are always looking for ways to get folks to sample their platforms, and having this lineup of films on offer gives AMC+ marketing execs a new hook with which to tempt consumers.

Let’s Overanalyze Netflix’s New Viewing Metrics