Craptions

Bad closed captions can be entertaining, but they can be serious, too, because captions are a critical tool for lots of lots of people. There are the people learning a new language, the easily confused, and of course captions are essential for people who are Deaf or hard-of-hearing. In the US, that’s about 15% of the population.

And with so many low quality closed captions out there, those folks are often left trying to piece together meaning from some bizarre sentences, or … to just stop watching.

The issue of crappy captions isn’t a new thing. Activists have been fighting for accurate, widespread closed captioning for decades.

Bad captions aren’t just annoying; they’re supposed to be illegal, at least on television. In America, the FCC is responsible for making sure the airwaves are accessible for everybody. And since 1996, they’ve required all TV shows to include captions that are “accurate, synchronous, complete, and properly placed.”

Broadcasters in the United States also have to comply with the Americans with Disabilities Act, which passed in 1990. The ADA says all businesses that serve the public, or ‘places of public accommodation,’ must be equally available to everyone, which means broadcast TV needs to be accessible for the Deaf and hard-of-hearing.

Taken together, the FCC and ADA rules are pretty clear cut. For TV, you need to caption, and you need to caption well. And for a while, those rules were enough. But in the twenty-first century, streaming television changed everything. Listen to the episode by clicking the play button above, and/or take a read below!

INTRO

[[MUSIC IN – “Architect’s Medley 2022,” a mid-tempo waltz with strings]]

RM: This is 99% Invisible. I’m Roman Mars. Last fall, the BBC made a huge announcement. They had picked a new lead actor for the legendary TV show Dr. Who.

CLIP: Hello! I’m Ncuti Gatwa, and I am the next Doctor in the next season of Doctor Who.

CB: If you need some context, a new lead actor on Doctor Who is like a new Pope being selected, but for nerds!

RM: That’s producer Chris Berube.

CB: This announcement should have been a triumphant moment. After all, the actor Ncuti Gatwa was the first Black performer cast as the doctor. But when this video came out, the announcement was quickly overshadowed but something that should have been totally mundane. The closed captions.

RM: Gatwa’s first name is spelled N-C-U-T-I. But the closed captions on YouTube replaced his name with a swear word.

CB: So, while the actor said:

CLIP: Hello! I’m Ncuti Gatwa.

CB: The captions autocorrected his name to “Hello I’m [[shitty]] Gatsworth.”

[[MUSIC OUT.]]

CB: That was…unfortunate. But it was far from an isolated incident. When you start looking for Closed Caption failures, you see them pretty much everywhere. I turned on closed captions for a week on broadcast TV, YouTube, pretty much anywhere I was watching stuff. And I noticed so many errors.

[[MUSIC IN – “Alvaar Alto”, a driving up-tempo beat with synthesizers]]

CB: The fictional country in Black Panther, Wakanda, became “War! Canada.” Bill Murray and Dan Akroyd were suddenly part of an elite group called the “Ghost Buses”. It went on and on.

RM: There’s actually an academic, high-minded term to describe this phenomenon.

Linda Besner: Craptions are captions… that are crappy.

CB: That’s Linda Besner. She wrote about craptions for The Atlantic magazine. And she says there are many, MANY ways a caption can be crappy.

Linda Besner: They are not accurate. They don’t differentiate between speakers. So it’s just, sort of, one long run-on sentence. It’s not grammatically correct. It doesn’t contain punctuation. So they are sort of halfway-there captions.

[[MUSIC OUT]]

CB: Look, bad closed captions can be really funny, but captions are a critical tool for lots and lots of people. There’s people learning a new language, people like me who for the life of me cannot understand the Irish accents on Derry Girls. And of course, captions are essential for people who are Deaf and hard of hearing. In the United States, that’s about 15% of the adult population. And with so many craptions out there, those folks are often left trying to piece together meaning from some bizarre sentences, or, to just stop watching.

RM: The issue of crappy captions isn’t a new thing. Activists have been fighting for accurate, widespread closed captioning for decades. And it’s not clear when, if ever, they will become a reality.

HISTORY + ADA

[[MUSIC IN – “Marching Pizzicato”, a mid-tempo pizzicato string arrangement]]

CB: Bad captions aren’t just annoying; they’re supposed to be illegal, at least for television. In America, the FCC is responsible for making sure the airwaves are accessible for everybody. And since 1996, they’ve required all TV shows to include captions that are, quote, [exaggerated reading voice] “accurate, synchronous, complete, and properly placed.”

RM: Broadcasters in the United States also have to comply with guidelines under the Americans with Disabilities Act, which was passed in 1990. The ADA says all businesses that serve the public, or ‘places of public accommodation,’ must be equally available to everyone, which means broadcast TV needs to be accessible for the Deaf and hard of hearing.

CB: Taken together, the FCC and ADA rules were pretty clear. For TV, you need to caption, and you need to caption well. And for a while, those rules were enough. Until the rise of streaming TV.

[[MUSIC OUT]]

CB: Howard Rosenblum is the CEO of the National Association of the Deaf or the N-A-D. Howard himself is Deaf, and we interviewed him over Zoom with the assistance of an ASL interpreter. Throughout this story, you’ll hear her voice representing his answers.

Howard interpreter: Here he comes. There we go! He says [sing song voice] “Hello, Chris.”

CB: Howard points to 2007 as the first time the N-A-D got involved in captions for streaming, when Netflix launched its on-demand service.
Howard interpreter: At the time. If you recall, it was the era of DVDs in the postal mail, right? Ironically, there was no law which required DVDs to be captioned. But I would say roughly 85 to 90% of them were. So many of the members of NAD were Netflix subscribers, we’d get our DVDs in the postal mail, and it was all good. Netflix decided then to pivot to streaming. However, streaming at the time, there was almost none. Maybe 10% of them were captioned.

[[MUSIC IN – “From Scratch”, a worried mid-tempo string arrangement]]

CB: Howard and the NAD approached Netflix about this issue, but at the time, Netflix argued the rules didn’t apply. They weren’t television, so they weren’t under the jurisdiction of the FCC. And Netflix argued the ADA didn’t apply to them either. Here’s writer Linda Besner again.

Linda Besner: Title Three of the ADA says that in places of public accommodation, Deaf and hard of hearing people must be accommodated. So they should have an experience equal to that of a person who is not deaf or hard of hearing. Netflix’s argument was, we are not subject to the ADA because we’re not a place. They were like, well, we’re not an amusement park or like, you know, some kind of public forum. We’re not a physical location, so we don’t have to do this.

CB: The National Association of the Deaf argued that, ok, sure, Netflix is not a physical place, but isn’t it a kind of social place?
Linda: You know, what about a family watching a Netflix show together like, this is a social experience. Why should there be a member of that family who is excluded from that content?

[[MUSIC OUT]]

RM: In 2011, the NAD sued Netflix in a Massachusetts court. The judge agreed that the ADA applied to Netflix, and issued a consent decree, saying Netflix had to provide captions for all of its streaming content within four years.

CB: Today, the full Netflix library is captioned. They have teams of captioners working on every show. And some of their captions are really good, and some of them are only ok. But according to Howard Rosenblum of the NAD, Netflix has done a fairly good job.

Howard interpreter: Once that case was done and dusted, we began to approach other companies like Hulu. Amazon. So all three of them agreed to the terms of a phased-in captioning approach.

[[MUSIC IN – “Persistent Myth”, a slow, melancholy synthesizer arrangement]]

CB: Since the Netflix case, the National Association of the Deaf has been playing a game of whack-a-mole with the internet. Basically, a new streaming thing will crop up, and the NAD will helpfully remind them about their legal requirements to caption stuff.

RM: The NAD has worked with companies like Zoom and Microsoft Teams to encourage live captions for online meetings. And in 2015 the NAD sued Harvard and MIT for not providing captions for their online courses.

CB: Most recently…the NAD sued SiriusXM for not providing transcripts on their podcasts.

Howard interpreter: Nearly all podcasts are recorded, right? They’re not live. There’s no excuse for not providing a transcript.

[[MUSIC OUT]]

RM: Full disclosure here, 99PI is owned by SiriusXM. But I’ll note, our show does provide transcripts for all our episodes.

CB: While there’s been a lot of progress on the issue of streaming captions, there is one internet goliath the NAD won’t go after. It’s actually a place used by 80% of Americans, where people watch billions of hours of content every day. I’m talking about YouTube.

THE YOUTUBE PROBLEM

[[MUSIC IN – “The Maracana”, a slow, off-kilter keyboard arrangement with driving guitar]]

CB: Linda Besner spoke to YouTube about this problem a couple of years ago, and back then, they made it clear – they had no plans to require captions for all of their videos. Here’s Linda.

Linda Besner: They did say, for example, for things like live citizen journalism, like the spokesperson I spoke with gave the example of Arab Spring. You know, if it’s somebody who is posting a video of, you know, some event that is occurring around them, are you really going to flag for removal that video because it doesn’t have transcriptions?

RM: Even though YouTube isn’t legally required to provide closed captions, they have a lot of Deaf and hard of hearing users. In 2009, YouTube tried addressing the problem of closed captions by rolling out AI-generated captions for some of their videos. It hasn’t been a smooth ride.

CB: The automatic captions are created with Google’s Universal Speech Model, which uses speech recognition technology and AI to identify words, and put them together into coherent sentences.

Linda Besner: Predictive text is something that, you know, you may be familiar with from autocorrect on your phone. You started a sentence and then it wants to fill in your blank, much like maybe your mother does. And much like your mother, sometimes they don’t know what you are going to say and you have to be like, [[voice imitating a teenager]] “Mom, that’s not like, I’m not…I’m actually trying to make the opposite point right now.”

[[MUSIC OUT]]

CB: The early days of automatic captions were pretty rough. And Rikki Poynter remembers them well. Rikki is a Deaf YouTuber. And in 2009, she was watching a lot of beauty videos, using the brand new automatic captions.

Rikki Poynter: I remember, when I was watching a video about concealers, and the automatic captions, I had them on, and they were saying “zebra”, the mammal, the animal, whatever, in place of the word “concealers,” which was so bizarre. [[Laughter.]]

[[MUSIC IN – “Vulnerable Grid”, upbeat, slow-moving guitar-driven beat]]

CB: Rikki started her YouTube career making videos about makeup and beauty products. But she wanted to do something different. So in 2014, Rikki produced a video called “Things You Shouldn’t Say To People Who are Deaf or Hard of Hearing,” where she lampoons the clueless questions she was always being asked by hearing folks.

Audio of Rikki from “Things You Shouldn’t Say To People Who are Deaf or Hard of Hearing” – [[Question voice]] Is Deafness contagious? [[Normal cadence]] I don’t know but come over, I’ll cough on you and we’ll find out tomorrow. [[Question voice]] Why don’t you know sign language, you must not really be Deaf? [[Normal cadence]] Well, last time I checked Deafness has to do with whether or not your ears work and not the languages that you know.

CB: A couple of Rikki’s videos focused on the issue of closed captioning. And in one video, Rikki made a call to arms, asking her fellow YouTubers to ditch automatic captions, and write their own captions instead. Or better yet, to hire a professional service to make the captions for them.

RM: The campaign for human closed captions went viral. And Rikki caught the attention of some of YouTube’s biggest stars.

[[MUSIC OUT]]

Archival Audio of Tyler Oakley “Hear Me Out” video: [[Ukelele music playing underneath]] I want to make my little community here on YouTube more inclusive. And one of the ways I have decided to do that in the new year is by closed captioning every single one of my videos.

[[Tyler’s voice fades out underneath.]]

Rikki Poynter: [00:08:38] Tyler Oakley made a video while I was on a plane on the way to L.A., actually. And it was titled “Hear Me Out,” or something, and I was in the thumbnail, and he was talking about the importance of captions. And I had no idea until I had landed in L.A., turned on my phone, and I was getting so many notifications.

Archival Audio of Tyler Oakley “Hear Me Out” video: [[Audio of Tyler fades back in]] I saw a video from a YouTuber named Rikki Poynter talking about the importance of it, and why…[[Tyler’s voice fades out]]

CB: Today, Oakley’s video about closed captions has been watched more than 1.8 million times. And after that, other big time YouTubers weighed in, like Lilly Singh. But Rikki Poynter wanted to push the campaign into a higher gear. So in 2016, she launched a hashtag, called “No More Craptions.”

RM: The term had been floating around the Internet for a while, but Rikki helped bring it to the mainstream. Lots of YouTubers, both Deaf and hearing, made videos pledging to include human-produced captions in their videos.

[[MUSIC IN – “Bonkers Color”, peppy keyboard beat that changes tempo mid-way through, becoming a slow, mournful tune]]

Montage of YouTubers Discussing Closed Captions

First YouTuber: Closed captions help the Deaf and the hard of hearing…

Second YouTuber: Hello, today’s video is about the No More Craptions campaign.

Third YouTuber: Why isn’t this video closed captioned when it’s labeled closed captioned? And I was freaking out, because, [[Exaggerated voice]] Ahhhh…

End of Montage

Rikki Poynter: There’s some big beauty creators that have made announcements that they were going to start captioning. And they would start captioning a couple of videos at a time.

CB: Things were getting better! Human captions were showing up on more and more videos.. Until…some YouTubers just stopped doing it.
Rikki Poynter: As the years went on and like a lot of them would have fallen off of that. So that’s been kind of upsetting. [[Sighs]]
[[MUSIC OUT]]

HUMAN CAPTIONING

RM: Rikki says it was disappointing to see many YouTubers stop captioning their videos. There’s no mystery as to why this happened. Do-it-yourself captions can be time consuming.

CB: Meanwhile, hiring a professional to make your closed captions can be expensive. Professional captioners, on the whole, they aren’t paid very well. But for a YouTuber, dropping $40 or $50 to caption a video can add up pretty quickly. Look, captioning videos at a professional level is hard work.

Emma Healey: Oh, I got headaches. Yeah. And like, yeah, my eyes. I mean, my vision is not great anyway, but it was, it was pretty bad for all that stuff.

CB: My friend Emma Healey started work as a closed captioner in Toronto in 2017.

Emma Healey: The company that I was working for had been subcontracted out by an Australian broadcasting company. So I was captioning with Australian broadcasting rules.

CB: For this job, Emma had to watch a lot of crappy TV.

Emma Healey: A big one that people would complain about was Crocamole, which was an Australian children’s television show about a crocodile that loved guacamole. Named Crocamole.

CB: I got to be honest, our interview devolved into watching Crocamole for like half an hour.

Crocamole song plays, then fades under.

Emma Healey: [[Laughing]] It’s not the most lyrically inventive.

Chris Berube: This is first thought, best thought.

Emma Healey: His eyes are really sad.

CB: Even though Emma was captioning some pretty simple TV shows, the work took a very long time. Emma says she would do about 40 minutes of captioning in an eight hour shift. And that was on a good day.

Emma Healey: The place that I was working there was, I believe, a 75 page manual full of rules, all very technical. And you had to memorize basically all of them.

CB: Emma showed me the manual, which outlines the rules for how captions are supposed to look on Australian broadcast TV. And it made professional captioning seem impossibly byzantine.
[[MUSIC IN – “Mass Trespass”, driving, military-style drum beat]]

Emma Healey: Here we go. Minimum duration is one second no maximum, but 7 to 8 seconds is typical. Do not leave captions on screen overly long for no reason, if the dialog has ended. There must be a one second blank captions starting on the first frame of the program… [[Voice fades out underneath narration]]

CB: It goes on like this. For 75 pages. Emma had to keep all of this stuff in mind, for what was ultimately a minimum wage job.

RM: Professional captioning is complicated, because it’s not just about getting the words right. You also have to nail the punctuation. And the placement of the caption. And lots of other small details. Here’s Linda Besner.

[[MUSIC OUT]]

Linda Besner: So the gold standard for closed captioning would be something where, you know, for example, you and I are talking. When I am talking it might say Linda, colon, and then there would be a capital letter to indicate that I am beginning a sentence. And then it would faithfully represent what I have said, with question marks, where I have asked a question, and commas where commas go. When there is, you know, one of the cats makes a noise in the background [[Cats meowing]], there would be square brackets, cat meowing. If you know, a violent windstorm swept over [[Raindrops sweep against windowpane sound effects]] and you could hear, you know, raindrops coming against the window, it would have, bracket, “raindrops sweep against window pane.” And so this kind of caption really gives you the feeling that you are you are privy to anything in the auditory environment of that video.

CB: YouTube creators could either hire a professional captioner, which is expensive, or do this work themselves, which takes a long time for a beginner. So lots of them just don’t bother. Since Rikki’s campaign, there have been signs of progress.

THE FUTURE

[[MUSIC IN – “Roof Test”, hopeful, mid-tempo keyboard song]]

CB: According to YouTube, in the past few years, there’s actually been an increase in the number of human captions submitted by creators. From 2020 to 2021, the number of videos with manual captions on YouTube went up by 30%. But despite this uptick, let’s think about the bigger picture here. Automatic captions are still by far the most common type of caption on the platform.

RM: To this day, Youtube has no plans to require human-generated captions on their videos. And the NAD has said it has no plans to sue YouTube. Legally speaking, Howard Rosenblum doesn’t think the NAD can make an argument YouTube is covered by the ADA. Also, Howard believes it would be incredibly difficult to caption everything on the site.

[[MUSIC OUT]]

Howard interpreter: For YouTube, we do work closely, for example, with Google, who, well, owns YouTube. And we’ve talked through this issue. And there are two separate problems at play, one of which is working retroactively on the millions of minutes of content that is already uploaded to YouTube. Staggers the imagination.

CB: The second issue is the huge amount of new content uploaded every day.

Howard interpreter: I don’t remember the number of hours of videos posted to YouTube every minute, but it’s mind boggling.

RM: YouTube has experimented with ways to encourage more human captioning. In 2018, they introduced a program called “community captions,” where volunteers could submit captions for popular videos. Many Deaf YouTubers liked the program. But it was discontinued in 2020, because many of the volunteer captioners were submitting jokes or promotional material instead of accurate captions.

CB: When I reached out to YouTube, they pointed to a couple of new programs they are rolling out, one to allow users to give ‘subtitle editors’ permission to fix up their captions, another allows you to submit corrections for inaccurate captions on certain videos. But ultimately, it feels like YouTube is putting all of its chips on AI to fill in the caption gap. Just let the automatic captions continue improving until they get closer to total accuracy.

RM: To be fair, the automatic captions have gotten a lot better. YouTube would not share accuracy numbers for this story. But according to the Media Hub at the University of Minnesota at Duluth, YouTube auto-captions are somewhere around 70% accurate. That’s a big improvement from the early days.

CB: Rikki Poynter says she’s noticed the difference. AI captions still don’t include contextual things like the rain is softly falling softly in the background against the windowpane. But the AI has become very good at identifying dialogue.

Rikki Poynter: If I were to read the back of this floss package for a video… I don’t know, somebody is talking about floss out there. Dentists, right? So if I were to say, you know, [[Reading]] “dentists recommend daily flossing to help remove decay causing material from between teeth and under gums,” the automatic captions would be able to write that out, word-wise usually, really pretty well.

CB: There are still some major problems that auto-captions haven’t figured out. Grammar remains a problem. And the AI is pretty good with words in English, but it really struggles with other languages, especially when it comes to names and places. This is a pretty common complaint about AI programs, the way they can adopt certain biases.

RM: And there’s another issue with the captions produced by AI. Many users find them hard to read.

Rikki Poynter: They move in a more, like kinetic… I think that’s the term I learned like a week ago. Kinetic way where it [[Long pauses]] sounds [pause] like [pause] words [pause] are [pause] coming [pause] like this. They show up [[Pauses]] one [pause] word [pause] at [pause] a [pause] time. And that’s not the most accessible way to follow.

[[MUSIC IN – “Dial Up”, driving mid-tempo beat punctuated by the sounds of a dial-up modem]]

CB: Still, despite the challenges and frequent wonkiness, automatic captions just keep getting better. And they will keep improving. I mean, have you been following the terrifying progress of AI lately? There’s an AI program good enough to pass the bar exam now. So how hard are closed captions? Howard Rosenblum thinks the AI captions will compete with human captioners a lot faster than we think.

Howard interpreter: Five years ago, I would have said, ain’t no way. There is no way ASR is going to get anywhere. But now? Like. It’s shocking how much better it’s gotten. In five years, who even knows? I suspect it will be better. Maybe it will be less than five years.

CB: To me, it feels very possible the automatic captions will get close to 100% accuracy, at least for speech. But even if we can get there, the fact is, we aren’t there yet. And Deaf and hard-of-hearing people have real accessibility needs, right now. People like Rikki Poynter.

Rikki Poynter: I mean, would I love for automatic captions to be 100%? Absolutely. But it’s hard to know if that’s ever going to be possible, maybe in the year 3000? [[Laughs.]] I don’t want to wait years and years and years and years for that to happen.

[[MUSIC IN – “Ours to Lose”, contemplative, cello-driven piano song]]

CB: Rikki doesn’t go on YouTube much anymore. Not because of accessibility. It’s because, frankly, she feels like she’s growing out of it. A lot of her friends have left. It doesn’t feel like such a special community these days. Still, once in a while, Rikki checks in on her favorite accounts. Recently, she saw that one of her favorite creators had published her first video after a long hiatus.

Rikki Poynter: Charlie McDonnell, Charlie is so cool. Like, she came back to YouTube.

CB: Rikki was excited to watch the video. But as always, she was dreading the automatic captions. To Rikki’s surprise, the video had human captions. They weren’t totally professional, it looked like the YouTube star had done it herself. There were small mistakes, just little things that might drive some people nuts. But for Rikki, it was good to feel like somebody actually cared.

Rikki: It was fully captioned, mixed case, full words, punctuation and all. And I was like…[[Pause.]]… That’s awesome. [[Pause.]] I would love to see more of that.

[[MUSIC OUT]]

RM: When we come back…is it just me, or are TV shows and movies becoming harder to understand? We have an answer…after this.

CLOSED CAPTIONS CODA

RM: So I’m back with Chris Berube.

CB: Hey Roman.

RM: So do you use closed captioning? Because I use it all the time for everything.

CB: Just all the time. Yeah, everything I watch on Netflix, everything I watch at this point where it’s available. Yeah.

RM: So why do you use them?

CB: Well, I use them because I get confused easily. Okay. I don’t use them for accessibility reasons, but I’m certainly not the only hearing person using closed captions. There’s this one recent survey. It was more than 1200 people which found half of Americans are using closed captions most of the time, and that number is actually much higher for Gen Z. That’s like 70%. So it’s not just people who have accessibility needs.

RM: So why do you think so many people are using closed captions?

CB: Yeah, there’s a lot of reasons. I mean, one is that captions are just normal now, right? Like, especially for younger viewers, you see this in a lot of TikTok and Instagram videos that they have open captions. Which means that the captions are on all the time. You can’t turn them off. And this kind of an aesthetic choice, it’s just become a thing you see in a lot of videos. But going back to the survey, there’s one big reason that most people cite for using the closed captions. And the big reason is that audio sounds muddled. So people are saying modern dialog is more confusing, it’s harder to follow, and that’s why they like the captions so much.

RM: Yeah, I have noticed this. I call this the Christopher Nolan problem because it’s like he has a penchant for having mumbly low key but very good actors in his movies next to large booming sounds.

CB: I mean, I love Tom Hardy, but Tom Hardy, like, I don’t think I’ve ever heard him deliver a line straight. Deliver a line intelligibly.

RM: And so why do you think that is? I mean, why is dialog harder to follow now?

CB: Well, there’s actually a journalist who looked into this.

Edward Vega: All my friends were like, I can’t understand anything that anybody’s saying. And I was like, Man, I really wish that there is an answer to this.

CB: So that’s Edward Vega. He is a video producer for the website Vox. And he decided to look into this. Actually, the catalyst for him was watching the movie The King of Staten Island with Pete Davidson.

Edward Vega: It’s like it’s the emotional height of the movie. And he says, like, it’s hard. I think it’ll always be hard.

Clip from The King of Staten Island: [[unintelligible]] It’ll always be hard.

Edward Vega: But he mumbles it and he says it so quietly that, like, I swear to God, I went back three times.

CB: So Ed did this big investigation and he discovered, like, this is real. This is not in our heads. And there’s a couple of reasons this is happening.

RM: Okay, What are the reasons?

CB: So big one is dynamic range. So basically, if your movie has voices in it and your movie also has explosions, you have to make sure the explosions are louder than the voices. Right? Like, I mean, that seems obvious, but it’s something where the explosions have to be quite a bit louder so that they have an impact.

Edward Vega: When you’re mixing a movie, if your explosion is going to be the loudest thing in the mix, then your dialog can’t be that loud. And so you have to choose which one you want to move. Do you want to move the dialog down or do you want to move the explosion up?

CB: One of these issues, it is this idea of dynamic range. It’s balancing things out so that the explosion has more impact. As a result, the dialog suffers. So another big issue is the rise of naturalism in performances.
Austin Olivia Kendrick: If an actor mumbles their way through a line, if their words start to run together, there’s nothing I can do to fix that, you know?

CB: This is Austin Olivia Kendrick.

Austin Olivia Kendrick: I currently work as a dialog editor at Warner Brothers and Warner Brothers Television specifically.

CB: So Edward Vega very nicely introduced the two of us, and Austin says movie technology has changed and as a result, actors’ performances have changed as well.

Austin Olivia Kendrick: Back in the day when, um, when sound technology was first introduced to film, there wasn’t that that technology was very kind of primitive. Um. The only real option when it came to filming movies with sound was you were on a soundstage, a microphone was planted above your head and the actors had to stand there and project into the microphone.

Austin Olivia Kendrick: But as sound technology evolved and all of a sudden we have wireless microphones that we can place on actors, that kind of loosened up actors’ performances. They now no longer have to stand in one spot and project. And gave them more freedom and kind of pushed them more into a naturalistic style of performance. And that subsequently could mean mumbling.

CB: So there you go, technology. The second thing, that’s why you’re getting these mumbly performances.

RM: Yeah, but for decades and decades I know that they’ve been doing dialog replacement where like an actor comes in after the fact and records dialog to make it more clear.

CB: Right. Yeah, that’s ADR. And I did talk to Austin about this and she says doing ADR is actually pretty difficult, especially when the dialog. It’s very mumbly.

Austin Olivia Kendrick: When we bring actors in to rerecord lines, they have to match their performance to the way that their lips are moving on screen. And if they mumbled their way through that take, their lips aren’t moving very much. So if they come in and re-record a line with more diction and a lot more separation between the words, that’s not gonna match.

CB: The last issue. And this might be the biggest one. This is something called down-mixing. So this is the idea that a lot of movies and some TV shows are mixed for the best possible sound system, right? Many of them are mixed for Dolby Atmos. That’s the special sound system you get in certain theaters that has 128 channels. Okay. So you can mix something in a very, very specific, particular way. And then after you’ve mixed it for the best sound system, you have to remix it for other platforms like a television or a phone. And that is when you start to get into trouble.

Edward Vega: You’re taking 128 channels and you’re compressing that down into different formats with lesser channels. And most of the time when you’re watching something on your TV, unless you have a surround sound system, it’s going to be stereo or mono. So if you’re going from 128 down to two or down to one, it gets muddy, it gets a lot muddier.

CB: So lots of movies and TV shows, you know, they’re getting mixed on this sound system. Then they’re getting down-mixed for other formats. And Austin says usually the down mix at the end, that’s the last thing they do in the editing process. And sometimes they don’t put quite the same care and effort into it that they would for the initial mix for a movie theater.

Austin Olivia Kendrick: A lot of the times, when it comes to that down mixing, often studios do not want to spend that the money because ultimately time is money and rerecording mixers are very expensive to hire for that extra time you know so oftentimes they don’t want to put they want to put as little money into that area as possible.

CB: So, Roman, there you have it. It’s down-mixing. It’s naturalistic performances. It’s this dynamic range issue. Those are some of the reasons why you did not understand the movie Tenet by Christopher Nolan.

RM: Well, they’re not the only reason I didn’t understand Tenet. But that helps.

CB: Yeah. The time travel stuff also contributes to it. I guess there’s a lot going on, working against Tenet being a comprehensible film. So I have to say before we go, the video by Edward Vega on Vox, it goes into so much more depth on this topic. It’s wonderful. I hope everybody checks it out. And also, I hope everyone checks out. Austin, Olivia Kendrick’s TikTok. If you use that platform, she explains movie sound. And it is so fascinating.

RM: It’s awesome. All right. Thank you so much, Chris.

CB: Thanks Roman.

CREDITS

RM: 99% Invisible was produced this week by Chris Berube, with editorial input from Linda Besner. She originally wrote about this topic for The Atlantic magazine. Linda is writing a book about people and their complicated relationships with money – look for it soon at a bookstore near you.

This story was edited by Kelly Prime. Original music by Swan Real. Sound mix by Martín Gonzalez. Fact-checking by Graham Hacia, who couldn’t independently verify that Crocamole loved guacamole, but we assured them that this was true and we went with it anyway. Kurt Kohlstedt is our digital director. Our intern is Avanti Nambiar. The rest of the team includes Emmett Fitzgerald, Senior Editor Delaney Hall, Christopher Johnson, Jayson De Leon, Vivian Le, Jeyca Maldonado Medina, Joe Rosenberg, and me, Roman Mars. The 99% Invisible logo was created by Stefan Lawrence.

Special thanks this week to Caroline Mincks and Anne Ballentine.

99% Invisible is part of Stitcher and SiriusXM, now headquartered six blocks north in the Pandora Building… in beautiful… uptown… Oakland, California. You can find the show and join discussions about the show on Facebook. You can tweet me @romanmars and the show @99piorg. We’re on Instagram, Reddit, and TikTok too. You can find links to other Stitcher shows I love as well as every past episode of 99PI at 99pi.org.