Craptions

Roman Mars [00:00:01] In Williamsburg, Virginia. There’s never too much of a good thing. Whether you’re a foodie, a golfer, history buff, a shopaholic, an outdoor enthusiast, or a thrill seeker, you’ll find what you’re looking for. Explore the grounds of America’s first English settlement in Jamestown, or shop along the quaint streets of historic Williamsburg and Yorktown. Dig into the forensics of the country’s earliest settlers or experience a day in the life of one. Williamsburg is the type of destination that you can go back to again and again and have a completely different experience. So, plan your visit now. With thousands of new podcasts being started every day, it seems, it’s more important than ever to make sure you stand out from the crowd. And this goes for starting any business. Squarespace helps you create an all-in-one platform to grow your business online through an engaging and aesthetically pleasing website, where you can reach anybody or sell anything. Squarespace lets you use customizable galleries to display images in unique ways. You can sell your products in an online store. You’ll find eCommerce templates, inventory management, and a simple checkout process. And it’s easy to get started with custom templates. Head to squarespace.com/invisible for a free trial. And when you’re ready to launch, use the code “invisible” to save 10% off your first purchase of a website or domain. This is 99% Invisible. I’m Roman Mars. Last fall, the BBC made a huge announcement. They had picked a new lead actor for their beloved TV show, Doctor Who. 

Ncuti Gatwa [00:01:34] Hello. I’m Ncuti Gatwa, and I am the next Doctor in the next season of Doctor Who. 

Chris Berube [00:01:40] If you need some context, a new lead actor on Doctor Who is like a new pope being selected, but for nerds. 

Roman Mars [00:01:46] That’s producer Chris Berube. 

Chris Berube [00:01:48] This announcement should have been a triumphant moment. After all, the actor Ncuti Gatwa was the first Black performer cast as the Doctor. But when this video came out, the announcement was quickly overshadowed by something that should have been totally mundane–the closed captions. 

Roman Mars [00:02:05] Gatwa’s first name is spelled N.C.U.T.I. But the closed captions on YouTube replaced his name with a swear word. 

Chris Berube [00:02:13] So while the actor said: 

Ncuti Gatwa [00:02:15] Hello, I’m Ncuti Gatwa. 

Chris Berube [00:02:17] The captions autocorrected his name to: “Hello. I’m Shitty Gatsworth.” That was unfortunate, but it was far from an isolated incident. When you start looking for closed caption failures, you see them pretty much everywhere. I turned on closed captions for a week on broadcast TV, YouTube, pretty much anywhere I was watching stuff. And I noticed so many errors. The fictional country in Black Panther, Wakanda? It became “War! Canada.” Bill Murray and Dan Aykroyd were suddenly part of an elite group called the “Ghost Buses.” It went on and on. 

Roman Mars [00:02:55] There’s actually an academic, high-minded term to describe this phenomenon. 

Linda Besner [00:02:59] Craptions are captions that are crappy. 

Chris Berube [00:03:03] That’s Linda Besner. She wrote about captions for The Atlantic magazine. And she says there are many, many ways a caption can be crappy. 

Linda Besner [00:03:12] They are not accurate. They don’t differentiate between speakers, so it’s just sort of one long run-on sentence. It’s not grammatically correct. It doesn’t contain punctuation. So, they are sort of halfway there captions. 

Chris Berube [00:03:27] Look, bad closed captions can be really funny, but captions are a critical tool for lots and lots of people. There’s people learning a new language. People like me who, for the life of me, cannot understand the Irish accents on Derry girls. And of course, captions are essential for people who are deaf and hard of hearing. In the United States, that’s about 15% of the adult population. And with so many craptions out there, those folks are often left trying to piece together meaning from some bizarre sentences or to just stop watching. 

Roman Mars [00:04:03] The issue of crappy captions isn’t a new thing. Activists have been fighting for accurate widespread closed captioning for decades. And it’s not clear when, if ever, they will become a reality. 

Chris Berube [00:04:17] Bad captions aren’t just annoying. They’re supposed to be illegal, at least for television. In America, the FCC is responsible for making sure the airwaves are accessible for everybody. And since 1996, they’ve required all TV shows to include captions that are, quote, “accurate, synchronous, complete, and properly placed.” 

Roman Mars [00:04:38] Broadcasters in the United States also have to comply with guidelines under the Americans with Disabilities Act, which was passed in 1990. The ADA says, “All businesses that serve the public or places of public accommodation must be equally available to everyone,” which means broadcast TV needs to be accessible for the deaf and hard of hearing. 

Chris Berube [00:04:59] Taken together, the FCC and ADA rules were pretty clear. For TV, you need to caption, and you need to caption well. And for a while those rules were enough. Until the rise of streaming TV. Howard Rosenblum is the CEO of the National Association of the Deaf–or the NAD. Howard himself is deaf. And we interviewed him over Zoom with the assistance of an ASL interpreter. Throughout this story, you’ll hear her voice representing his answers. 

Interpreter [00:05:32] Here he comes. “There we go,” he says. “Hello, Chris.”

Chris Berube [00:05:37] Howard points to 2007 as the first time the NAD got involved in captions for streaming, when Netflix launched its on demand service. 

Interpreter [00:05:46] “At the time, if you recall, it was the era of DVDs in the postal mail, right? Ironically, there was no law which required DVDs to be captioned. But I would say roughly 85% to 90% of them were. So many of the members of NAD were Netflix subscribers, we’d get our DVDs in postal mail, and it was all good. Netflix decided then to pivot to streaming. However, streaming at the time–there was almost none. Maybe 10% of them were captioned.” 

Chris Berube [00:06:26] Howard and the NAD approached Netflix about this issue. But at the time, Netflix argued the rules didn’t apply to them. They weren’t television, so they weren’t under the jurisdiction of the FCC. And Netflix argued the ADA didn’t apply to them either. Here’s writer Linda Besner again. 

Linda Besner [00:06:43] Title III of the ADA says that “in places of public accommodation, deaf and hard of hearing people must be accommodated.” So, they should have an experience equal to that of a person who is not deaf or hard of hearing. Netflix’s argument was: “We are not subject to the ADA because we’re not a place.” They were like, “Well, we’re not an amusement park or, like, you know, some kind of public forum. We’re not a physical location, so we don’t have to do this.” 

Chris Berube [00:07:11] The National Association of the Deaf argued that “okay, sure, Netflix is not a physical place, but isn’t it kind of a social place?” 

Linda Besner [00:07:19] You know, what about a family watching a Netflix show together? This is a social experience. Why should there be a member of that family who is excluded from that content? 

Roman Mars [00:07:33] In 2011, the NAD sued Netflix in a Massachusetts court. The judge agreed that the ADA applies to Netflix and issued a consent decree saying Netflix had to provide captions for all of its streaming content within four years. 

Chris Berube [00:07:47] Today, the full Netflix library is captioned. They have teams of captioners working on every show. And some of their captions are really good, and some of them are only okay. But according to Howard Rosenblum of the NAD, Netflix has done a fairly good job. 

Interpreter [00:08:02] “Once that case was done and dusted, we began to approach other companies like Hulu, Amazon. So, all three of them agreed to the terms of a phased in captioning approach.”

Chris Berube [00:08:17] Since the Netflix case, the National Association of the Deaf has been playing a game of whack a mole with the internet. Basically, a new streaming thing will crop up, and the NAD will helpfully remind them about their legal requirements to caption stuff. 

Roman Mars [00:08:31] The NAD has worked with companies like Zoom and Microsoft teams to encourage live captions for online meetings. And in 2015, the NAD sued Harvard and MIT for not providing captions for their online courses. 

Chris Berube [00:08:45] Most recently, the NAD sued SiriusXM for not providing transcripts on their podcasts. 

Interpreter [00:08:52] “Nearly all podcasts are recorded, right? They’re not live. There’s no excuse for not providing a transcript.”

Roman Mars [00:09:01] Full disclosure here, 99PI is owned by SiriusXM–but I’ll note that our show does provide transcripts to all of our episodes. 

Chris Berube [00:09:08] While there’s been a lot of progress on the issue of streaming captions, there’s one internet Goliath the need won’t go after. It’s actually a place used by 80% of Americans where people watch billions of hours of content every day. I’m talking about YouTube. Linda Besner spoke to YouTube about this problem a couple of years ago. And back then, they made it clear they had no plans to require captions for all of their videos. Here’s Linda. 

Linda Besner [00:09:39] They did say, for example, for things like live citizen journalism– The spokesperson I spoke with gave the example of Arab Spring. If it’s somebody who is, you know, posting a video of some event that is occurring around them, are you really going to flag for removal that video because it doesn’t have transcriptions?

Roman Mars [00:10:01] Even though YouTube isn’t legally required to provide closed captions, they have a lot of deaf and hard of hearing users. In 2009, YouTube tried addressing the problem of closed captions by rolling out AI-generated captions for some of their videos. It hasn’t been a smooth ride. 

Chris Berube [00:10:17] The automatic captions are created with Google’s universal speech model, which uses speech recognition technology and AI to identify words and then put them together into coherent sentences. 

Linda Besner [00:10:29] Predictive text is something that, you know, you may be familiar with from autocorrect on your phone. You started a sentence and then it wants to fill in your blank, much like maybe your mother does. And much like your mother, sometimes they don’t know what you are going to say, and you have to be like, “Mom, that’s not… I’m not… I’m actually trying to make the opposite point right now.” 

Chris Berube [00:10:50] The early days of automatic captions were pretty rough, and Rikki Poynter remembers them well. Rikki is a deaf YouTuber, and in 2009, she was watching a lot of beauty videos using the brand-new automatic captions. 

Rikki Poynter [00:11:03] I remember I was watching a video about concealers. And the automatic captions–I had them on–were saying, “Zebra,” the mammal, the animal, whatever, in place of the word “concealers,” which was so bizarre. 

Chris Berube [00:11:22] Rikki started her YouTube career making videos about makeup and beauty products. But she wanted to do something different. So, in 2014, Rikki produced a video called Things You Shouldn’t Say to People Who are Deaf or Hard of Hearing, where she lampoons the clueless questions she was always being asked by hearing folks. 

Rikki Poynter [00:11:40] “Is deafness contagious?” I don’t know. But come over here. I’ll cough on you. And we’ll find out tomorrow. “Why don’t you know sign language? You must not really be deaf.” Well, last time I checked, deafness has to do with whether or not your ears work and not the languages that you know…

Chris Berube [00:11:57] A couple of Rikki’s videos focused on the issue of closed captioning. And in one video, Rikki made a call to arms, asking her fellow YouTubers to ditch automatic captions and write their own captions instead–or better yet, to hire a professional service to make the captions for them. 

Roman Mars [00:12:15] The Campaign for Human Closed Captions went viral. And Rikki caught the attention of some of YouTube’s biggest stars. 

Tyler Oakley [00:12:22] I want to make my little community here on YouTube more inclusive. And one of the ways that I have decided to do that in the new year is by closed captioning every single one of my videos…

Rikki Poynter [00:12:33] Yeah. Tyler Oakley made a video while I was on a plane on the way to LA actually. And it was, like, titled “Hear Me Out” or something. And I was in that thumbnail, and he was talking about the importance of captions. And I had no idea until I had landed in LA, turned on my phone, and was getting so many notifications… 

Tyler Oakley [00:12:51] I saw a video from a YouTuber named Rikki Poynter, who talked about the importance of it… 

Chris Berube [00:12:57] Today, Oakley’s video about closed captions has been watched more than 1.8 million times. And after that, other big time YouTubers weighed in, like Lilly Singh. But Rikki Poynter wanted to push the campaign into a higher gear. So, in 2016, she launched a hashtag called “No More Craptions.”

Roman Mars [00:13:18] The term had been floating around the internet for a while, but Rikki helped bring it to the mainstream. Lots of YouTubers, both deaf and hearing, made videos pledging to include human produced captions in their videos. 

YouTuber #1 [00:13:30] Closed captions help the deaf and the hard of hearing… 

YouTuber #2 [00:13:34] Hello. Today’s video is about the No More Craptions campaign… 

YouTuber #3 [00:13:38] Why isn’t this video closed captioned when it’s labeled “closed captioned.” And I was freaking out because ahhh… 

Rikki Poynter [00:13:46] There’s some big beauty creators that have made announcements that they were going to start captioning. And then they would start captioning, you know, a couple of videos at a time. 

Chris Berube [00:13:56] Things were getting better. Human captions were showing up on more and more videos until some YouTubers just stopped doing it. 

Rikki Poynter [00:14:07] As the years went on, a lot of them would have fallen off of that. So that’s been kind of upsetting. 

Roman Mars [00:14:16] Rikki says it was disappointing to see many YouTubers stop captioning their videos. There’s no mystery as to why this happened. Do-it-yourself captions can be time consuming. 

Chris Berube [00:14:25] Meanwhile, hiring a professional to make your closed captions can be expensive. Professional captioners–on the whole–they aren’t paid very well. But for a YouTuber, dropping $40 or $50 to caption a video can add up pretty quickly. Look, captioning videos at a professional level is hard work. 

Emma Healey [00:14:46] Oh, I got headaches. Yeah. And like… Yeah, my eyes. I mean, my vision is not great anyway. But it was pretty bad for all that stuff. 

Chris Berube [00:14:55] My friend Emma Healey started work as a closed captioner in Toronto in 2017. 

Emma Healey [00:15:00] The company that I was working for had been subcontracted out by an Australian broadcasting company, so I was captioning with Australian broadcasting rules. 

Chris Berube [00:15:11] For this job, Emma had to watch a lot of crappy TV. 

Emma Healey [00:15:15] A big one that people would complain about was Crocamole, which was an Australian children’s television show about a crocodile that loved guacamole named Crocamole.

Chris Berube [00:15:26] I got to be honest, our interview devolved into watching Crocamole for, like, half an hour. 

Crocamole Theme Song [00:15:32] Crocamole, not ravioli, makes you big and strong! Crocamole!

Emma Healey [00:15:37] It’s not the most lyrically inventive.

Chris Berube [00:15:40] This is first thought best thought.

Emma Healey [00:15:45] Yes. His eyes are really sad. 

Chris Berube [00:15:48] Even though Emma was captioning some pretty simple TV shows, the work took a very long time. Emma says she would do about 40 minutes of captioning in an eight-hour shift. And that was on a good day. 

Emma Healey [00:16:00] The place that I was working–there was, I believe, a 75-page manual full of rules, all very technical, and you had to memorize basically all of them. 

Chris Berube [00:16:11] Emma showed me the manual, which outlines the rules for how captions are supposed to look on Australian broadcast TV. And it made professional captioning seem impossibly byzantine. 

Emma Healey [00:16:23] Here we go. “Minimum duration is one second. No maximum, but seven to eight seconds is typical. Do not leave captions on screen overly long for no reason if the dialogue has ended. There must be a one second blank…”

Chris Berube [00:16:37] It goes on like this–for 75 pages. Emma had to keep all of this stuff in mind for what was ultimately a minimum wage job. 

Roman Mars [00:16:47] Professional captioning is complicated because it’s not just about getting the words right. You also have to nail the punctuation and the placement of the caption and lots of other small details. Here’s Linda Besner. 

Linda Besner [00:16:58] So the gold standard for closed captioning would be something where, you know, for example, you and I are talking. When I am talking, it might say, “Linda:” and then there would be a capital letter to indicate that I am beginning a sentence. And then it would faithfully represent what I have said with question marks where I have asked a question and commas where commas go. When the cat makes a noise in the background, there would be square brackets–“[Cat meowing].” If a violent windstorm swept over and you could hear, you know, raindrops coming against the window, it would have “[Raindrops sweep against windowpane].” So, this kind of caption really gives you the feeling that you are privy to anything in the auditory environment of that video. 

Chris Berube [00:17:49] YouTube creators can either hire a professional captioner, which is expensive, or do this work themselves, which can take a long time for a beginner. So, a lot of them just don’t bother. Since Rikki’s campaign, there have been signs of progress. According to YouTube, in the past few years, there’s actually been an increase in the number of human captions submitted by creators. From 2020 to 2021, the number of videos with manual captions on YouTube went up by 30%. But despite this uptick, let’s think about the bigger picture here. Automatic captions are still by far the most common type of caption on the platform. 

Roman Mars [00:18:28] To this day, YouTube has no plans to require human generated captions on their videos. And the NAD has said it has no plans to sue YouTube. Legally speaking, Howard Rosenblum doesn’t think the NAD can make an argument that YouTube is covered by the ADA. Also, Howard believes it would be incredibly difficult to caption everything on the site.

Interpreter [00:18:49] “For YouTube, we do work closely, for example, with Google, who, well, owns YouTube. And we’ve talked through this issue. And there are two separate problems at play, one of which is the working retroactively on the millions of minutes of content that is already uploaded to YouTube. Staggers the imagination.” 

Chris Berube [00:19:11] The second issue is the huge amount of new content that’s uploaded every day. 

Interpreter [00:19:17] “I don’t remember the number of hours of videos posted to YouTube every minute, but it’s mind boggling.” 

Roman Mars [00:19:26] YouTube has experimented with ways to encourage more human captioning. In 2018, they introduced a program called Community Captions where volunteers could submit captions for popular videos. Many deaf YouTubers liked the program, but it was discontinued in 2020 because many of the volunteer captures were submitting jokes or promotional material instead of accurate captions. 

Chris Berube [00:19:49] When I reached out to YouTube, they pointed to a couple of new programs they’re rolling out to help people with captioning. One allows users to give subtitle editors permission to fix up their captions. Another allows you to submit corrections if you see an inaccurate caption on certain videos. But ultimately it feels like YouTube is putting all of its chips on AI to fill in the caption gap–just let the automatic captions continue improving until they get closer to total accuracy. 

Roman Mars [00:20:19] To be fair, the automatic captions have gotten a lot better. YouTube would not share accuracy numbers for this story, but according to the Media Hub at the University of Minnesota at Duluth, YouTube auto captions are somewhere around 70% accurate. That’s a big improvement from the early days. 

Chris Berube [00:20:36] Rikki Poynter says she’s noticed the difference. AI captions still don’t include contextual things like, “[The rain is falling softly in the background against the windowpane].” But the AI has become very good at identifying dialog. 

Rikki Poynter [00:20:49] If I were to read the back of this floss package for a video… I don’t know. Somebody’s talking about floss out there–dentists–right? So, if I were to say, “Dentists recommend daily flossing to help remove decay-causing material from between teeth and under gums,” the automatic captions would be able to write that out word-wise usually really pretty well. 

Chris Berube [00:21:12] There are still some major problems the auto captions haven’t figured out. Grammar remains an issue. And the AI is pretty good with words in English, but it really struggles with other languages, especially when it comes to names and places. This is a pretty common complaint about AI programs–the way they can adopt certain biases. 

Roman Mars [00:21:32] And there’s another issue with the captions produced by AI. Many users find them hard to read. 

Rikki Poynter [00:21:39] They move in a more kinetic–that’s the term I learned, like, a week ago–way where it sounds. Like. Words. Are. Coming. Like. This. They show up one word at a time–and that’s not the most accessible way to follow. 

Chris Berube [00:21:58] Still, despite the challenges and the frequent wonkiness, automatic captions just keep getting better. And they will keep improving. I mean, have you been following the terrifying progress of AI recently? There’s an AI program now good enough to pass the bar exam. So how hard are closed captions? Howard Rosenblum thinks the AI captions will compete with human captioners a lot faster than we think. 

Interpreter [00:22:24] “Five years ago, I would have said, ‘Ain’t no way. There is no way ASR is going to get anywhere.’ But now? Like, it’s shocking how much better it’s gotten. In five years, who even knows? I suspect it will be better. Maybe it will be less than five years.” 

Chris Berube [00:22:40] To me, it feels very possible the automatic captions will get close to 100% accuracy, at least for speech. But even if we can get there, the fact is we aren’t there yet. And deaf and hard of hearing people have real accessibility needs right now. People like Rikki Poynter. 

Rikki Poynter [00:22:58] I mean, would I love for automatic captions to be 100%. Absolutely. But it’s hard to know if that’s ever going to be possible. Maybe in the year 3000. I don’t want to wait years and years and years and years for that to happen. 

Chris Berube [00:23:19] Rikki doesn’t go on YouTube much anymore–not because of accessibility. It’s because, frankly, she feels like she’s growing out of it. A lot of her friends have left. It doesn’t feel like such a special community these days. Still, once in a while, Rikki checks in on her favorite accounts. And recently she saw one of her favorite creators had published her first video after a long hiatus. 

Rikki Poynter [00:23:42] Charlie McDonnell. Charlie is so cool. Like, she came back to YouTube. 

Chris Berube [00:23:48] Rikki was excited to watch the video. But as always, she was dreading the automatic captions. To Rikki’s surprise, the video had human captions. They weren’t totally professional. It looked like the YouTube star had done it herself. There were small mistakes, just little things that might drive some people nuts. But for Rikki, it was good to feel like somebody actually cared. 

Rikki Poynter [00:24:12] It was fully captioned–mixed case, full words, punctuation, and all. And it was like, “That’s awesome. I would love to see more of that.” 

Roman Mars [00:24:33] When we come back, is it just me or are TV shows and movies becoming harder to understand? We have an answer after this. Summer vacation season is around the corner. But before you pack your bags, be sure to secure your home with SimpliSafe home security. With SimpliSafe, you’ll feel good knowing your home is monitored around the clock by trained agents ready to act to defend against break-ins, fires, and other threats. SimpliSafe’s Fast Protect Technology enables monitoring agents to visually verify the threat is real and conveys eyewitness evidence to 911 dispatch for fast police or fire response. With the SimpliSafe app, you can arm or disarm your system remotely, check your cameras, or even unlock the front door to let a guest in. SimpliSafe ships to your door and is easy to set up yourself in about 30 minutes. Or you can have SimpliSafe-certified technicians install it for you. Summer is a weird time at the house. You’re gone for long stretches. The kids are coming and going at all hours. And their discipline? Locking the door is not the greatest. SimpliSafe just adds a layer of stability in this chaotic season. Customize the perfect system for your home in just a few minutes at simplisafe.com/invisible. Go today to claim your free indoor security camera, plus 20% off your order with interactive monitoring. That’s simplisafe.com /invisible. There’s no safe like SimpliSafe. It’s celebration season, from weddings to baby showers, graduations to birthdays. Whole Foods Market has what you need to elevate all your events and make them more delicious. When it comes time to fire up the grill, Whole Foods Market offers welfare-certified meat, plus wallet happy prices on organic pickles, organic coleslaw mix, spicy guacamole, roasted salsa verde, and more from 365 by Whole Foods Market. There’s always something delicious in the in-house bakery. And I love that my Oakland store always carries all my favorite local bakeries, too. And if you want to level up your get-togethers, always bring flowers, or get flowers for your own table. Their onsite florist will help you arrange something stylish and striking. Whole Foods is my one stop shop for getting everything done. Start the celebration at Whole Foods Market. The best thing about summer is getting to do all your favorite indoor things outside, like sharing meals and accidentally falling asleep on the sofa. Article’s curated catalog of outdoor furniture is here to help you do all your favorite things this summer. They’ve got everything you need, from outdoor sofas to dining sets to decor. Their team of designers is all about finding the perfect balance between style, quality, and price. And they’re dedicated to thoughtful craftsmanship that stands the test of time. Article offers fast, affordable shipping across the U.S. and Canada. You just pick the delivery time, and they’ll send you updates every step of the way. I have a beautiful Article outdoor dining set. It’s kind of this heavy, wood, three-piece picnic table kind of thing, and it is robust. It is quality. Article is offering our listeners $50 off your first purchase of $100 or more. To claim go to article.com/99, and that discount will be automatically applied at checkout. That’s article.com/99 for $50 off your first purchase of $100 or more. As a business-to-business marketer, your needs are unique. B2B buying cycles are long, and your customers face incredibly complex decisions. Isn’t it time you had a marketing platform built specifically for you? LinkedIn Ads empowers marketers with solutions for you and your customers. LinkedIn Ads allows you to build the right relationships, drive results, and reach your customers in a respectful environment. You have direct access to and build relationships with decision makers. 900 million members, 180 million senior level executives, and 10 million C-level executives. You’ll be able to drive results with targeting and measurement tools built specifically for B2B. Audiences exposed to brand messages on LinkedIn are six times more likely to convert. Audiences on LinkedIn have two times the buying power of the average web audience. Make B2B marketing everything it can be and get $100 credit on your next campaign. Go to linkedin.com/invisible to claim your credit. That’s linkedin.com/invisible. Terms and conditions apply. So, I’m back with Chris Berube. Hey, Chris. 

Chris Berube [00:28:58] Hey, Roman. 

Roman Mars [00:28:59] So do you use closed captioning? Because I use it all the time. 

Chris Berube [00:29:03] For everything. Just all the time. Everything I watch on Netflix. Everything I watch on everywhere, at this point, that it’s available.  

Roman Mars [00:29:11] So why do you use them? 

Chris Berube [00:29:12] Well, I use them because I get confused easily. I don’t use them for accessibility reasons. But I’m certainly not the only hearing person using closed captions. There’s this one recent survey–it was more than 1,200 people–which found half of Americans are using closed captions most of the time. And that number is actually much higher for Gen Z. That’s, like, 70%. So, it’s not just people who have accessibility needs. 

Roman Mars [00:29:37] So why do you think so many people are using closed captions? 

Chris Berube [00:29:40] Yeah, there’s a lot of reasons. I mean, one is that captions are just normal now, right? Like, especially for younger viewers. You see this in a lot of TikTok and Instagram videos–that they have open captions, which means that the captions are on all the time. You can’t turn them off. And this is kind of an aesthetic choice. It’s just become a thing you see in a lot of videos. But going back to the survey, there’s one big reason that most people cite for using the closed captions. And the big reason is that audio sounds “muddled.” So, people are saying modern dialog is more confusing, it’s harder to follow, and that’s why they like the caption so much. 

Roman Mars [00:30:15] Yeah. Yeah, I have noticed this. I call this the “Christopher Nolan Problem” because he has a penchant for having mumbly, lowkey, but very good actors in his movies next to large booming sounds. 

Chris Berube [00:30:26] Yeah, I mean, I love Tom Hardy, but Tom Hardy… I don’t think I’ve ever heard him deliver a line straight–deliver a line intelligibly. 

Roman Mars [00:30:36] And so why do you think that is? I mean, why is dialogue harder to follow now? 

Chris Berube [00:30:40] Well, there is actually a journalist who looked into this recently. 

Edward Vega [00:30:43] All my friends were like, “I can’t understand anything that anybody is saying.” And I was like, “Man, I really wish that there was an answer to this.” 

Chris Berube [00:30:51] So that’s Edward Vega. He is a video producer for the website Vox. And he decided to look into this. Actually, the catalyst for him was watching the movie The King of Staten Island with Pete Davidson. 

Roman Mars [00:31:01] Okay. ‘. 

Edward Vega [00:31:03] It’s the emotional height of the movie. And he says, like, “It’s hard. I think it’ll always be hard.” 

Scott Carlin [00:31:10] I think it’ll always be hard. 

Edward Vega [00:31:12] But he mumbles it, and he says it so quietly that, like–I swear to God–I went back three times. 

Chris Berube [00:31:20] So Ed did this big investigation, and he discovered, like, this is real, this is not in our heads, and there’s a couple of reasons this is happening. 

Roman Mars [00:31:28] Okay. What are the reasons? 

Chris Berube [00:31:29] So a big one is dynamic range. So basically, if your movie has voices in it and your movie also has explosions, you have to make sure the explosions are louder than the voices, right? Like, I mean, that seems obvious. But it’s something where the explosions have to be quite a bit louder so that they have an impact. 

Edward Vega [00:31:46] When you’re mixing a movie, if your explosion is going to be the loudest thing in the mix, then your dialogue can’t be that loud. And so, you have to choose which one you want to move. Do you want to move the dialogue down, or do you want to move the explosion up? 

Chris Berube [00:32:05] One of these issues is this idea of dynamic range. It’s balancing things out so that the explosion has more impact. As a result, the dialogue suffers. So, another big issue is the rise of naturalism in performances. 

Austin Olivia Kendrick [00:32:18] If an actor mumbles their way through a line–if their words start to run together–there’s nothing I can do to fix that. 

Chris Berube [00:32:26] So this is Austin Olivia Kendrick. 

Austin Olivia Kendrick [00:32:28] I currently work as a dialogue editor at Warner Brothers–Warner Brothers Television, specifically. 

Chris Berube [00:32:34] So Edward Vega very nicely introduced the two of us. And Austin says, “Movie technology has changed, and as a result, actors’ performances have changed as well.” 

Austin Olivia Kendrick [00:32:43] Back in the day when sound technology was first introduced to film, that technology was very kind of primitive. The only real option when it came to filming movies with sound was you were on a soundstage, a microphone was planted above your head, and the actors had to stand there and project into the microphone. But as sound technology evolved and all of a sudden, we have wireless microphones that we can place on actors, that kind of loosened up actors’ performances. They now no longer have to stand in one spot and project. And it gave them more freedom and kind of pushed them more into a naturalistic style of performance. And that subsequently could mean mumbling. 

Chris Berube [00:33:33] So there you go. Technology–the second thing–that’s why you’re getting these mumbly performances. 

Roman Mars [00:33:37] Yeah, but for decades and decades I know that they’ve been doing dialogue replacement, where an actor comes in after the fact and records dialogue to make it more clear. 

Chris Berube [00:33:47] Right. Yeah, that’s ADR. And I did talk to Austin about this. And he says doing ADR is actually pretty difficult, especially when the dialogue is very mumbly. 

Austin Olivia Kendrick [00:33:56] When we bring actors in to rerecord lines, they have to match their performance to the way that their lips are moving onscreen. And if they mumbled their way through that take, their lips aren’t moving very much. So, if they come in and rerecord a line with a lot more diction and a lot more separation between the words, that’s not going to match. 

Chris Berube [00:34:18] The last issue–and this might be the biggest one–is something called “downmixing.” So, this is the idea that a lot of movies and some TV shows are mixed for the best possible sound system, right? Many of them are mixed for Dolby Atmos. That’s the special sound system you get in certain theaters that has 128 channels. So, you can mix something in a very, very specific, particular way. And then after you’ve mixed it for the best sound system, you have to remix it for other platforms, like a television or a phone. And that is when you start to get into trouble. 

Roman Mars [00:34:52] You’re taking 128 channels, and you’re compressing that down into different formats with lesser channels. 

Edward Vega [00:35:00] And most of the time when you’re watching something on your TV, unless you have a surround sound system, it’s going to be stereo or mono. So, if you’re going from 128 down to two or down to one, it gets muddy. It gets a lot muddier. 

Chris Berube [00:35:17] So lots of movies and TV shows–they’re getting mixed on the sound system, then they’re getting downmixed for other formats. And Austin says usually the downmix at the end–that’s the last thing they do in the editing process. And sometimes they don’t put quite the same care and effort into it that they would for the initial mix for a movie theater. 

Austin Olivia Kendrick [00:35:36] Yeah, a lot of the time, when it comes to that kind of sound mixing, often studios do not want to spend the money because ultimately time is money and rerecording mixers are very expensive to hire for that extra time. So oftentimes they want to put as little money into that area as possible. 

Chris Berube [00:35:57] So, Roman, there you have it. It’s downmixing. It’s naturalistic performances. It’s this dynamic range issue. Those are some of the reasons why you did not understand the movie Tenet by Christopher Nolan.

Roman Mars [00:36:10] Well, they’re not the only reason I didn’t understand Tenet. But that helps. 

Chris Berube [00:36:15] Yeah, the time travel stuff also contributes to it, I guess. There’s a lot going on, working against Tenet being a comprehensible film. So, Roman, I have to say before we go, the video by Edward Vega on Vox–it goes into so much more depth on this topic. It’s wonderful. I hope everybody checks it out. And also, I hope everyone checks out Austin, Olivia Kendrick’s TikTok, if you use that platform. She explains movie sound, and it is so fascinating. 

Roman Mars [00:36:43] That’s awesome. All right. Thank you so much, Chris. 

Chris Berube [00:36:44] Thanks, Roman.

Roman Mars [00:36:51] 99% Invisible was produced this week by Chris Berube, with editorial input from Linda Besner. She originally wrote about this topic for The Atlantic magazine. Linda is writing a book about people and their complicated relationships with money – look for it soon at a bookstore near you. This story was edited by Kelly Prime. Original music by Swan Real. Sound mix by Martín Gonzalez. Fact-checking by Graham Hacia, who couldn’t independently verify that Crocamole loved guacamole, but we assured them that this was true, and we went with it anyway. Kurt Kohlstedt is our digital director. Our intern is Avanti Nambiar. The rest of the team includes Senior Editor Delaney Hall, Christopher Johnson, Jayson De Leon, Emmett FitzGerald, Vivian Le, Jeyca Maldonado Medina, Joe Rosenberg, and me, Roman Mars. The 99% Invisible logo was created by Stefan Lawrence. Special thanks this week to Caroline Mincks and Anne Ballentine. We are part of Stitcher and SiriusXM, now headquartered six blocks north in the Pandora Building… in beautiful… uptown… Oakland, California. You can find the show and join discussions about the show on Facebook. You can tweet me @romanmars and the show @99piorg. We’re on Instagram, Reddit, and TikTok too. You can find links to other Stitcher shows I love as well as every past episode of 99PI at 99pi.org.

Leave a Reply

Your email address will not be published. Required fields are marked *

All Categories

Minimize Maximize

Playlist