The Weather Machine

Roman Mars:
This is 99% Invisible. I’m Roman Mars.

Roman Mars:
Andrew Blum is a journalist who writes about some of the biggest infrastructure projects in the world. His specialty is revealing how systems we think of as intangible, like the internet, are actually made up of very real stuff. The internet relies on cables and wires and data centers which are maintained by actual people who keep the whole thing running. A few years back, Andrew got interested in the weather forecast. It’s this mundane everyday service, that like the internet, is made possible by a vast and interconnected global machine that took decades to build. This system is a huge scientific project, but it’s also a diplomatic one. The atmosphere crosses all political boundaries. And so, knowing the weather requires international collaboration. As weather becomes more extreme, the forecast becomes increasingly important. But ironically because of its growing value, there are now forces threatening to undermine the global system that makes it possible. It is fascinating stuff. I talked to Andrew about his book, ‘The Weather Machine,’ and he told me that he first got interested in the forecast back in 2012.

Andrew Blum:
It was a kind of busy season for me. My first book had come out, my dog had died, my son was born. It all kind of happened at once and there was a weekend afternoon, a kind of Sunday in October when I had my kind of newborn in one hand and my phone in the other. And I was on Twitter and all of a sudden the meteorologist who I followed kind of went into a tizzy. They all just kind of erupted all at once based on the output of a weather model. And what they were seeing was a storm that they had kind of been watching out in the Atlantic and the Southern Atlantic, but suddenly it was going to turn left towards New York where I live.

Roman Mars:
Yeah.

Andrew Blum:
And it was remarkable because this was eight days ahead. This was a big storm, potentially. And they all kind of trusted the output of this model. They weren’t saying this is definitely going to happen, but it was so far ahead of sort of what I understood as the work of meteorologists, especially kind of hurricane forecasters.

Roman Mars:
And this storm that you could kind of see eight days out eventually became Hurricane Sandy.

News Report:
“It is chaos along the Jersey shore.”

News Report:
“The superstorm already stretching across one-third of this country from Florida to Canada.”

News Report:
“I came to ABC News, I covered New York weather for 25 years. I have never seen water in lower Manhattan. There is water now on the streets in lower Manhattan.”

Andrew Blum:
I mean the overall feeling when the storm actually came was that our kind of luck had run out, that New York city had sort of finally begun to reckon with what the storms of the future might be like. With the subways flooded and shut down, nobody did anything for that week. Along the coast, it was months and years and if you live on the L train, that’s still being fixed. These are, the consequence of it was really clear, I mean 147 people were killed. But when it came, for me it was a recognition that, that forecast eight days ago was right, that an eight-day forecast is not stuff of science fiction, but had just happened in the most consequential way.

Roman Mars:
And the real difference here between the idea of knowing a thing that’s coming in the way that knowing a cold front is coming or knowing that a tornado is coming is that Hurricane Sandy didn’t exist eight days before. It was just particles in the atmosphere moving around. And it was a mathematical model that predicted that it would form into this thing that would affect people so dramatically.

Andrew Blum:
I mean, the experience of Sandy made me want to know not only what the weather models were, but where they came from, sort of who built them, how they had evolved over time. I recognize them as this kind of complex global infrastructure, but as is often the case with complex global infrastructures, their authorship was really vague and longstanding.

Roman Mars:
Right. And so you’ve spent a lot of time thinking about physical infrastructure and there’s something about the weather forecast then it kind of has this vibe of it’s about mathematical models and in physics and stuff in the air, but it still really is rooted in infrastructure. It kind of dovetailed with the thing that you already care about. So what does the modern weather machine actually physically look like?

Andrew Blum:
Well, to kind of see it like that, you kind of have to have this hallucination about of a sort of planetary scale. It’s made up of so many kind of tens of thousands of tiny, tiny little pieces. I always like when you’re flying out of LaGuardia Airport in New York, if you’re lucky, you kind of pass by the weather station there by the runway and it looks like a kind of jumble of equipment and that’s one piece of the weather machine. When we see satellite pictures, the kind of familiar weather satellites, that’s kind of another piece of the weather machine. And then that’s repeated, tens of thousands of times all over the world.

Roman Mars:
Yeah. So as you started looking at the history of the weather forecast and how it started, you found that it was actually a revolution in telecommunications that made the first weather forecast possible. So tell me why the development of the telegraph was important for understanding the weather.

Andrew Blum:
It’s really about having this picture of the earth across space. We have maps, that’s the kind of one way of imagining the earth. But until you can communicate instantaneously across distance, basically until you have the telegraph and then all of the communications technology that comes after it, you can’t really know what’s happening simultaneously in many places all at once.

Roman Mars:
Right.

Andrew Blum:
And it turns out the kind of first step towards knowing what the weather is going to be in one place at many times is knowing what the weather is at one time in many places. That’s the kind of key to it. And so you end up as soon as the telegraph is invented and as soon as there’s a kind of rudimentary telegraph network, the telegraph operators begin sending messages to each other about the weather conditions and they quickly realized that especially in the U.S., the weather is often moving from West to East and they can give some advance notice of what’s going to happen that afternoon based on if you’re in New York, what it’s doing in Ohio. And that kind of basic sense that you could move faster than the clouds, that the news could move faster than the clouds, begins to open up this idea of a kind of holistic view of the planet. Suddenly you can kind of imagine yourself looking down not just on a map as a political idea, but really live, seeing how the weather is changing over space.

Roman Mars:
And so in the 1840s, the Smithsonian Institute takes this sort of grand theoretical idea and turns it into an actual map, which is a kind of beautiful, quirky analog, fun thing that I loved your description of. Could you describe the map and how it functioned?

Andrew Blum:
Yeah, as with any corporate or government headquarters, when they built their new building, the centerpiece in the lobby of the Smithsonian Institution on the Mall in Washington was a big map of the fledgling United States up on the wall, pre-Civil War 1840s. And whenever they got a report in from their Smithsonian observers, they’re kind of brand new network of weather observers, they would put a little paper disc up at the disc, would have the temperature, it would have a different color for the weather. So white for fair weather, black for rain, brown for clouds, blue for snow. And so when you arrived at the Smithsonian, you could look up at the wall and you could see what the weather was across the country and you could begin to have that first inference of the weather of the future, the forecast based on how those patterns might be changing.

Roman Mars:
Yeah. So then we get to the 1870s when there’s an international coalition forming to expand the weather forecast. And people are starting to think about how to collect and share weather data more widely.

Andrew Blum:
I mean from the beginnings of essentially international networks of any kind – in the 1870s, you have international telegraph networks, the postal union is formed, you have the meteor convention – there’s this really this kind of vogue for standardization. And a big part of that is the recognition that if you have brand new national weather services, they need a common language for communicating their observations with each other. And we’re each going to maybe make our own forecasts, but certainly knowing what the sky is in your country is useful to my country. And that kind of basic sense of meteorology is a common good of the Earth’s atmosphere as continuous. It really becomes part of meteorological culture from the beginning. They are very good from a very early stage at cooperating with each other.

Roman Mars:
And so it becomes as much of a diplomatic project as a scientific one.

Andrew Blum:
Yeah, yeah. Absolutely.

Roman Mars:
So how do people go from gathering data about the weather to actually doing something about it? Where do they actually start to look into the future of what was to come?

Andrew Blum:
Well, the first person who kind of codified the process that has become the weather models as we know them today was a Norwegian meteorologist named Vilhelm Bjerknes. And it was in the 1890s that he first began to play around with the idea that you could treat the weather forecast as a hypothesis, as a kind of mathematical hypothesis, that if you could calculate the weather, if you could calculate the evolution of the atmosphere (its temperature, its pressure, its wind direction) and you could do that mathematically, then you could be quite sure the next day if you were right or wrong. And if you were wrong, you could begin to refine your equations and then do it again the next day. Or you could even go back and use the previous days observations and calculate it again.

Roman Mars:
Right. But the mathematical models, how complex are they and how far in the future can he really look at this point?

Andrew Blum:
Well, his basic equations, which are now kind of known in meteorology, is the ‘primitive equations,’ which I kind of love. His basic equations were right, but he couldn’t solve them. He neither had enough observations, especially at different levels of altitude and high up into the atmosphere, nor could he solve the differential equations required to sort of solve his own equations. He couldn’t actually plug the numbers in. So theoretically he was mostly right. And in fact, the primitive equations are still at the root of the weather models. They’re deep in there. They have evolved dramatically, but they’re still there. They’re still relevant. But practically he got nowhere. He neither had enough to put into his math nor was he able to actually calculate what came out.

Roman Mars:
Yeah. So then people began to imagine these ways to get around this issue of the computation. So a mathematician named Louis Fry Richardson had this crazy idea that I want you to tell us about.

Andrew Blum:
Yeah, so Bjerknes writes his paper in 1904. He says that we can predict the weather using math and physics. And about 10 years later Lewis Fry Richardson, an English mathematician, comes to it and says, well I think I might actually give this a try. And he actually uses a set of observations that Bjerknes himself would organize the collection of from a single day above Europe. And he begins this sort of furious six-week process of actually calculating that into a weather forecast. And he does it while he’s working as an ambulance driver on the Western Front during World War One.

Roman Mars:
Wow.

Andrew Blum:
He was a Quaker so he wouldn’t fight, but he drove an ambulance. And so he talks about going back to his billet and sort of running the calculations with his slide rule and spending six weeks on this sort of single afternoon’s forecast, which famously and spectacularly was wrong, sort of famous errors in meteorology. But he was convinced that if he had better observations and if he had a greater ability to actually make these calculations, you could have a useful forecast.

Andrew Blum:
And he comes to the idea that what it would really take would be 64,000 computers, which is to say, 64,000 humans (human computers) arranged in a stadium and there would be a conductor in the middle who would shine a light on them if they were going too fast or too slow and they would write their calculations and then pass it to the person next to them. And with 64,000 people, you could go fast enough to have a useful weather forecast, which is to say a forecast that is completed before the weather actually arrives.

Roman Mars:
In one day.

Andrew Blum:
I mean that’s the thing, you can have a very detailed forecast, but it’s useless if the future comes before your calculations. He also somewhat amazingly predicts like the Google campus. He thinks that like his 64,000 computers should have like ball fields and cafeterias and entertainments and things like that. And he also predicts the kind of steampunk aesthetic as well. He describes these offices with like, levers and desks and things that rise up on to roof decks. And basically what Facebook is in Menlo Park today.

Roman Mars:
That’s so funny. So we have these people thinking in these big ways about the weather and how to forecast it and we have these couple of limitations that they’re budding up against. One is computational limitation. The other one is kind of data limitation, like access to measuring these points. So how is weather forecasting moving forward in the rest of the world and what were they doing to come up with what was going to happen in the future.

Andrew Blum:
When Richardson and Bjerknes, when their project essentially fails, there’s this kind of amazing and pretty successful basically 40-year history of meteorology that actually makes a lot of progress where the weather forecast gets better and useful and helps with early aviation. Most famously, the forecast for D-day was a solid two-day forecast that allowed the allies to postpone their invasion. It’s sort of always pointed out as this kind of forecast that changed the course of history, but none of it had anything to do with these calculations.

Roman Mars:
It’s sort of the equivalent of like looking at a cloud or a cold front and just seeing it go across the country and it has not a lot of math in it but as a lot of just like history and past precedent and stuff that lets you predict what’s going to happen in the future.

Andrew Blum:
Yeah, yeah, absolutely. And it wasn’t until the post-war era when you have the beginnings of space flight and the beginnings of real computing that the idea of actually functionally creating a weather forecast based on mathematical analysis of the atmosphere becomes possible again.

Roman Mars:
So after the war we sort of get into the ’50s and ’60s and there’s a big breakthrough. So new technologies emerge and there’s the political will to build this whole earth map and make it really, really good. So tell us what happens in the ’60s that makes Bjerknes’s dream of calculating the weather finally come true.

Andrew Blum:
The most important thing is you have this kind of love affair with the earth, with the earth as a planet. You suddenly have this collective societal vision of what the earth will look like from space. You have all this science fiction, you have the first people orbiting the earth and everyone’s sort of imagining what it is like to look back. And as soon as you kind of have that in the popular imagination, the idea of a map of the complete atmosphere becomes real.

John F. Kennedy :
“We go into space because whatever mankind must undertake, free men must fully share.”

Andrew Blum:
There’s this incredible moment in 1961 right after the Soviets first launch Sputnik, where Kennedy gives a speech where he says, we must put a man on the moon before the decade is out.

John F. Kennedy :
“… provide funds which are needed to meet the following national goals. First, I believe that this nation should commit itself to achieving the goal before this decade is out, of landing a man on the moon and returning him safely to the earth.”

Andrew Blum:
And that’s point number one. And it turns out point number three is $75 million for weather satellites.

John F. Kennedy :
“… will help give us at the earliest possible time a satellite system for world-wide weather observation. Let it be clear…”

Andrew Blum:
As familiar as that man in the moon line is, the line about weather satellites comes like 30 seconds later. And for Kennedy, the global view of this was kind of part of the larger project of the triumph of American ideals around the globe otherwise. So you have this sort of moment where all of the kind of Imperial ideas of kind of American view of the globe and an American dominance of the globe become wrapped up in a view of the atmosphere for scientific good, for meteorological good, for this sort of what we now think of as this banal project of creating better weather forecasts.

Roman Mars:
So JFK’s vision came true in many ways. Throughout the ’60s and ’70s, a lot of satellites went up into space both for military surveillance and for weather forecasting. And as the weather machine grew, a worldwide alliance developed between nations. They figured out how to share data and how to maintain the infrastructure that they’d collectively built. The main part of the UN that now deals with weather is called the World Meteorological Organization. And they get together every four years to talk about policy. And Andrew went to one of these gatherings.

Andrew Blum:
In 2015 in Geneva, the World Meteorological Congress is the big event every four years and it’s the world’s weather diplomats coming together and sort of methodically kind of hashing through their issues and then breaking for receptions, which is the diplomatic word for ‘party’ as it turns out. It’s mostly very specific and technical. But the dynamic between the countries that essentially run supercomputers and the countries that don’t was increasingly apparent. And not surprisingly, the effects of climate change are sort of more pronounced for less wealthy countries, which are also the countries that don’t fly weather satellites and run weather supercomputers. So there was really a sense that they were all in it together, that this was a kind of thing that governments did. And there was a 150-year tradition of governments around the world sharing their data with each other, sharing their forecast with each other. And especially now when storms are more powerful, when the effects of those storms are more pronounced, when there is the sort of growing threat of what will happen with the weather in the future, it was very clear that that cooperation was needed now more than ever.

Roman Mars:
There’s this whole notion the weather machines is this global project that’s carried out by governments, that’s done for the public good, but increasingly private companies are getting into the weather forecasting business. So tell us about that and how this is interacting with the sort of global project that’s been going on for decades and decades.

Andrew Blum:
Well, there has been the assumption, essentially since the birth of satellites and computers, that supercomputers and satellites are things that governments do. They’re too expensive for private companies to do. If you have a weather service and you need a $30 million computer, that’s going to be something that a government buys and it’s going to be in service not only to its citizens but to the entire world. But a couple of current are colliding. I mean one you have the sort of rise of private space flight. You have the kind of Space X’s of the world and you have private space observation companies. You have more severe weather and more money at stake to predict that weather and you have a rise in the recognition of big data and what we can do with data and how important it is to sort of understand the world using big data. And so you end up now with the idea that private weather forecasting is probably a pretty good business. And so after 150-year tradition of weather forecasting being something that governments do for their citizens, there’s now a bit of a gold rush where companies can run their own weather models, can fly their own weather satellites, can collect their own weather observations and provide a private forecast that is a value that exceeds the usefulness of the publicly available forecasts.

Roman Mars:
And how do you think about that as someone who’s seen the long view of the weather machine, it turning towards privatization? What do you think are the complications like now and maybe the complications in the future?

Andrew Blum:
Well, I mean the first thing that I saw was the real angst among the sort of ‘dyed in the wool’ government meteorologists over what this meant for the long tradition of government weather services protecting life and property, and that being something that governments do for their citizens. But, of course, from a technical standpoint, you have the possibility of even better weather forecasts. And so there’s certainly a kind of technological thrill with the idea that this could be improved, but it’s not hard to recognize the kind of global inequality suddenly appearing in the technology of the weather forecast itself. And this real exacerbation of the effects of climate change, when you have hurricane forecasts accessible to the rich before they are accessible to the poor, and when of course they will affect the most vulnerable more directly.

Roman Mars:
We get to the sort of the crux of this, which is at this moment, being aware of the new extremes when it comes to our climate is more important than ever. We’re at this moment where privatization and preparatory data and models could break that apart.

Andrew Blum:
And it wouldn’t take much is the sort of strange thing. All the weather observations that are collected by the U.S. Government are put kind of right in the global bucket. And in exchange, we get all the world’s weather observations back. And if, for example, the National Weather Service decides to buy private satellite observations for one category and that company says, ‘no, you can’t share that,’ and that spigot is it is turned off, the possibility that other – start with European countries – will say, ‘well if you’re not giving us that data, we’re not giving you our data.’ And within two or three days, the entire system falls apart. And it’s not as if, we only need observations over the United States for forecasts over the United States. As soon as you’re past three or four days, you need that entire global view. And all of the weather models are kind of built on that holistic global view. And so the idea that this is within our borders, that this is a kind of local issue, that it isn’t entirely international interdependent, is preposterous. And, of course, it’s deep in the kind of global order that the U.S. built up in the second half of the 20th century. It is the kind of American ideal of leading the world with technology and cooperation. And at the moment, and not only in the Trump era but really over the last 10 years, particularly with the kind of new technological dominance of the U.S., with the Googles and Facebooks of the world, that the idea of this sort of proprietariness of this data as something that is deep in the heart of our system becomes more consequential in the way that we put together weather forecasts.

Roman Mars:
I think one of the things that’s fascinating about all this and all the work that you’ve done and my thinking on it that has evolved since reading your book, is this weird mixing of the idea of weather and knowing the weather being so kind of the now and every day and how much it’s about little tiny decisions about whether you bring an umbrella and also about hurricanes. It kind of gets your mind reeling in this very strange way about the human desire to know what’s coming.

Andrew Blum:
Yeah. This book took me several years to write. And in the course of writing it, my older child, my daughter, went from kind of a toddler to like a proper elementary school student. And at the beginning, I would be working on this and she would say, what’s it going to be tomorrow? Like that would be kind of her last words before going to sleep. She meant like what are we doing. But I would be like, oh, how do you consider the future? What does it mean that the sky is coming this way and I’m sort of rooted in place in time and how is this going on? And so I kind of heard it as like what’s the weather going to be tomorrow? And that sort of contrast between, my watch is going to tell me what the weather’s going to be tomorrow and that’s just super easy and no worries. And then the existential dread of what’s it going to be tomorrow was right there with it. It’s the most banal thing. It is the ultimate small talk.

Roman Mars:
Yeah.

Andrew Blum:
And yet it’s also, of course, the core of our existential planetary dread, and this is in some ways the sort of parable of climate change as well. We can be pretty sure about what’s going to happen and the ability to change it or to do something about it is completely independent of that foresight.

Roman Mars:
In other words, we have the information. We can effectively see into the future. But what we do with that information and how it is used for planning and preparation is up to us. To find out more about Andrew Blum’s book, ‘The Weather Machine,’ go to 99pi.org. We’re going to visit a tiny island in the North Atlantic that’s a tiny cog and the gigantic weather machine, after this.

[BREAK]

Roman Mars:
One of the most fascinating things about the entire system used to protect the weather is how reliant it is on space-age satellites orbiting the earth, and hundreds of more humble weather stations located on the land. Both technologies are needed to inform our global view because the weather machine is so vast and made of so many parts, no one thing exemplifies at all, but I asked Andrew Blum if he could zoom in on one of his favorite places that’s essential to the whole data gathering apparatus.

Andrew Blum:
When you try to kind of peel open the weather machine and see what it’s made of, you end up with this kind of challenge of choosing a single place to represent all the places, which of course is kind of impossible. You know, it is the nature of places that they are all different. They all occupy a kind of different spot on the map, partly because of Bjerknes and partly because of this Norwegian meteorological tradition. I latched on to Norway’s system of weather observation, and in fact, I fell completely in love with this island called Jan Mayen that’s this Arctic Island way off, you know, kind of towards Greenland that only has a weather station on it with like an army crew. They get serviced a few times a year and like a couple of Huskies, and it sounds like this really kind of incredible wild place, which needless to say, you can’t go there. Like if you go, you have to like go for three months.

Andrew Blum:
So I kind of abandoned that dream of actually visiting this weather station, but found instead a place called Utsira which is a sort of small Island off the coast of Norway. And when I say off the coast of Norway, I mean, it’s like a 25-minute ferry ride, like, you know, no big deal. It runs a few times a day, but because of its location in the North Sea, it has been an important weather observation point, basically for 150 years. And so you have a very early telegraph line there and you have a single spot kind of up on the top of the hill in the center of the island that consistently has been the point where the Norwegian weather service has observed the weather.
It’s a very windy place. Utsira is known for its birding and for its winds. And when you’re there, you realize, I realized what it meant for the kind of wind to be rushing by this single point. And I realized that that’s kind of what wind is. You know, wind is the passage of the atmosphere past a single spot. And you know, that has to then be tied back into the kind of global computational system. You know, you need to sort of send word back to Oslo and Oslo needs to send word back to Frankfurt where the sort of European collector is, and then that gets sent to Virginia. And the entire thing kind of gets networked together into typing in, you know, “Utsira” on Google, and then the temperature shows up. But all of those things have to fit together, and that system has to be deliberately designed and the kind of design of that system goes back to the middle of the 19th century with a sort of first recognition that not only was it useful to know what the weather was and other places, but it was suddenly technologically possible to get that news pretty speedily.

Roman Mars:
Right. And it’s gathered by a human or tended to by humans.

Andrew Blum:
It’s tended to by a human. Yeah. He runs the restaurant and he does the weather observations, so it’s four times a day. He goes on his back stoop and he has a cigarette and he kind of looks at the sky and then he goes to his computer and he logs in and he, in the kind of Norwegian weather services dropdown menus, he sort of does a qualitative analysis of what the clouds are according to the sort of rules that he’s been taught. And that gets them sort of put in the whole way. And the same thing happens in every airport in the country. Every major airport has a round-the-clock weather observer. Some person who’s like in an office somewhere on the grounds of the airport who, you know, once an hour is checking the observations that the automated system has made to make sure the system’s working and if the clouds are slightly different than the ceilometer the cloud observing machine can read, to correct those.

Roman Mars:
Andrew Blum is the author of “The Weather Machine,” a journey inside the forecast.
99% Invisible was produced this week by Delaney Hall, mix and tech production by Sharif Youssef, music by Sean Real. Our senior producer is Katie Mingle. Kurt Kohlstedt is the digital director. The rest of the team is Emmett FitzGerald, Vivian Le, Joe Rosenberg, Chris Berube, Avery Trufelman, Sofia Klatzker, and me, Roman Mars.

We are a project of 91.7 KALW in San Francisco and produced on radio row in beautiful downtown Oakland, California. You can find the show and join discussions about the show on Facebook. You can tweet at me @romanmars and the show @99piorg. We’re on Instagram and Reddit too, but we should really talk about the weather @99pi.org.