99% Reality (1% Augmented)

Roman Mars:
This is 99% Invisible. I’m Roman Mars.

Roman Mars:
Our modern world is determined by them: the architects, the designers. They decide how you experience the physical world. But the ubiquity of smartphones with GPS, a compass, a network, and more computing power than we know what to do with, has created this new space, where a new reality can lie on top of this physical, real world. It’s an augmented reality and it adds a whole new level of creativity and design to the everyday world. Most augmented reality uses the smartphone’s camera. You turn on the application, you press the augmented reality button and then you look through your phone’s camera at the now labeled and visible stars and planets in the daytime sky. Or another app could pop up cool infographics with details about a bridge or a building. It’s still in its infancy, but the possibility of experiencing a hidden world on top of the world in front of you is tantalizing and so full of potential. But I personally like to augment my reality with sound. And that brings us to the musical smartphone app called RjDj.

Michael Breidenbrücker:
We’re combining the world around you with your listening experience.

Roman Mars:
That’s the founder of RjDj, Michael Breidenbrücker.

Michael Breidenbrücker:
So you know, the door closes and something really amazing acoustically could happen.

Roman Mars:
Robert Thomas is one of RjDj’s composers.

Robert Thomas:
RjDj creates what we call reactive music. Which is a new format of music which changes in relation to what you’re doing.

Roman Mars:
RjDj is comprised of scenes which are analogous to songs. But they’re more like instructions for songs. There are bits of music, code for the phone’s own built-in synthesizer, audio filters, and these are triggered by the user according to the composer’s plan for the scene

Robert Thomas:
I call this ‘quantum composing.’ I think of all these different possible ways that they could be listened to and try to come up with lots of different ways that will create something very interesting to happen in the music. When a sound coming in from your surrounding is very loud, you could have lots of really high notes kind of going, “doo-doo-doo-doo-doo-doo-doo!”

Roman Mars:
So sounds from the mic change the song, but so does the accelerometer in the phone, that’s the thing that senses movement, time spent listening can be a factor, and even GPS data can be incorporated. And all these stimuli work in concert to create a unique augmented reality. And the way you experience your city will be altered from its basic form. I mean you’ve walked around with music playing in your headphones and felt those moments of musical synchronicity that changed how you saw and felt about your surroundings. Well, imagine that times a thousand. Right now RjDj is mainly ambient pop and structured noise, but imagine what it could be. The world will become a playground for interaction designers.

Robert Thomas:
It kind of feels like you’re seeing some inner beauty in everything. The most famous one that people get that from is a scene called ‘the eargasm.’ This kind of harmonic overlay of reality.

Roman Mars:
I had my wife, Mae, put on the patented white earbuds to check out one of the scenes for the first time.

Mae Mars:
“Woooaaaahhh. Doo-doo-doo-doo-doo! It’s what a steel girder would hear if a steel girder could hear. It’s kind of like being in a horror movie, but not quite horrible.”

Roman Mars:
Rather than putting on a set of headphones to block out the world, RjDj listeners will plug into a constantly changing, unique soundscape that connects them to what they are doing and where they are in the world. I imagine walking around San Francisco, listening to a mashed-up composition of punk and hip-hop that gives me a history of the neighborhood I’m traveling through. And the GPS will trigger a sound clip from Alfred Hitchcock’s ‘Vertigo’ when I pass through Nob Hill.

[Clip from ‘Vertigo’ plays.]

And the tempo will speed up as I scurry out of the way of an obnoxious bike messenger into the path of an unnoticed oncoming cable car.

[Cable car bell rings.]

My sudden lack of all movement as I lie on the pavement will elicit a pleasing harmonic drone from the iPhone earbuds, which will mix beautifully with the sirens from the paramedics as they approach.

Roman Mars:
99% invisible is produced by me, Roman Mars, with support from Lunar. It’s a project of KALW, the American Institute of Architects in San Francisco, and the Center for Architecture and Design.