Of Mice And Men

If you are looking at a computer screen, your right hand is probably resting on a mouse.  To the left of that mouse (or above, if you’re on a laptop) is your keyboard. As you work on the computer, your right hand moves back and forth from keyboard to mouse. You can’t do everything you need to do on a computer without constantly moving between input devices.

There is another way.

10616196_10154835436925223_1491661547919992466_n
A “keyset.” Courtesy of SRI International and the Doug Engelbart Institute

A device called a “keyset” could help us navigate virtual environments without moving your hands back and forth. With the mouse in your right hand, the keyset would occupy your left hand. Its five buttons resembled piano keys.

Used in tandem with a mouse (specifically, a three-button mouse), the keyset enabled the user to type out all the characters of the alphabet, and execute shortcut commands. The keyboard, thus, would be secondary — perhaps even irrelevant — meaning users could keep their eyes the screen, and not glance down at their fingers.

This could be the way of the future. But it’s actually the way of the past.

1395899_10154835437185223_8587998942824671563_n
Courtesy of SRI International and the Doug Engelbart Institute

Doug Engelbart invented the keyset in the 1960s. Engelbart was also the person who invented the mouse.

10615512_10154835435215223_2654485253578840072_n
Doug Engelbart with a three-button mouse and keyset. Courtesy of SRI International and the Doug Engelbart Institute

Engelbart’s first mouse was a block of wood about the size of three decks of cards stacked on top of each other. It pivoted atop a metal wheel, and had three buttons.

IMG_1781
Engelbart’s first mouse. Engelbart quickly added two more buttons. Credit: Luisa Beck

Not satisfied merely pointing and clicking, Engelbart imagined that his mouse would be used in combination with a keyset to execute all kinds of commands that today are difficult to imagine doing without a keyboard.

Take word processing, for instance. The five-button keyset could produce all 26 letters by memorizing combinations of these keys used together. Learning to type would take a lot of practice, but Engelbart believed that with lots of repetition, the muscle memory would take over.

8960_10154835437145223_3692785734764473770_n
Courtesy of SRI International and the Doug Engelbart Institute

Even Doug Engelbart realized that learning the keyset was difficult. But for Engelbart, ease of use wasn’t the top priority. He wanted the computer inputs to be as powerful possible, and that required some complexity. He imagined that consumers would learn how to use the mouse and keyset slowly over time, like how one learns to operate a car.

The virtuosity and discipline required for Engelbart’s mouse and keyset illustrate his larger ideas about what computers had the potential to do. Engelbart thought computers could help us communicate and collaborate — a total departure from how most engineers thought of computers in the 1950s, which was just as giant calculators.

64393_10154835437155223_3134956238556339341_n
Courtesy of SRI International and the Doug Engelbart Institute

In the early 1960s, Engelbart got funding to start his own lab. His team of researchers started building entire online collaboration systems, experimenting with video conferencing, collaborative text editing, outlining tools, hyperlinks, and, of course, devices like the mouse and keyset. Here’s Engelbart showing off some of his ideas in something that is now referred to as the “mother of all demos.”

These complicated ideas weren’t particularly popular or marketable, and by the late 1970s, Engelbart’s team ran out of funding to continue their research. Many of Engelbart’s researchers migrated to Xerox PARC (one of the most cutting-edge computing research institutes in its time), as did many of Engelbart’s prototypes.

And that’s how, in 1979, Steve Jobs first saw the mouse (and keyset) when visited PARC.

A dramatization of Steve Jobs’s visit to Xerox PARC in the film Pirates of Silicon Valley.)

Although Jobs found the mouse-and-keyset setup intriguing, he thought it was way too complex. Jobs believed that computers should be as simple and user-friendly as possible. Per Jobs’s directive, Apple’s version of the mouse would have only one button.

Apple_Macintosh_Plus_mouse
Courtesy of the All About Apple museum

Apple never even considered including Engelbart’s keyset in early personal computers. The keyset was too costly, clunky and complicated, and Jobs knew that consumers didn’t want to buy a computer they’d have to learn how to use over the course of many months.  Jobs wanted customers to come into an Apple store, look at their products, try them out, figure them out in the first few minutes and buy them immediately. Even today, Apple products are so simple that a toddler can operate them perfectly.

Englebart used to compare the sleek, simplified Apple products to a tricycle. You don’t need any special training to operate a tricycle, and that’s fine if you’re just going to go around the block. If you’re trying to go up a hill or go a long distance, you want a real bike. The kind with gears and brakes– the kind that takes time to learn how to steer and balance on.

10301175_10154835435395223_7648627696756390032_n
Photo courtesy of SRI International and the Doug Engelbart Institute

Englebart thought that because the consumer market prioritizes simple and “user-friendly” devices over more complex and “learnable” devices, they’ve only been selling us tricycles.

IMG_1783
Christina Engelbart. Credit: Luisa Beck

Although, without people like Jobs, perhaps only a few people would be on bikes, and the rest of us would be too intimidated to get on wheels at all.

Though the ease-of-use paradigm has mostly won out, Engelbart’s philosophy continues to influence technologists today. A device called a monome — used mostly for music — is complex enough that even its creators haven’t totally mastered it. One of them, Brian Crabtree, traces its lineage to Doug Engelbart.

The best design may be the one that gives us a clear path to learning if we choose to. Put another way, designs that help us transition from tricycle-riding to bicycle-riding, so that if we want, we can choose to go up some really big hills.

968882_10154835435765223_8775700294040222960_n
Courtesy of SRI International and the Doug Engelbart Institute

Credits

Production

For this episode, producer Luisa Beck spoke with Christina Engelbart, executive director of the Doug Engelbart Institute, and Larry Tesler, who worked at Apple from 1980 to 1997 as VP and Chief Scientist.

Special thanks to Marc Weber, The Computer History Museum, Thierry Bardini, Bjoern Hartmann, Eugene Kim, Joe Blaylock, Jim Yurchenco, Glenn Fleishman, Harvey Lehtman, Martin Hardy, Caroline Rose, Lora Oehlberg, Wesley Willett, Myles Byrne, and Fred Turner for their help researching this story.

Music

“Yesterday” — Lullatone
“Pretty Eyes” — Jahari Massamba Unit
“Line” — Portico Quartet
“Way Out of Living” — Linear Movement
“Space & Time” — R.M.C.
“Who’s Really Listening” — Mark Lane
“Begleitung Fur Tuba” — Ursula Bogner
“Knee Play 1” — The Philip Glass Ensemble
More to come

  1. I always thought Photoshop was a pretty good example of transition-friendly software. You can open it up with no experience and the layout is pretty welcoming, the interface is mostly fairly straightforward, and though it’ll be clear to any novice that there’s a lot to learn, the whole experience is not utterly baffling – it IS straightforward enough to at least WANT to learn. I feel the same way about SLR camera design, too – you can pick up a modern one and use it straight away, with no training, but the buttons and dials and settings to enable you to absolutely master the thing are right there in front of you, too.

    AutoCAD and other similar software, though – wow. I’ve been using it for fifteen years and I still don’t understand half of what it can do, and it’s designed – for the most part – in such a way that I wouldn’t even have a clue how to even find out what it can do at all. Powerful software, for sure, but it comes with a learning curve which should be labeled as a tripping hazard.

    Still, for both of these software examples and others, it’s truly fascinating to see an expert at work. If you can memorise all those keyboard shortcuts, the speed and efficiency of a ‘power user’ can be boggling. If you go so far as to add an extra – and expensive – gadget which most people wouldn’t have a clue about, there’s another quantum leap of effectiveness to be reached. One of the guys in our building, for instance, uses a 3D mouse. (something like this one: http://www.3dconnexion.co.uk/products/spacepilot-pro.html) I have absolutely no clue where to start with the device, but watching him work is like watching someone in charge of a spaceship, and from looking at the screen as he works, there’s no doubt that it’s a FAST spaceship.

    Sadly, this is probably where the wheel metaphor breaks down. Going from three wheels to two is hard, but the payoff is huge. Going from two wheels to one is REALLY hard, but I don’t think a unicycle is much quicker! Maybe I just haven’t mastered it yet…

    1. Frank Varro

      What I love about AutoCAD, and watching other people use it, is actually how flexible it is. Some people only use keyboard commands. Some only use a mouse with Icons and Menu bars for things not covered by icons. They just revamped their system, which muddy’s the water a bit, but it always fascinated me to watch people use it because you could tell how they learned it. I was half class taught, and self taught for the rest. I use keyboard commands for 40% of things, Icons for 40%, and menus for the other 20%. But the Keyboard commands are things I use every 20 seconds. Icons I things I use every hour, and Menus are for things I MIGHT use once a week.

  2. There are so many advanced interfaces that exist that require mastery and extend our ability and relationship with computers…

    Wacom, Leap Motion, midi, dmx, accelerometers, kinect, video game controllers, davinci resolve (I work as a colorist…), arduino controlled sensors, muse bci… That’s just off the top of my head.

    I wouldn’t lament that there are no options for advanced computer interfaces and expanding possibilities.

  3. This episode reads to me as a huge argument for windows products against apple products. anytime I’ve been in a situation where I’ve been forced to use damnable apple products my reaction is always “well it’s a pretty machine and everything”‘a single button but you can’t actually do anything useful with this expensive, neutered, gutless machine” and any time I’m on a windows product my reaction is “well it’s an ugly box and the programs give you a million options that take ages to go through and learn effectively, but once you learn it I can do so much more way the fuck cheaper”. The tricycle metaphor is great.

    1. David Warman

      ? I use both, have equivalent advanced tools on both, and run multi-button mice on both. And spend less time fighting OS configuration mysteries on Apple than on Windows. They just get there by a moderately different route, but they both get there. And even with the classic one button mouse there are always the ctl – opt – cmd keyboard chords to get to all the multi-button functions on a Mac. Not difficult at all. Were you talking about the original Macs or the current ones? And as to price, equivalent engineering quality for Windows machines costs the same as a Mac. You get what you pay for.

    2. Frank Varro

      @David- The thing I could never get used to on Apple products is how locked out of the background you are. I can always get under the hood of a windows machine if I’m having problems, or want to tweak something. Sure, less problems might come up, but that is due to the non-customizable nature of Apple products.

      Also, As someone who sold computers at Best Buy, Price is not equal for build quality between Apple and PCs. You can get a comparably spec’d PC for roughly half the price of a Mac, and while the exterior build quality might be better on the Mac, the battery and RAM isn’t soldered into place of a PC. That glowing apple costs a pretty penny on its own.

      It’s a bit like Bose, though not as extreme by any extent. Bose speakers are expensive because of the name. Where it differs from Apple is that Apple is at least a good machine, where Bose speakers are, generally, pretty terrible.

      TLDR: Apple costs 2x what a similar PC would cost, with less flexibility. Bonus info: Don’t buy Bose speakers- They are terrible.

    3. tux

      Lord have mercy, a debate between Windows and OSX. If you want to have some real control, and get the best performance out of the highest-spec’d hardware for less money, you buy a PC and erase Windows off your hard drive first thing. Or better yet, ask MS for a refund for the virus prone dump truck of an OS you’re forced to buy. Then you download an iso of a free linux OS (Mint, Ubuntu, Manjaro or another) and install it. You’ll never voluntarily go back to your slave masters again. And if you want to lose the mouse you install as tiling window manager on your desktop like i3, Awesome, or Openbox (Ob is stacking) that allows you to configure keyboard shortcuts for manipulating your entire graphical environment on multiple virtual desktops. If you don’t know what I’m talking about it’s because you’ve never looked outside the corporate walled garden of commercial lock-in.

  4. This is the first episode that moves me to commenting.
    The metaphor is totally wrong.
    The OS on the computer is not the tricycle or the bicycle, the OS is the Park in which they run. The base system need to be simpler enough for anybody to use, but the level of complexity comes from the task you want to do, the program you want to run. You can take a picture with instagram (tricycle) and be happy, or edit a high res image for days in Photoshop (21 gears, disk breaks, double suspension mountain bike).

    Appart from that, let’s be clear, I love this podcast.

  5. Daniel Barkalow

    This weekend, I was on a puzzle solving team which was working on a puzzle involving Disney songs. We’d identified that a song from Frozen was relevant, and gotten the message “Yes you do. Call when done.” We didn’t understand this until the team leaders’ 17-month-old queued up “Do You Want to Build a Snowman?” on an iPad. We got a lot of mileage out of everybody having a computer tuned to their particular abilities, at both ends of the scale.

    It was actually also notable that iPads are great for people who aren’t sitting at a desk, something that Doug Engelbart probably didn’t consider at the time.

  6. Noam Deplum

    Great dissection of advanced precursor technology that gets missed due to complicated user entry level. Although, I feel there was a key design element that was brushed over in this story. Design is communication, and the best form of communication is the most direct and necessary route. Just as Apple can be blamed for over-simplifying their designs for the of ultra-usability, had Engelbart distilled some of his concepts to a more approachable level–removing excess and creating a more direct user experience, we may have had these advancements sooner.

    I suppose what I am getting at is this: simplicity is only a danger when you sacrifice necessary tools for entry-level use. If all necessary tools are retained, design should always be simplified to the most accessible route. A complicated system should only exist if it cannot be further simplified and achieve the same function.

  7. David Maynard

    I had the honor and privilege of working in Doug’s Augmentation Research Center at SRI in the 70’s. Using Doug’s NLS system with a mouse and keyset was a better code editor than I can find today – provided it was late at night and I didn’t have to share the PDP-10 mainframe with many other users. This is pretty amazing considering that my laptop has several thousand times the memory and cpu speed of the PDP-10. I know that the only people that will believe me are those who have actually experienced using a mouse and keyset based editor. The efficiency of the editor was entirely due to the UI design and the use of the mouse and keyset together. For example to make a copy of an existing procedure within a large source file with NLS was a simple as pressing two buttons on the mouse and hitting a “b” on the keyset (^B) which changed the view of the file to one level of hierarchy showing first line of each existing procedure. Then I hit “c” on the keyset – the screen echoed >Copy. Then I hit a “b” on the keyset and the screen echoed >Copy Branch, then I point at the procedure I want to copy and click the left mouse button, the screen echoed >Copy Branch To and I pointed to a destination and clicked the mouse and the procedure is copied. This may seem like a lot of steps (3 keyset chords plus 2 mouse clicks) but these were all accomplished without ever moving my left hand from the keyset nor my right hand from the mouse. So all 5 steps could easily be completed in under a second.

    Doug Engelbart was absolutely correct in asserting a system that takes an investment to learn can be more powerful than an system designed for ease of use. Steve Jobs was absolutely correct in asserting that taking two days to learn how to use a keyset is an insurmountable commercial barrier. As others have pointed out, designs which allow progressive refinement of skills needed to complete ever more complex tasks efficiently are ideal.

    When Bill English and Doug Engelbart applied for their patent on the mouse they also applied for a patent on the keyset. They were granted the patent on the mouse and found out that the keyset was patented in 1865 by Western Union as a telegraph input device! The patent on mouse expired, after 17 years, three years before the first Mac shipped. Doug was indeed a visionary, and his vision was much, much more than just a mouse and keyset. It was “Augmenting the Human Intellect” using the best tools available. The quest continues.

    David Maynard

  8. Miles

    This is an interesting article. The next interactive tech should be better eye tracking and clicking with mind impulse. The hands should not be use at all. Many people like me have repetitive stress injuries due to poor design of the computer input devices and the software that does not streamline input requirements.

  9. pooptaco

    I have long felt that this was the problem with Apple. That it might be easier to start, but they intentionally made it so that you couldn’t do more complicated things, and yes it starts with the 1 button mouse, but it goes all the way to a lack of modularity in hardware. To me, this is what made apple a niche market for so many years, their overemphasis on simplicity and their over-enthusiasm for making everything proprietary and nothing modular. It bothers me a bit that PCs are following them down that route in recent years, trying to make computer operating systems more like smart phone operating systems. it bothers me that with each generation of computing i seem to get relegated closer and closer to the grunting class and further and further from the IT class as the chasm between the 2 widens.

  10. Where if felt this episode fell short is that beyond networking, it did not really specify any of the functions Engelbart wanted to enable for the deeper and more holistic computing experience he envisioned.

  11. Mario

    I love that you use so much of Philip Glass in your episodes. Here it was a fragment of Einstein on the Beach.

  12. Did anyone else notice the keyset diagram is in binary. I guess that would be an aid to memorizing too, as long as you knew the system.

    ie A=1, B=10 (2), C=11 (3), D=100 (4), etc

  13. Chris

    So… Does Christina Engelbart use a chord keyset? Theres a nice theory behind the argument, but I have to wonder–if it’s so amazing–does anybody actually use the thing?

  14. Mark Rainey

    A chord keyboard did go into commercial production in the form of the Microwriter which was an early personal organiser. It then moved on to the AgendA and now there is the CyKey which plugs into a PC. It was developed at the end of the 1970s by Cy Endfield and Chris Rainey (my father).

    It had limited commercial success partly due to the concern over learning the chords. In this case the chords could be remembered by the shape of the letters they represented rather than just their order in the alphabet – there is image of the alphabet here with the device …

    http://research.microsoft.com/en-us/um/people/bibuxton/buxtoncollection/detail.aspx?id=5

    Despite concerns, most people could learn most of the alphabet in under half an hour and with limited practice could become proficient enough to type at a usable speed.

    I also found out last year that my father met Doug Engelbart and discussed the chord keyboard approach with him after the Microwriter was developed.

  15. Wonderful piece on my dad! I was honored to be included. And great comments everyone! If anyone’s interested I posted some archive photos of people working in the lab http://on.fb.me/1yOoDNO. Note the default “stance” with left hand on the keyset and right hand on the mouse. Only for lengthy typing did one move hands to the keyboard. I definitely concur with Dave Maynard, as would most of the alumni from the group. I would love to have a keyset to use with my laptop (pc or mac) – it would definitely increase efficiency, but barely scratch the surface of the tremendous potential that Doug and his team envisioned and pioneered.

  16. Paul Matsumoto

    Anyone who has ever studied or who works with machine shorthand (stenography) would probably recognize the similarities between Englebart’s keyset and the steno machine — the former appearing to be an extremely simple stepchild of the latter. Likewise, they would recognize that while learning machine shorthand takes time to master, the result can be the ability to report spoken words in realtime at speeds of around 200-plus words per minute. Finally, I would note that Englebart’s binary-like code for writing the alphabet resembles a shorthand system developed withing the past fifteen or so years by steno world-record-holder Mark Kislingbury, whose method for writing any year between 1900 and 2999 in a single keystroke takes advantage of the same basic principle.

  17. Peter Laundy

    Yes, I agree. I found myself wanting to shout at the podcast ” what about keyboard shortcuts!” when the assertion that the only interface offered was dead simple and required no learning.

  18. David Schleider

    Loved the episode, but i was excepted you to go in to how Microsoft “tricked” users in to learning how to interface with windows by using the games, solitary and minesweeper.
    I always thought that was a triumph of design

Leave a Reply

Your email address will not be published. Required fields are marked *

All Categories

Minimize Maximize

Playlist