James Patten: Frontiers of the Human Digital Interface - OZY | A Modern Media Company

James Patten: Frontiers of the Human Digital Interface

James Patten: Frontiers of the Human Digital Interface

By Laura Secorun Palet


Using the full potential of computers will require better tools for human interaction. James Patten is designing them.

By Laura Secorun Palet

The mouse. It utterly transformed human interaction with the personal computer by letting people click on an object rather than type in obscure commands.

Then, remember Steve Jobs strutting about the stage as he introduced Apple’s iPhone in 2007? It wasn’t just a phone, but a revolutionary touch screen for humans to control ever more powerful mobile computers, to get useful stuff done easily.

What’s next? Just possibly it’s Thumbles, the brainchild of New York-based interface designer James Patten.

“Humans and machines can do extraordinary things together,” says Patten. “The challenge is to find the best way to combine the two.”

Thumbles allow multiple users to simultaneously make decisions under pressure, like an airline company having to reorganize its entire flight schedule…

Thumbles are motorized, mouse-sized robots on wheels that humans can manipulate to tell a computer what to do. But in this case it’s a two-way street. The computer also moves the Thumble, providing not just visual feedback but a physical object that moves about.

Why Thumbles? Keyboards, mice and touch screens are great for manipulating text, selecting objects, entering data, playing games. But it’s one-way communication. What if you had a hundred objects to move about while coordinating an emergency response and wanted to watch as they moved on their own? What if a storm forced the rerouting of hundreds of aircraft on short notice, with multiple controllers in different locations?

That’s Patten’s challenge, to provide a solution you can literally put your hands on.

Patten works at the experimental frontier between the physical and digital realms, developing new ways of making technology more intuitively tactile, something you can put your hands on. His design firm, Patten Studio, develops futuristic interactive interfaces ranging from musical instruments for Bjork to a chemical reaction exhibition at the Museum of Science and Industry in Chicago, where visitors can grab representational atoms from the periodic table to produce reactions.

The 37-year-old developer is a hybrid tech-nerd and philosopher. Shy at first, he warms up to speak excitedly about his little robots.

Growing up in the remote town of Mathews, Virginia, close to where the Rappahannock River empties into the Chesapeake, Patten had few distractions, so he dismantled things for fun. “I would beg my mom to take me on trips to RadioShack to buy electronic devices. I would always ‘fix’ — that’s to say, dismantle — any electronics that stopped working in the house, like the VHS,” he recalls.

His love of “fixing” things led him to study engineering at the University of Virginia, but he quickly found the curriculum too confining and transferred to an interdisciplinary program where he learned computer science, psychology and philosophy.

Man with microphone on headset as he speaks

James Patten speaking at TED

Source TED

Later, at the MIT Media Lab, professor Hiroshi Ishii taught him how “technology could be beautiful and playful, even in 1998, when computers where ugly beige boxes,” Patten recalls. There, with colleagues in 2002, he developed his first successful physical-digital interface, the Audiopad — an electronic musical instrument that tracks objects on a tabletop and converts motion into sound.

This system planted the idea of the “Thumble,” but at the time, the components were too expensive to be commercially viable. Patten sees countless possible applications for the new interface, from film-editing software to scientific research and online gaming. But it’s more than that.

“It’s about opening people’s minds,” says Patten, who thinks the satisfaction of physical manipulation and instantaneous feedback taps into human nature. “When we are born, the first way we learn about the world around us is by touching it.”

Imagine walking into a room and interacting with a 3-D representation of your data; you could literally walk inside it.

— James Patten

And manipulating real objects can help problem solving and promote collaborative learning.

“If you give someone a pile of coins, they will start moving them around making piles,” says Patten. “They are changing the problem to make it easier by moving physical objects. But they won’t do the same on a screen because it’s more difficult.”

The prototype devices only work on a screen, though Patten hopes his little robots will soon operate on any surface. To start with, Patten Studio is looking at professional applications like scientific visualization, database management and complex logistics.

“Thumbles allow multiple users to simultaneously make decisions under pressure, like an airline company having to reorganize its entire flight schedule at the drop of a hat,” he says. In fact, the studio is already in preliminary discussions with several companies that produce emergency response software.

Interaction design expert Eric Gould Bear believes Thumbles have potential but are far from proven. “Consumers are asking for more intuitive devices,” he says. But he cautions: “To succeed it will have to demonstrate that it can add more value than a multiuser touch screen.”

Patten admits it will take time before people rethink how to use a computer, but he believes his product fits well into the growing interest in the Internet of Things and the declining use of the written word.

Will the world care for these futuristic little robots? We’ll find out by mid-2015, when Patten hopes to have the third and final Thumble prototype ready for the market. Meanwhile, he is working on other commercial projects and dreaming of what’s next.

“Drone technology might one day allow Thumbles to become a fully 3-D experience,” he says. “Imagine walking into a room and interacting with a 3-D representation of your data; you could literally walk inside it.”

Hard to envision. But that’s the point. You have to touch it. 

Sign up for the weekly newsletter!

Related Stories