---

The Human Mouse

IBM Research demonstrates a computer that sees and hears.

(11/26/97) -- "It's a joke that the keyboard works only for people who know the Roman alphabet. In 20 years, people will laugh at us for calling this technology. No one should ever have to see a computer. The complexity should be soaked into the world around you," says Mark Lucente, a researcher at IBM’s T.J. Watson Research Center in Yorktown Heights, New York.

So instead of being handcuffed to the keyboard, Lucente walks about his lab, stands, and sits, all while talking to his computer and gesturing in a natural way. "Show me the Gulf of Mexico," he says. And a graphic of the gulf is displayed. "Make it this big," he says, spreading his arms. "Grab that," he says, pointing to an object on the display. "Leave it there. Change the color." And the computer follows his orders.

Lucente’s Visualization Space is an experiment in so-called multimodal computing. The system uses off-the-shelf components including a four-processor IBM PC with half a gigabyte of RAM, a Diamond Fire GL advanced 3-D graphics board, and a video camera to track hands and feet. The lab has also run the system off a notebook computer.

The system also uses IBM’s continuous-speech recognition software ViaVoice. The hard part, Lucente explains, is coming up with the software algorithms to combine gesture and voice.

Besides enabling cool games, Visualization Space has already piqued the interest of pharmaceutical, medical-imaging, and petroleum companies. Lucente predicts that the world he envisions--devoid of mice, keyboards, and other computer paraphernalia--is still 15 years away.--Carol Levin