Lecture Presentations

Whether you’re a Mac or a PC, you have, inevitably, been hailed by a bouncing icon desperate for your attention or chastised by a cartoonish dialogue bubble reminding you to update your virus software. Welcome to the world of human-computer interaction—or, well, interruption.

On October 27, Barbara J. Grosz, dean of the Radcliffe Institute for Advanced Study and Higgins Professor of Natural Sciences in the School of Engineering and Applied Sciences (SEAS), dedicated her decanal lecture —“Can’t You See I’m Busy? Computers That Know When To Interrupt”—to the seemingly straightforward but exponentially complex task of building systems that know if and when to tug on our coattails.

We interact “with our devices and machines almost constantly, and they with us,” Grosz began. Today’s highly responsive computers are pocket-size and literally hands-on—thanks to innovations like the iPhone—and proactive, as in talking GPS navigators. Not all electronic interruptions, however, are equally astute or even desirable.

To illustrate, Grosz offered a rogue’s gallery of poor interrupters: She played a radio segment about Microsoft’s animated editorial assistant, the much-loathed Clippy, and highlighted counterintuitive computer-error messages such as Warning: Mouse not found . . . Please click the left mouse button to continue. After an appreciative laugh from the audience, Grosz summarized her aim as creating collaborative computer systems “that adapt to us rather than forcing us to click okay when we are absolutely sure it is positively not okay to click okay!”

As an example of an “okay” human-computer partnership, Grosz presented the Writer’s Aid program (created with Stuart M. Shieber, the James O. Welch Jr. and Virginia B. Welch Professor of Computer Science at SEAS, and Tamara Babaian, then a postdoctoral fellow and now at Bentley University). The program searches for and appropriately integrates citations into a document without bothering the author, because, Grosz said, “computers are good at searching; we’re good at writing.”

Building on this exemplar, she used the remainder of her talk to tackle thornier  interactions that involve “quick decisions,” such as a driver’s receiving information en route, or occur during “periods of crisis,” such as a fire or a medical emergency. In both cases, “interrupting at the wrong time is costly, but not interrupting at the right time is also costly.”

Designing a framework to understand interruption requires figuring out the probability that a user wants or needs the information and determining the cost of the interruption, the value of the new information, and the user’s willingness to accept the interruption. In fact, the same criteria apply when a computer system wants to ask—that is, interrupt—a user for information. For example, a smart phone might query a user in a remote location with “Is it okay to switch to roaming?” or a GPS system might interrupt a driver to “talk” about the weather.

To discover when you need to know, what you need to know, and how much, in some sense, you are willing to pay for the information, David Sarne, a former postdoctoral fellow, and Ece Kamar, a graduate student currently working with Grosz, repurposed a program called Colored Trails (CT). Although Grosz originally codeveloped CT in 2004 for conducting decision-making studies, Sarne and Kamar realized that with some imaginative engineering, CT could also serve as an ideal environment in which to study interruption management. (CT can be downloaded free at www.eecs.harvard.edu/ai/ct/.)

Specifically, Kamar is using the modified program to study a situation in which a human player can decide whether or not to be interrupted by the computer. In a typical CT interruption game, the computer and the player each have a goal. Those goals may change, and although the player can see them, the computer cannot. The computer may ask for information from the player, but at a cost to its overall score.

The CT environment is less abstract than it may at first seem. To begin with, players in a CT interruption game are actually performing a task by moving through squares to reach a goal. Further, players do not necessarily have complete information in the CT environment, as they do in a standard behavioral economics game.

With this in mind, Kamar’s aim is to use the game to reveal what factors might influence a player to agree to provide information the computer needs to reach a shared goal. “Supporting positive interaction between a computer and a person is incredibly important,” explained Kamar. “If one aspect of the interaction goes poorly, the person is likely to ignore or ‘kill’ the program.” She is looking to find the kind of interactions that will encourage people to collaborate.

Not surprisingly, the perceived utility of an interruption is a major factor affecting a person’s acceptance rate in these studies, Grosz said. People know when butting in is valuable, as when a GPS navigator exclaims, “Take the next exit to avoid traffic!” But utility is a consideration for both sides. The analogous computational challenge is to ensure that a system, busy figuring out how best to meet the needs of a driver, does not neglect its own priorities, such as downloading the latest satellite map data, at a critical moment.

Saving processing cycles brought to mind another game: Battleships. Once a player has made a hit, she or he knows that other hits can be made nearby. In the same way, an efficient algorithm that is designed to compute the value of information rather than to ask about every possible hit focuses on the most probable “hits” and thus asks fewer and better questions. Furthermore, Grosz explained, “You don’t need to compute the exact value of a piece of information. All you need to compute is whether the value of the information is greater than the cost”—thereby making an interruption worthwhile.

Of course, the value of interruption is best realized in action. In November, Kamar will intern at Microsoft to work on a prototype technology called MC Market, which is designed to help shoppers on the move find the best deals in nearby stores. Success will depend on how much money they can save and how much irritation they can avoid. If the program does not impress a savvy shopper, Grosz said, he or she is likely to “throw it out the window.”

The practical aspects of Kamar’s research led Grosz to recall her own days as a student, when she first became interested in using computers to improve the teaching of mathematics. Appropriately, she is now working with a local educational research and development company to give middle school teachers that proverbial eye in the back of their heads with the help of learning software called TinkerPlots (www.keypress.com/x5715.xml). The goal of this effort is to create an agent, using a predictive algorithm, that monitors students as they model dice rolling and tackle other statistical problems. By “watching” their progress, the program can alert a teacher about who may need particular help. “How do they know who to pay attention to?” Grosz asked. “The idea of the collaboration system is to watch what the student is doing and be able to tell which students are headed in a good or bad direction.”

Returning to games, Grosz ended her talk by highlighting women’s basketball—specifically, the Harvard victory over number-one-ranked Stanford in 1998. Although Harvard was ranked sixteenth, the players on its team won the game by passing—working together—rather than going for individual glory. Grosz used the victory to illustrate that interruption done right enables cooperation, and to thank her own academic and administrative team for their help.

Her final slide announced, “No one (man or machine) is an island.” Kamar says that Grosz takes that notion to heart. “She’s a wonderful manager. I think the most important thing about her is that she can always see the big picture. This is what makes Barbara Barbara—and is why she is so successful.”

For more information, read "A Rogue’s Gallery of Poor Interrupters."

Michael Rutter is the director of communications in Harvard’s School of Engineering and Applied Sciences.

Photos by Tony Rinaldo

Return to Lecture Presentations