Realistic robot faces aren't enough – we need emotion to put us at ease with androids

When the first guests arrive at Nagasaki’s Hotel Henn Na in July this year, they will be greeted and served by robots. In a similar approach, Toshiba’s android Aiko recently held a short-term role greeting customers at a department store in Tokyo. Customers were comfortable approaching Aiko to ask for directions and even the receptionist who normally holds the post felt her android colleague was doing a reasonable job.

There is no doubt that developments in android technology mean that robots now look more lifelike than ever. It is even possible to imagine mistaking a robot such as Aiko for a human, at least at first glance. However, encountering near-human agents may not always be a comfortable experience.

Hotel Henn Na claims it is looking to explore the elements that “personify” hotels and to recreate that by using person-like entities. The big question is how people interact with robots, such as Aiko, that have a human appearance. Such interactions require considerably more than a robot with an apparently human face.

Our research at the Open University has considered aspects of how it might feel to be in a situation similar to interacting with a robot receptionist. It asks whether the properties of a robot receptionist’s appearance and behaviour are able to make us feel comfortable or may make us decidedly uncomfortable. Our work into the so-called uncanny valley effect also suggests that a key aspect of such interactions will be the ability of the robot to reproduce and convey realistic emotions, particularly through their facial expressions.

The uncanny valley effect is now a well-known relationship how “human-like” a robot is and how comfortable people feel when interacting with it. There is a point where something can seem too close to human to be comfortable – and our research has explored some explanations for why this might occur. For a long time, research in this area was mainly focused on the practical aspects of how near-human agents appear and on the techniques that might make them look more realistic and acceptable enough to bridge this uncanny valley.

Our early research had observed that some of the eeriest and most unsettling near-human agents were those with exaggerated features, such as a “cute” doll with dark, wide and blank eyes. Building on the idea of disturbing facial features, our more recent research looked at the role of emotions in the uncanny valley effect and considered whether that eeriness may actually be a result of those near-human faces being unable to present realistic emotional expressions.

Most of the research into this area has considered how artificial faces can be made to appear more human-like, but we looked at how even human faces can appear eerie under certain conditions. We have found that when parts of photographs of faces posing emotional expressions were combined, certain combinations of emotions evoked a sense of eeriness. This effect was particularly strong when happy mouths were paired with scared or angry eyes.

Easier to love?Daniel Karmann/EPA

In other words, encountering a face where the mouth is clearly smiling, but the eyes are not displaying the same emotion, can be quite disturbing. It follows that unless a robot face can display emotions accurately and appropriately, just as a human would, a human viewer could be left with a distinct feeling of unease.

So, it will be a long time before human receptionists superseded. Even now that the design of robot faces is at a point where they can be made to appear human-like, robots will need better abilities to be able to perform convincing emotional interactions.

Creating near-human agents that can portray genuinely believable emotional expressions presents a highly complex technological challenge. So while this tension between appearance and emotional interaction may currently confine robots to the uncanny valley, it could also offer a way out.

An alternative would be for robot designers to avoid trying to mimic human faces too closely until technology has progressed to allow their feelings to be represented realistically as well. Otherwise, our expectations about emotions would just be raised by the their human-like nature and then not met by their expressive ability. Choosing to make robots less human in appearance could avoid a constant race where robots that appear more human-like also require ever more convincing emotional abilities, offering a way around the uncanny valley.

Graham Pike has received funding from the EPSRC, Home Office, College of Policing, HEFCE, British Psychological Society and from the charitable arm of Reed.

Stephanie Lay does not work for, consult to, own shares in or receive funding from any company or organisation that would benefit from this article, and has no relevant affiliations.