Virtual Characters and Real Emotions

Jesse Schell is one of the most articulate, insightful people developing games and talking about their future.    At a recent keynote at a Unity3D conference, he talked about virtual characters  as a crucial part of the future of games and other online experiences.  As usual he makes a lot of excellent points about virtual characters remembering you and conversing with you, but on one — how we interact emotionally with virtual characters — I have to disagree with him:

“Emotions are easily recognized by humans, but computers must be part of that, said Schell. “Once we can do that we can sense your emotions,” said Schell, developers can create “a game where you actually have to act, or feel emotions. A game where someone tells you where there dog just died and if you can’t manage to cry then no, you’re not getting to the next level!”” (as covered by Gamasutra)

First, I appreciate Jesse stepping up with concrete predictions and other musings — as he says, this is a great way to predict (and create) the future.  That said, this one is exactly backwards: the emotional connection with virtual characters doesn’t come because we emote effectively, but because the characters themselves have and display emotions that we then relate to.  Their emotions make them more real to us, and allow us to feel something similar.

Think of it this way: we don’t miss on the climax of a movie if we aren’t actually crying.  We want the actors to emote as their characters — they have to lead us, so we can follow.  This is as much about mirror neurons as feeling like it’s okay and appropriate to feel an emotional release in a particular situation.  When the characters and situations in fiction and art (including games) provide an authentic emotional context, we also experience it. Doing so deepens our engagement and our relationship with those characters.  Requiring that we first feel and display an emotion puts the onus in the wrong place.

So what we need here is characters that can feel and show emotions appropriate to a situation and relationship, so that we can feel free to feel them too.  This requires that virtual characters not just understand our emotions, but that they provide authentic, subtle, layered, conflicting emotional responses just as humans do.  Only then will virtual characters be real enough to us to warrant our interest and our relationship, which makes a lot of other things possible.

For example, Jesse mentions possibilities like talking to your avatar in a game — in effect, taking on more the role of the director working with an autonomous actor than being the character yourself.  I can see this happening (in fact I have a whole design predicated on the idea of influencing rather than driving characters in a game.)

Other examples include a virtual character being an assistant, concierge, tutor, or continuous virtual presence across games, purchases, email, and other experiences. All make a great deal of sense.  However, as these experiences expand from the merely utilitarian (a shopkeeper with whom you interact in only a narrow domain) to the more relational (coach, concierge, assistant), an emotional connection becomes more vital.

While virtual characters need to be able to read human emotional responses (via voice stress, facial expression, etc.), it is at least as important that they be able to maintain and display authentic emotions themselves.  Otherwise their responses to us will always seem oddly flat in a way that subverts any connection between a person and a virtual character.

On the other hand, I believe that as virtual characters begin to display authentic emotions — not limited to a small palette of canned emotional states — we will see greater connections between us and virtual characters.  This relationship connection will, I believe, be the doorway to seeing virtual characters becoming much more prevalent in our lives in many different contexts.

Explore posts in the same categories: AI, games, psychology

6 Comments on “Virtual Characters and Real Emotions”

  1. Mani Says:

    And with that I’m unplugging my webcam, My Playstation eye and my Google Voice commands. They are collecting data at an alarming rate and after all human emotion is just the brain extrapolating what you are trying to communicate based on the database of your own contexts.
    I’m thinking with millions of people making faces at webcams daily paired with the emotional cues and words Google will have my number -or “Intent” down in no time flat.


  2. Jesse Schell Says:

    Your commentary is very thoughtful, and rightly points to things I neglected to discuss in my talk. Creating more powerful models of emotion in virtual characters is a big part of what will help us connect to them. There is always much talk of “sentient characters”, and we should always be reminded of what the Latin word “sentient” actually means — it is a sentence, in itself: “They will feel.”


  3. Allan Says:

    Does the uncanny valley play into this? Do we loose intimacy and empathy for virtual characters if they get the emotions almost, but not quite, right? I haven’t encountered that, but then I haven’t yet felt emotionally connected to an AI either.


  4. Mike Sellers Says:

    I think we might, Allan, in the same way that someone whom you believe to be falsely or inappropriately showing some emotion becomes less attractive or even repellent.

    I have to say though, the point from Jesse’s talk that sticks in my head is the very simple recognition of the player by in-game characters: “It’s a-you, Allan!” instead of “it’s a-me, Mario!” as he said.

    Animal Crossing did this to some degree; seems like for persistent world games it should be an easy trick, but… not so far.


  5. Allan Says:

    Recognition. I can see that.

    We were playing a cycling game on the Wii this weekend and the developers had painted my son’s name on the virtual street like virtual cycling fans might. It was a good detail to get right.

    IIRC in the pre-graphics days of computers a therapist put together a program that asked the questions a therapist might and used cues in the responses to ask more qustions and draw the patient into talking about themselves. There were people who spilled their guts to the thing even though it was clearly a shell. The desire to be heard and understood is a powerful human need.


  6. maxG Says:

    this is really great! Thanks and Cheers 🙂


Comments are closed.

%d bloggers like this: