In this classic presentation from CGDC 1998, Adam Frank and Andrew Stern explain what working on their seminal virtual pets franchise has taught them about designing believable characters. Presenters were required to submit a formal paper at the time, and we are reprinting the original paper as it was written.

Multiple Character Interaction 
Between Believable Characters

PF.Magic
501 2nd St, Suite 400
San Francisco, CA 94107
+1 415 495 0400
http://www.pfmagic.com
http://www.netcom.com/~apstern

Introduction

Computer games have always had characters. In a typical game the player controls the “main character” to fight or compete against one or more computer-controlled “non-player characters”. The game’s “artificial intelligence” (AI) controls the behavior of the non-player characters in an effort to out-maneuver, out-shoot or out-smart the player.

But even to this day, with few exceptions, computer characters are really no more than shallow cardboard-cutouts. While they are beginning to look more realistic, they still act in very limited and often predictable ways, with almost no variety in their behavior. They tend to do and say the same things over and over no matter how hard the player tries to get them to react differently. 

Fortunately, as the power of the personal computer grows, it is becoming more and more possible to create truly lifelike and believable autonomous characters. In this paper we first propose what lifelike, believable characters should be, and design principles involved in making them. The first multiple character interaction we discuss is between the user and a single lifelike computer character. Next we discuss what should happen when multiple lifelike computer characters interact with each other, and what new kinds of interactive experiences for the user can result from this.

We use the Virtual Petz we developed at PF. Magic to demonstrate these key points. The original Dogz, initially released in 1995, was the world’s first virtual pet, followed soon by Catz, Oddballz and the new Dogz II and Catz II. Virtual Petz are autonomous real-time 3D characters that live on your PC computer desktop, grow up over time, and strive to be the user's friends and companions. 

What are Lifelike, Believable Characters?

These days most people think the way to make a computer character more “real” is to make it look more photorealistic, and to give it a better AI. Since almost all interactive entertainment software products are games, the typical AI is designed to figure out the best move to make, such as the next punch or kick move, where to run to and shoot from, and so on. But figuring out the best next move is an incomplete definition of AI. When people talk about wanting great AI, or great “artificial life” for that matter, what they really want is the experience of interacting with something truly alive. 

When would a player believe a computer character is truly alive? What constitutes a convincing, interactive “illusion of life”? It seems that the best way to define what truly alive means is to study real living “characters” – that is, animals and people. 

- Real living things are very reactive and interactive. Given this, when users physically interact with a lifelike computer character in its environment, the character should be aware of what is happening and react immediately. If a character reacts in a way that makes no sense, or doesn’t react at all, then the illusion of life is broken, and the character cannot be considered believable. Also, real living things don’t always react the exact same way each time! A believable character should not appear to be repeating itself verbatim every time the user interacts with it. Along the same lines, if the user is given little or no way to interact with the character in the first place, the whole interface itself does not support believability. 

- Real living things are able to communicate. Most animals, and people especially, have several ways to communicate with each other. These include gestures, actions, body language, sounds, and spoken or written language. To be truly believable, users and computer characters should be able communicate with each other in a natural, lifelike way. If a user comes face-to-face with on-screen human character, they really should be able to speak to communicate, not type. 

- Real living things are very expressive and able to show a broad range of emotions, desires and personalities. A believable character wouldn’t just fight or shoot, it would have a broad base of behavior – the ability to eat, sleep, play, chase, explore, groom, build, and so on. Lifelike characters should be emotional as well. Depending on what is happening in their lives they would act happy, sad, angry, afraid, jealous, and so on. They may get energetic or tired, hungry or full, sick or healthy, and may tend to be grumpy, hyper, sweet, or finicky. All of these emotions, moods and personalities must be expressed by how the computer character behaves, animates and sounds. Even if these things are being kept track of in the character’s mind, if they’re not expressed, they’re effectively not there.

- Real living things are able to learn, adapt and change over time. Believable characters should be able to pay attention to what is going on in their environment and learn from it. They should have attitudes and feelings towards objects and other characters, and act appropriately upon these attitudes. They should form relationships with the other characters in the environment, including the user. As a believable character grows and ages over time, its attitudes and relationships will change, and it will gain experience and knowledge.

If you follow what’s happening these days in academic media research laboratories, the concept of believability is now a key design goal for many synthetic character research projects, such as the Oz Project at CMU (1), Improv at NYU (2), Synthetic Characters at MIT Media Lab (3), the Virtual Theater Project at Stanford (4), and so on. Refer to my personal web page for a complete listing (5).

Examples of Believable Characters: Virtual Petz

Clearly, making truly lifelike believable characters is a difficult problem. At PF.Magic we created the Virtual Petz characters using the above criteria, incorporating as much of each component of believability as we possibly could. As far as we can tell, the Petz are the first consumer software products to address all of these criteria. 

The most direct way the user can interact with the Petz is through petting. Using a mouse the user moves a hand-shaped cursor to pet, scratch and stroke the character. The pet immediately reacts in a variety of ways depending on where its body is being petted, how fast, and how the pet feels at the time. By right-clicking the user can pick up and carry a pet. We call this direct interaction. The fact that you can touch the character is a very effective way of building an intimate relationship and creating an illusion of life.

The toys and objects in the pet's environment have direct object-like interaction for both the user and the Petz. By picking up and using a toy, the user can initiate play. For example, throwing a ball may initiate a game of fetch, or holding a tugtoy in front of a pet may initiate a game of tug-of-war. Similarly a pet can get its own toy and bring it to the user to initiate play. This cooperative decision-making helps build the owner-pet relationship. 

We have created a variety of personalities - playful terriers, grumpy bulldogs, hyper Chihuahuas, finicky Siamese cats, lazy Persian cats, aggressive hunter cats, timid scaredy cats and so on. Additionally each pet has its own individual likes and dislikes, spots and body coloration, and personality quirks. Users get to play with a pet and see if they like them before deciding to adopt. Once adopted, the user gives them a name. This individual variation allows the user to develop a unique relationship with a particular pet. Every owner-pet relationship is different. 

Petz communicate with the user in a variety of ways. Emotion can be expressed through different facial expressions (eyebrows, mouth, ears), styles of movement and body language (sad walks, happy trots, various postures, a variety of tail motions), and sounds (excited playful barks and meows, sad whines and whimpers, yelps of pain, etc.). They get fat when they eat too much and get skinny when hungry. 

Petz communicate their intentions and desires through their actions. We have created a broad base of behavior for the Petz -- they can eat, sleep, play, attack, groom, hiss, explore and so on. When Petz want attention they may start barking or meowing. If hungry they may go searching for food, hunt mice, or start begging. When upset or scared they may run and hide, act aggressively, or cower and shiver.

The user is given a few specific hand gestures that Petz may respond to. By double-clicking in an empty space on the screen the hand cursor animates a "come here" motion and plays a whistling sound. Petz tend to stop what they're doing and come to the user's cursor. Users can also pick up and wiggle food treats in different directions to train their Petz to jump, sit, rollover, somersault, and so on. Additionally, users can reward or discipline any behavior. Methods of rewarding include a giving treats, feeding, and petting, while discipline is achieved by squirting a pet with a water spray bottle. In this way users can modify a pet's overall behavior through positive and negative reinforcement. Additionally, the personalities of each pet can change under extreme circumstances. For instance if any pet is underfed it will begin hunting and acting aggressively. If a pet is overfed, it will become fat and lazy. So depending on how their user treats them, their behavior can slide along a broad spectrum of personalities.

A User-Perception Based Approach

It is important to note that the only way the user can understand what the Petz are feeling is to interpret their actions and physical cues, in the same way an audience interprets an actor's performance. We do not display bar graphs or text messages describing the pet's internal variables, biorhythms or emotional state. By forcing a natural interpretation of the pet's behavior, we don't break the illusion of a relationship with something alive.

Again, “real” AI or A-Life should be defined as the illusion of life. In entertainment software, all that matters is the strength of this illusion. It does not matter what internal techniques are used or how sophisticated the programming is. Lifelike believable characters are ultimately perceived and judged through human eyes. All that matters is what you see and believe on-screen. This should be the standard by which any believable characters are held to – does the audience believe that the characters are alive. Simply making a computer character look photorealistic won’t work – without the interactivity, communication, expressiveness and adaptability, it’s just a mannequin.

Design Principles for Making Lifelike Believable Characters

Certainly there are many ways to go about making lifelike computer characters. In the process of working on Virtual Petz we have come up with a few design principles that have worked well for us. 

- Choose characters you can successfully implement. For Virtual Petz, choosing dogs and cats as our characters helps us in a few ways. People understand what pets are and essentially know how to interact with them. This is critical for producing believable characters. Pets are also a good choice because they are expected to behave in ways that we can successfully implement. They have relatively simple gestures, language and cognitive abilities. Also, animated dogs and cats have well established cartoon archetypes. We base the look and behavior of the Petz on these archetypes.

- Frame the experience in a looser, more character-centric way. Instead of the typical goal-oriented computer game, the Virtual Petz experience is non-goal-oriented. We allow users to explore the characters and their toys in any order they like. This freedom allows users to socialize with their Petz in their own way and at their own pace. This also encourages users to come up with their own interpretation of their pet's feelings and thoughts. Often the scope of their interpretations far exceeds what we originally planned for. 

Note that we have focused almost all of our energy on creating rich characters and not on creating a sophisticated environment. The Petz come to the user’s environment – the desktop – not the other way around. We only built the objects and toys that were needed to support the interaction with the characters. By doing this we are making the Petz the stars of the show. 

- Design your animation to be as dynamic and responsive as possible. A high degree of interactivity and lifelikeness truly requires the use of real-time 3D animation. Pre-rendered animation, however beautiful it may be, cannot be as responsive, dynamic, or varied as required for lifelikeness and believability. One should choose to err on the side of frame-rate versus a photorealistic image that interacts poorly.

The Petz products use a network of basic 3D body animation which has a high resolution of branch points. For example, from any basic pose such as standing or sitting, the pet can transition to any other pose. As the pet's general body animation moves across this network, additional animation is layered on top. This allows the pet to look in all directions no matter what its body doing. Also we layer breathing, blinking, meowing, head shaking and even character-specific posture. In fact the look of the each breed and their ages are layers. The subsequent number of permutations is enormous and therefore passes a critical threshold in human perception. This method greatly enhances our ability to create an illusion of life.

- Don’t over-engineer the behavior logic. The Petz behavior is controlled by a variety of mechanisms that operate simultaneously, each with varying complexity. There are small mechanisms to control blinking and breathing, biorhythm mechanisms to keep track of things like energy and hunger, and a variety of cognitive mechanisms to keep track of multiple goals, different plans to execute each goal, multiple emotions towards other characters and objects, attitudes towards other characters and objects, and personality-specific quirks. In general we’ve been able to reduce complexity and still retain a high-degree of lifelikeness and freshness with some relatively simple techniques. To achieve a real, unpredictable, non-repetitive quality in these mechanisms, we often use constrained randomness. For example, there are rules for each individual pet that determine which toys it most likes to play with, but there is some randomness in that decision. This results in the possibility that a pet might play with any toy, not just the toys it likes the most. A lot of development time is spent tweaking the constraints on the randomness to achieve a good balance between consistent personality and unpredictability. Again we are focusing on the user's perception of the character and being careful to not create a behavior system that models more than can be expressed.

Interaction Between Multiple Lifelike Computer Characters

Now that we’ve talked about what believable characters are, how should two lifelike computer characters interact with each other? Again let’s look how real living things behave. Animals and people perceive and react to one another in all sorts of ways -- they look each other over; approach and investigate one another; communicate using gesture, body language, sounds or speech; directly contact one another by touching, holding, fighting, hugging, mating, and so on. A believable character should be able to do all of these things.

On a conceptual level, the two are forming a relationship with each other. Believable characters have attitudes and feelings about one another – how much they like each other, how much they know about each other, what they can gain from each other. This relationship evolves over time as they interact and events happen. If two characters help and support each other, they should become friends; likewise if they hurt or abuse another, they should become enemies. The characters’ individual personalities should influence the ways and rate at which the relationship can change – a very timid character may quickly distrust another at the slightest provocation, whereas a happy-go-lucky character would need to be severely abused to begin distrusting others.

Again we’ll use Petz as an example. In our newest products,