Virtual Humans

Only fifty years ago, computers were barely able to compute useful mathematical functions. Twenty-five years ago, enthusiastic computer researchers were predicting that all sorts of human tasks from game-playing to automatic robots that travel and communicate with us would be in our future. Today's truth lies somewhere in-between. We have balanced our expectations of complete machine autonomy with a more rational view that machines should assist people to accomplish meaningful, difficult, and often enormously complex tasks. When those tasks involve human interaction with the physical world, computational representations of the human body can be used to escape the constraints of presence, safety, and even physicality.

Virtual humans are computer models of people that can be used

• as substitutes for ``the real thing'' in ergonomic evaluations of computer-based designs for vehicles, work areas, machine tools, assembly lines, etc., prior to the actual construction of those spaces;
• for embedding real-time representations of ourselves or other live participants into virtual environments.

In the efforts that we describe here, we cross several domains which in turn build from various interrelated facets of human beings (Fig. 1):

• Human Factors Analysis: Human size, capabilities, behavior, and performance affects work in and use of designed environments.

• Real-Time Agents and Avatars: People come from different cultures and have different personalities; this richness and diversity must be reflected in virtual humans since it influences appearance as well as reaction and choice.

• Instruction Understanding and Generation: Humans communicate with one another within a rich context of shared language, senses, and experience and this needs to be extended to computer-generated agents and avatars.

• Bio-Medical Simulation: The human machine is a complex of physical structures and functions; to understand human behavior, physiological responses, and injuries we need to represent biological systems.

• Motion and Shape Analysis: Understanding what we perceive when we see or sense the world leads to models of the physical world (physics) and the geometric shapes and deformations of objects.

From these virtual humans research areas, many current, emergent, or future major applications are enabled:

• Engineering: Analysis and simulation for virtual prototyping and simulation-based design.
• Virtual-Conferencing: Efficient tele-conferencing using virtual representations of participants to reduce transmission bandwidth requirements.
• Interaction: Agents and avatars that insert real-time humans into virtual worlds with virtual reality.
• Monitoring: Acquiring, interpreting, and understanding shape and motion data on human movement, performance, activities, or intent.
• Virtual Environments: Living and working in a virtual place for visualization, analysis, training, or just the experience.
• Games: Real-time characters with actions and personality for fun and profit.
• Training: Skill development, team coordination, and decision-making.
• Education: Distance mentoring, interactive assistance, and personalized instruction.
• Military: Battlefield simulation with individual participants, team training, and peace-keeping operations.
• Design/Maintenance: Design for access, ease of repair, safety, tool clearance, visibility, and hazard avoidance.

  
Figure 1: Virtual human applications, technology, and science.

In building models of virtual humans, there are varying notions of virtual fidelity. Understandably, these are application dependent. For example, fidelity to human size, capabilities, and joint and strength limits are essential to some applications such as design evaluation; whereas in games, training, and military simulations, temporal fidelity (real-time behavior) is essential. In our efforts we have attacked both.

Understanding that different applications require different sorts of virtual fidelity leads to the question of what makes a virtual human ``right''?

• What do you want to do with it?
• What do you want it to look like?
• What characteristics are important to success of the application?

In a very general way, we can characterize the state of virtual human modeling along at least five dimensions:

• Appearance: Cartoon shape -----+--> Physiologically accurate model
• Function: Cartoon actions -----+-> Human limitations
• Time: Off-line generation -----+-> Real-time production
• Autonomy: Direct animation -----+--> Intelligent
• Individuality: Specific person ---+----> Varying personalities

The University of Pennsylvania has been very actively engaged in research and development of human-like simulated figures. Our interest in human simulation is not unique, but the complex of activities surrounding our approach is. The framework for our research is a software system called Jack [3]. Jack is an interactive system for definition, manipulation, animation, and performance analysis of virtual human figures. Our philosophy has led to a particular realization of a virtual human model that pushes the above five dimensions toward the right:

• can be substituted for live individuals for workspace or cockpit evaluation.
• demonstrates various (useful) human limitations, constraints, and capabilities.
• may be moved live (in real-time) by position and orientation information or other motion generators such as walk-to or look-at.
• may have its actions synthesized by a program so that it can make its own decisions, navigate spaces, and so on.
• represents ``anyone'' rather than a single specific person or character.

Virtual humans are different than simplified cartoon and game characters. What are the characteristics of this difference and why are virtual humans more difficult to construct? After all, anyone who goes to the movies can see marvelous synthetic characters (aliens, toys, dinosaurs, etc.), but they have been created typically for one scene or one movie and are not meant to be re-used (except possibly by the animator -- and certainly not by the viewer). The difference lies in the interactivity and autonomy of virtual humans. What makes a virtual human human is not just a well-executed exterior design but movements, reactions, and decision-making which appear ``natural,'' appropriate, and contextually-sensitive.