1994 VR and Persons with Disabilities Conference Proceedings
By: Gregory Peter Panos, PersonaForm ( currently the Persona Foundation )
Abstract
Virtual Reality technology consists of a variety of methods to develop three dimensional data, to track and/or capture object and/or viewer motion information and to obtain human behavioral data. These techniques can be combined to create a "human data acquisition system" that serves to document and archive a human form, behavior and reaction profile. The use of virtual reality tools to create human archives will become an important and necessary application for a variety of future needs. The development of specialized systems that are optimized for the task of human data acquisition is already proceeding at PersonaForm in Lakewood, CA. The system and its intended uses are described here.
Typical Application Areas for Human Simulation:
While a number of commercial and research efforts have existed where the human form has been the target of simulation, much work has been highly directed at specific applications:
Entertainment
Cartoon characters have been successfully animated in real-time using advanced computer graphics technology. Many of the characters developed are anthropomorphic in nature, however, they have not been developed from real human data or to emulate a real human performer. Rising interest in the future uses of "Digital Actors" will create new developments and many actor-performers will wish to create accurate digital representations of themselves that they can license for use in a variety of interactive venues and film / TV special effects.
Motion Picture Special Effects
The demand for ever dazzling special effects in the stories being told by our creative talent is insatiable. Terminator 2, the Abyss, Lawnmower Man, Looker, WestWorld, and many other motion pictures have used the 3D human digitizing process as a tool to create exciting new transformational effects. The use of innovative technologies in these applications has been limited to the exact requirements of the story and director. Few tools have evolved out of their use for any other purpose or commercial development. Hollywood is very protective about their effects technologies and they often discard their developments from one project to the next, often to insure that the next creative challenge will exceed previous efforts and force a fresher look.
Medicine and Prosthetics
Human forms, of both body and head, have been digitized in 3 dimensions to assist in the development of prosthetics, as an aid to surgical procedures, and to assist in physical therapeutic treatments in a variety of ways. As 3D computer graphics technology is increasingly utilized in the medical environment, new and valuable uses for 3D data of human structures will continue to evolve.
Computer-Human Interface
A number of research efforts have been under way to enhance the interface between computer systems and their human user counterparts. The field of "Human Centered Computing" is just now evolving as what many believe will be the predominant way that we, as people, will work with the increasingly more complex systems that are being developed for our benefit. Digitized human voice synthesizers, facial animations with moving lips and eyes, and speech recognition are key areas being focused on through research and development efforts. Use of 3D digitized human databases to represent intelligent agent interfaces has not yet presented itself to the consumer, however, this is a key goal of the PersonaForm effort. We strongly believe that computers that look and act like familiar humans will be less threatening, more interesting, entertaining and easier to use. Apparently many computer companies share our hopes as they have developed crude interfaces that are rapidly moving in this exciting new direction.
Clothing Design
3D Human form digitizing has been used on occasion to create carefully fitted clothing. This concept will evolve further when the clothing design process becomes further automated and designers begin to accept the advantages that this fast, accurate human form digitizing technology can provide them in their work.
Fitness training
Very little has been done in the area of fitness training to use 3D human form digitization. This application will become one of the most widespread uses for this technology in the future. Use of motion capture, 3D morphing and 3D holographic projection will aid in the development of useful and effective applications for fitness training aids and services.
Missing Persons and Security Identification
Research has shown spectacular results in the area of documenting the human form, face and body in the field of missing persons and security identification. Some application of two dimensional, predictive aging of photo images of people is being used to identify missing children and to find criminals. We believe that 3D databases of humans can more effectively be used to create highly accurate descriptions of those persons that have aged and/or changed their appearance. Rapid and cost effective results can be created more easily and automatically with 3D sampling, manipulation and display.
Human Digitizing Requirements
At PersonaForm we have several key areas of focus. We believe that current 3D object surface acquisition, facial expression sampling and temporal motion capture technologies are, as yet, very primitive. Much work will have to be done to improve each of these necessary technologies to make the process of sampling and creating a believable human database construct (i.e.: PersonaForm) simple, effective, error-free and affordable. Below is an enumeration of several of the major discreet processes required to adequately archive a human being:
3D object surface scanning
Obtaining an accurate 3D digital representation of a complex object shape is a very difficult task to perform in an automated fashion within a reasonable time-frame. A variety of technologies have evolved which are capable of this task, however, each technique makes certain trade-offs in accuracy, rate of data acquisition, surface anomaly detection/resolution, general resolution and error rate. Use of non-contact, visible laser-based photogram metric range finding with simultaneous surface photo recording has been used recently to satisfy 3D object acquisition requirements. Currently available systems are fairly rapid, accurate enough for our purposes and can be corrected through human interface and adaptive intervention to solve problems encountered in unusual scanning situations.
At PersonaForm we believe that the crucial process of human form digitization must be made into an interesting, simple to understand and entertaining experience to the person being acquired. The system must "celebrate" them as it creates their recording. They must believe, through the scanning experience, that they are deriving great benefit from this wondrous celebration of their human form. Something akin to "a process of immortalization".
Facial expression sampling
Each of us is not only comprised of our physical shape but we exist as the collective manifestation of our specific human expressions as well, many of them facial. Sampling human facial expressions is an extremely important part of the human archiving process. Through our smiles, frowns, winks, and numerous other more subtle facial representations, we convey the reality of our existence to other humans. The process of recording each of these discreet expressions is evolving rapidly with a variety of exciting new techniques.
At PersonaForm, we are currently working with optical facial sampling technology which can rapidly and accurately capture a person's facial expressions. The system requires several small, infrared reflective markers to be worn by the subject during the recording process. This technique might appear to be perceived as an encumbrance to the wearer, however, tests have indicated that any discomfort or interference with the natural evolution and sampling of expression events is negligible. Once a number of expressions are elicited and sampled, the recordings are analyzed by the computer and reduced to digital motion representations of the various points on the face that were involved in the creation of each expression. The previously acquired 3D model of the person's head and face is then registered with the sampled points from their expression data and calibrated for storage into a single PersonaForm database.
Body Motion Capture
Much of what others perceive as specific to each of us can be represented by the way we move, walk, run, skip, sit down, get up, jump, stand idly and a variety of other body positions and motions. It is an important process to record many of our moves and positions and to use this information so a human simulation will be enhanced by the realism that these characteristics offer.
The field of human factors research has yielded a number of useful techniques to capture and process human motion. Some of the tools utilize magnetic tracking, others use optical motion analysis. We plan on using a hybrid approach to best allow the advantages of each technology to be combined most efficiently. As each movement of position is sampled, the data is corrected, conditioned and assigned to points on the original 3D database of the person's digitized form. Once all the motion data has been calibrated and registered with the body it is stored with the rest of the database.
Human Personality Encoding
Capturing human form, motion, and expression is a technical problem that many consider to be straightforward and solvable. Capturing the human personality, with all of its soul and secrets will probably be one of the only eternally unsolvable problems that mankind will ever truly encounter. Should direct brain interfacing become a reality in the future it might be easier to gather data from a human nervous system, however, the delicate and transitory construct that we call the mind and the even more elusive and somewhat debated existence of the soul may be out of reach to our scanners and elaborate sampling systems for all of eternity. Thus we are left with only that which we already know, the things that we believe to be special to us. Our memories of our experiences, the choices we've made in life, feelings of love, hate, pain, happiness, etc. are the things that really make us who we are to our selves and to those others that took the time to get to know us as real human spirits.
In this direction we are devising a comprehensive personality profile that is designed to acquire data on a number of key points: Favorite things, colors, music, art, places, people, songs, books, movies, TV shows, fictional characters, etc. Things we did not like, experiences we wanted to have, people we wanted to love or be loved by, Jobs we had, lost, never got, accidents, feeling of loss, and on and on and on. A subject would be interviewed by an automated kiosk and would , hopefully with truth, answer as many questions as possible. The results of this interview would provide enough data in order to allow us to create a fairly crude database that could be used as background information in a personality simulation. As technologies evolve and our understanding of human psychology progresses, we will improve the data gathering process to include more valuable and significant information that can help us insure fidelity and realism into human simulations.
The Process and Reasoning of Human Simulation
The simulation of a human being would presumable use all of the physically gathered data, the personality profile information and a series of artificial intelligence models to create a realistic, responsive, believable simulation of the person sampled. If the subject wanted, they could interact with their own simulation and calibrate its responses so it behaved in an even more reasonably accurate way, true to form. This is an extremely complex process that all of us take for granted in real life. Some ask why even try to duplicate a person in this way. It is our belief that the simulation of real people is a valuable and worthwhile task.
Technology is typically applied toward specific applications needs that are involved with our work, entertainment and physical environment management. Very little of our great new technology has been used to document, understand and to celebrate the human condition, existence and form. There is evidence that our need to document our lives with greater fidelity is increasing. Newer video camcorders available on the consumer market are showing up with features such as the "Age" button where one can set a number that shows up in the corner of the recorded image, presumably because people are buying these cameras to document the development of their children. As technology improves, people will wish to take full advantage of the documentation process and create archives not just of their experiences but with 3D data-recordings of their physical self.
Future Applications
A "PersonaForm" can be used to allow a person to self-calibrate. With it they can maintain a reference point into their own appearance, personality and behavior from a particular moment in their life. The use of this reference can be very valuable in rehabilitative therapy, recovering from an addiction, abuse or bad experience, etc. People often forget how they became who they are, the reference simulation could help them trace back and presumably help them in therapy.
Grief and the loss of a loved one is one of the most emotionally and sometimes physically devastating experiences that a person can go through and all of us eventually must deal with this experience up to and including the loss of our own self to death, the inevitable. We believe that if one were to have access to a human simulation of a loved one that has been lost, there could be a substantial "emotional cushion" that could be used to comfort one from the pain of separation and/or the reality of their loss. The use of their simulation could allow for a more gradual resolution and personal encounter to occur, thus softening the hard effects. Loss syndrome is a well documented condition that many suffer, usually due to accident, early terminal disease or disaster. Anything that could help one integrate and transcend the pain of loss could be extremely valuable toward maintaining the delicate balance of happiness and misery that we are all at risk to experience in life.
Children could experience their lost grandparents or older ancestors with greater fidelity, perhaps enhancing their own sense of self, heritage, ancestry and culture. Parents could re-experience the joy of their children who have since grown, moved away and centered their lives around others. All who have considered the great things that human simulation could provide, agree that it is a goal worthy of pursuing and that any risk would be outweighed by the benefits. Ultimately, the value of access to human simulations will have to be determined by each of us in our own time and for our own reasons.
Conclusion
The process of human simulation will be very commonplace in the future. This does not, however, reduce our need to begin the documentation process of real people today. The technology is just now good enough, affordable and being put into service. Many of our celebrities and famous figures will be the first to take advantage of the opportunity to create digital versions of themselves. This is, however, not our intended target base for such wonderful technology.
The general public with their own elderly, terminally ill and new young are the ones that have a vast and important need for human archiving. We wish to establish the value of this process and its benefits deeply into our traditional society, hopefully to fulfill a common spiritual need that each of share to adequately preserve their sacred existence, their journey through life.