(1995) Instructional Science( pp. ).

^[1]IMMERSIVE TRAINING SYSTEMS:

VIRTUAL REALITY AND EDUCATION AND TRAINING

Joseph Psotka, Ph.D.

U. S. Army Research Institute

ATTN: PERI-IIC

2511 Jefferson Davis

Arlington, VA 22202

(703)602-7945

Psotka@ari.army.mil

What is VR? 3

Immersive VR 3

The Technology of VR 4

HMD 4

Tracking 4

Gestures and Force Feedback 4

Stereo Sound 5

Voice Synthesis and Recognition 5

Smell 5

The Psychological Experience of Immersion 5

The Benefits of Immersion 6

Immersion and Visual Perspective 7

Allocentric Viewing 8

Immersion and Field of View (FOV ) 8

Viewpoint Manipulation 10

VR and Motion Platforms 10

Motion Cues 11

VR, Instruction, and Transfer 12

VR and Intelligent Tutoring Systems 13

VR and Simulations 13

Situated Learning Through VR 14

Some Examples 14

VR and Science 14

An Education Scenario: A Day in the Life of ... 15

A Maintenance Training Scenario: MACH-III 16

Augmented Reality vs Virtual Reality 16

Augmented reality limitations 16

Virtual Reality 17

A Real Life Training Example 17

Networked Virtual Reality 18

Other Future Issues 18

Authoring systems 18

Spatial Browsers and Abstract Displays 19

Equity 19

Edutainment 19

Organizational Change 19

Summary 19

References 19

In the half dozen years since a previous thorough overview of intelligent tutoring and computer - based instruction ( Nickerson and Zodhiates, 1988) the change of technologies has been breath-taking. Although we knew back then that what we were doing on expensive Lisp machines would soon be possible on ordinary personal computers, it is still unnerving to see not only that it is now possible, but that so much more is possible. The virtual reality (VR) technologies that have transformed the landscape in the intervening years (e.g. Rheingold, 1991) offer unique new viewpoints on the core goals of training and education. What distinguishes VR from all preceding technology is the sense of immediacy and control created by immersion: the feeling of ”being there” or presence that comes from a changing visual display dependent on head and eye movements. This paper will provide an introduction to the technology of VR and its possibilities for education and training. It will focus on immersion as the key added value of VR, and begin to analyze what cognitive variables are connected to immersion, how it is generated in synthetic environments, what immersion is, and what its benefits are. It is clear that a principled program of research is needed to uncover the instructional conditions that VR is best suited for, over other available media and technologies, if this new technology is to be used wisely and effectively. The central research question is the value of tracked, immersive visual displays over non-immersive simulations. The paper will provide a brief overview of existing VR research on training and transfer, education, and procedural, cognitive and maintenance training. It will close with an examination of important future issues: augmented reality, networked VR, edutainment, authoring systems, and equity.

What is VR?

There really are two kinds of VR, although in some ways they are complementary an indistinguishable. The two basic varieties are sensory immersive VR and text-based networked VR. This paper will deal mainly with visually immersive VR, the kind that makes your view of the world change when you move your head, and call text-based networked VR “Cyberspace”, to distinguish it pragmatically here. Although both are very useful for education and training, Cyberspace is better handled as an aspect of distance learning (Hunter, 1993). Another variety of VR, desktop VR or “fish tank VR” is not immersive (Ware, Kevin, and Kellogg, 1993), and so it is treated as another form of simulation technology in this paper. It may have a special use for abstract visualization. In some sense, though, it is similar to immersive VR, except that it partitions a smaller amount of the surrounding space than wide field of view (FOV) VR. No kind of VR was explicitly mentioned by Nickerson and Zhodiates such a short time ago as 1988. Now there are many books available on the topic, ranging from popular anecdotal overviews (e.g. Krueger, 1991; and Rheingold, 1991; ), collections of research papers (Benedikt, 1991; Earnshaw, Gigante, & Jones , 1993; Ellis, 1991; VRAIS’93, 1993; and Wexelblatt, 1993), discussions of the social and educational implications of the technologies ( Laurel, 1991; Turkle, 1993), analyses of the educational implications (Middleton, 1992), overviews of the hardware and software technologies (Pimentel and Teixeira, 1992), and stories of homebrew VR ( Jacobson, 1994), or detailed scientific documentation (Kaslawsky, 1993 ).

Immersive VR

Immersive VR can be defined by its technology and its effects. Its primary effect is to place a person into a simulated environment that looks and feels to some degree like the real world. A person in this synthetic environment has a specific sense of self - location within it, can move her or his head and eyes to explore it, feels that the space surrounds her or him, and can interact with the objects in it. In immersive VR, simulated objects appear solid and have an egocentric location much like real objects in the real world. They can be picked up, examined from all sides, navigated around, heard, smelled, touched, hefted, and explored in many sensory ways. The objects can also be autonomous (especially if they are other people) and interact with the virtual voyager, or respond to voice commands (Middleton and Boman, 1994). The fundamental limitation to all these effects is in the computational technology that supports them.

The Technology of VR

The technology of VR is rapidly changing and improving within a very active research community (VRAIS’93, 1993). The following sections discuss some of the more important components of this technology for current working environments. The core technology that makes immersive Virtual Reality possible, the head mounted display (HMD), is progressing particularly fast, with projections common that eyeglass - size and weight HMDs will be available by the turn of the century (Chien and Jenkins, 1994).

HMD: The essential ingredient of VR is a tracked head - mounted display (HMD) that lets you see new views of the visual world as you move your head. Wearing an HMD, one can look around and see the rest of the simulated world just like in the real world. Current image generation computers are limited in their ability to create a realistic, changing world. Special image generators cost hundreds of thousands of dollars, and the special lightweight, high-resolution displays can be equally expensive. Current microcomputers can realistically generate only a few thousand polygons per second, while it has been estimated that nearly a billion polygons per second may be needed for near realism. These limitations not only lead to low resolution and cartoon - like shapes, they also lead to long lags between changes in the head position and updates of the display. Narrow fields of view (often about half the normal field of view of 180 degrees) lead to distortions of perceived space, to inaccurate self - localization (Psotka, Davison, and Lewis, 1993), errors in the judgement of distances (Henry and Furness, 1993), and simulator sickness (feelings of discomfort that can range from mild eyestrain and headaches to nausea and vomiting)(Kennedy, Lane, Lilienthal , Berbaum, and Hettinger, 1993).

Tracking: An unobtrusive tracking mechanism, (magnetic, mechanical, infrared, gyroscopic, sound, or based on many innovative alternatives) registers any head motion and provides the signals to a computer to make the required changes in viewpoint in the modelled display. When your head moves, the visual scene changes. The result is a change of viewpoint just as if the eyes and head had moved in the virtual world. In advanced systems the scene changes when your eyes move. Such eye tracking is often used to provide a more detailed “fovea” or Area Of Interest (AOI) display (Warner, Serfoss, and Hubbard, 1993) of high resolution imagery that tracks the viewpoint. Any of these tracked displays usually result in a compelling sense of “being there”, of being immersed in the simulation as if it is a real world. Long lags between any user’s action and the resulting computed change in the display unfortunately often destroy this illusion and can lead to simulator sickness.

Gestures and Force Feedback: Gloves to gesture and interact with objects, and force - reflective feedback all add to the compellingness of the experience. They add to the willingness of a participant to suspend disbelief so that they can become immersed, but the main core of the experience is still primarily visual. Tactile reinforcement of the presence of an object, its shape, weight, solidity, and texture, adds considerably to the experience. Force feedback about the collisions with objects is a fundamental aid to navigation in VR: It prevents you from going through walls and floor, and other objects. Otherwise such sudden unnatural transitions often lead to disorientation and confusion. Gestures based on sensing of hand position and shape provide a natural means for interacting and communicating with the computer. For instance, one can select a distant object simply by pointing at it. Sometimes this selection is facilitated by having a ray extrude from a finger to the object. Others have suggested that one should be able to select objects by throwing something at them.

Stereo Sound: Localizing objects from stereo sound adds to the sense of presence and immersion. Unfortunately, accurate localization depends on the shape of each individual’s pinna or outer ear, so only ambiguous localization is currently possible.

Voice Synthesis and Recognition: Voice input and output capabilities are progressing rapidly and may soon be added to general VR environments, but remain currently largely unexploited. Magee (1994) has used them effectively in a VR training simulator for Navy ship commanders. Middleton and Boman (1994) have conducted a practically and theoretically ground breaking study of the conditions in a VR environment where voice recognition is useful. They observed that voice is best used for discrete changes in the environment, such as “Put me near object X”, but not as good for continuously varying dynamic dimensions such as the direction or speed of one’s flight.

Smell: There are many different ways to use odors to create a striking sense of presence. The technology of delivering odors has been well-developed (Varner, 1993) in trials at Southwest Research Institute. The system uses a microencapsulation technique that can be dry packaged in cartridges that are safe and easy to handle. Human chemical senses such as taste and smell create particularly salient memories. They are also useful for alerting us to danger, sexual arousal, and emotional experience.

The Psychological Experience of Immersion

In spite of the many technological limitations, many VR environments easily create a compelling sense of “being there”, of presence or immersion. The psychological and human interface issues that affect immersion are beginning to be analyzed by several researchers (Barfield and Weghorst, 1993; Psotka and Davison, 1993; Psotka and Calvert, 1994; Slater and Usoh, 1993). Clearly the burdensome equipment and limited motion often stir feelings of claustrophobia to reduce the sense of immersion, and open the way to simulation sickness (cf. Kennedy, Lane, Lilienthal, Berbaum, and Hettinger, 1992).

Immersion seems to be facilitated by the ability to control attention and focus on the new VR to the exclusion of the real world. Being able to see parts of one’s own body, even in cartoon form, adds to the experience. It also depends on the use of a good visual imagination. There is a great range of individual differences in the experience of immersion in VR environments. The technological limitations are largely responsible, but temperamental differences among individuals result in different reactions to these limitations. Perhaps if the technological limitations of burdensome equipment, lack of detail, and slow computers were overcome, these individual differences would disappear. But some difficulty may still remain to destroy the illusion, because voyagers will always possess the knowledge that it is all virtual. Even slight disturbances in the VR environment, such as obtrusively measuring heartrate, destroy the experience (Psotka and Calvert, 1994; See Figure 1.).