3D Interaction
Encyclopedia
In computing
, 3D interaction is a form of human-machine interaction
where users
are able to move and perform interaction
in 3D space
. Both human and machine process information where the physical position of elements in the 3D space is relevant.
The 3D space used for interaction can be the real physical space, a virtual space
representation simulated in the computer, or a combination of both. When the real space is used for data input, humans perform actions or give command
s to the machine using an input device
that detects the 3D position of the human action. When it's used for data output
, the simulated 3D virtual scene is projected onto the real environment through one output device
or a combination of them.
invented the Sensorama
simulator. It provided 3D video feedback, as well motion, audio, and haptic feedbacks to produce a virtual environment. The next stage of development was Dr. Ivan Sutherland's
completion of his pioneering work in 1968. He created a head-mounted display that produced a 3D, virtual environment by presenting a left and right still image of that environment.
Availability of technology as well as impractical costs held back the development and application of virtual environments until the 1980s. Applications were limited to military ventures in the United States. Since then, further research and technological advancements have allowed new doors to be opened to application in various other areas such as education, entertainment, and manufacturing.
3D interfaces have been used in applications that feature virtual environments, and augmented
and mixed realities
. In virtual environments, users may interact directly with the environment or use tools with specific functionalities to do so. 3D interaction occurs when physical tools are controlled in 3D spatial context to control a corresponding virtual tool.
Users experience a sense of presence when engaged in an immersive virtual world. Enabling the users to interact with this world in 3D allows them to make use of natural and intrinsic knowledge of how information exchange takes place with physical objects in the real world. Texture, sound, and speech can all be used to augment 3D interaction. Currently, users still have difficulty in interpreting 3D space visuals and understanding how interaction occurs. Although it’s a natural way for humans to move around in a three-dimensional world, the difficulty exists because many of the cues present in real environments are missing from virtual environments. Perception and occlusion are the primary perceptual cues used by humans. Also, even though scenes in virtual space appear three-dimensional, they are still displayed on a 2D surface so some inconsistencies in depth perception will still exist.
s are the means for communication between users and systems. 3D interfaces include media for 3D representation of system state, and media for 3D user input or manipulation. Using 3D representations is not enough to create 3D interaction. The users must have a way of performing actions in 3D as well. To that effect, special input and output devices have been developed to support this type of interaction. Some, such as the 3D mouse, were developed based on existing devices for 2D interaction.
Standard input devices include keyboards, tablets and stylus, joysticks, mice, touch screens, knobs, and trackballs.
Trackers detect or monitor head, hand or body movements and send that information to the computer. The computer then translates it and ensures that position and orientation are reflected accurately in the virtual world. Tracking is important in presenting the correct viewpoint, coordinating the spatial and sound information presented to users as well the tasks or functions that they could perform. 3D trackers have been identified as mechanical, magnetic, ultrasonic, optical, and hybrid inertial. Examples of trackers include motion trackers, eye trackers, and data gloves.
A simple 2D mouse may be considered a navigation device if it allows the user to move to a different location in a virtual 3D space. Navigation devices such as the treadmill
and bicycle
make use of the natural ways that humans travel in the real world. Treadmills simulate walking or running and bicycles or similar type equipment simulate vehicular travel. In the case of navigation devices, the information passed on to the machine is the user’s location and movements in virtual space.
Wired glove
s and bodysuits allow gestural interaction to occur. These send hand or body position and movement information to the computer using sensors.
) are examples of a fully immersive visual display, where the user can see only the virtual world and not the real world. Semi-immersive displays allow users to see both. Monitors and workbenches are examples of semi-immersive displays. Auditory displays provide information in auditory form. This is especially useful when supplying location and spatial information to the users. Adding background audio component to a display adds to the sense of realism. Haptic displays send tactile feedback or feeling back to the user.
over the past 3 years regarding interactive surfaces & classroom interactive whiteboards, grade school students, and 3D natural user interfaces known as Edusim
.
Techniques that support system control tasks in three-dimensions are classified as:
Computing
Computing is usually defined as the activity of using and improving computer hardware and software. It is the computer-specific part of information technology...
, 3D interaction is a form of human-machine interaction
Human-machine interaction
Human–machine Interaction is the interaction between machines and the persons who operate them. An old-fashioned term is is Man-machined Interaction - Related terms:...
where users
User (computing)
A user is an agent, either a human agent or software agent, who uses a computer or network service. A user often has a user account and is identified by a username , screen name , nickname , or handle, which is derived from the identical Citizen's Band radio term.Users are...
are able to move and perform interaction
Interaction
Interaction is a kind of action that occurs as two or more objects have an effect upon one another. The idea of a two-way effect is essential in the concept of interaction, as opposed to a one-way causal effect...
in 3D space
Three-dimensional space
Three-dimensional space is a geometric 3-parameters model of the physical universe in which we live. These three dimensions are commonly called length, width, and depth , although any three directions can be chosen, provided that they do not lie in the same plane.In physics and mathematics, a...
. Both human and machine process information where the physical position of elements in the 3D space is relevant.
The 3D space used for interaction can be the real physical space, a virtual space
Virtual reality
Virtual reality , also known as virtuality, is a term that applies to computer-simulated environments that can simulate physical presence in places in the real world, as well as in imaginary worlds...
representation simulated in the computer, or a combination of both. When the real space is used for data input, humans perform actions or give command
Command (computing)
In computing, a command is a directive to a computer program acting as an interpreter of some kind, in order to perform a specific task. Most commonly a command is a directive to some kind of command line interface, such as a shell....
s to the machine using an input device
Input device
In computing, an input device is any peripheral used to provide data and control signals to an information processing system such as a computer or other information appliance...
that detects the 3D position of the human action. When it's used for data output
Input/output
In computing, input/output, or I/O, refers to the communication between an information processing system , and the outside world, possibly a human, or another information processing system. Inputs are the signals or data received by the system, and outputs are the signals or data sent from it...
, the simulated 3D virtual scene is projected onto the real environment through one output device
Output device
An output device is any piece of computer hardware equipment used to communicate the results of data processing carried out by an information processing system to the outside world....
or a combination of them.
Background
3D’s early beginnings can be traced back to 1962 when Morton HeiligMorton Heilig
Morton Heilig was a thought-leader in Virtual Reality . He applied his cinematographer experience and with the help of his partner developed the Sensorama over several years from 1957, patenting it in 1962....
invented the Sensorama
Sensorama
The Sensorama was a machine that is one of the earliest known examples of immersive, multi-sensory technology. Morton Heilig, who today would be thought of as a “multimedia” specialist, in the 1950s saw theater as an activity that could encompass all the senses in an effective manner, thus...
simulator. It provided 3D video feedback, as well motion, audio, and haptic feedbacks to produce a virtual environment. The next stage of development was Dr. Ivan Sutherland's
Ivan Sutherland
Ivan Edward Sutherland is an American computer scientist and Internet pioneer. He received the Turing Award from the Association for Computing Machinery in 1988 for the invention of Sketchpad, an early predecessor to the sort of graphical user interface that has become ubiquitous in personal...
completion of his pioneering work in 1968. He created a head-mounted display that produced a 3D, virtual environment by presenting a left and right still image of that environment.
Availability of technology as well as impractical costs held back the development and application of virtual environments until the 1980s. Applications were limited to military ventures in the United States. Since then, further research and technological advancements have allowed new doors to be opened to application in various other areas such as education, entertainment, and manufacturing.
3D interaction
In 3D interaction, users carry out their tasks and perform functions by exchanging information with computer systems in 3D space. It is an intuitive type of interaction because humans interact in three dimensions in the real world. The tasks that users perform have been classified as selection and manipulation of objects in virtual space, navigation, and system control. Tasks can be performed in virtual space through interaction techniques and by utilizing interaction devices. 3D interaction techniques were classified according to the task group it supports. Techniques that support navigation tasks are classified as navigation techniques. Techniques that support object selection and manipulation are labeled selection and manipulation techniques. Lastly, system control techniques support tasks that have to do with controlling the application itself. A consistent and efficient mapping between techniques and interaction devices must be made in order for the system to be usable and effective. Interfaces associated with 3D interaction are called 3D interfaces. Like other types of user interfaces, it involves two-way communication between users and system, but allows users to perform action in 3D space. Input devices permit the users to give directions and commands to the system, while output devices allow the machine to present information back to them.3D interfaces have been used in applications that feature virtual environments, and augmented
Augmented reality
Augmented reality is a live, direct or indirect, view of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphics or GPS data. It is related to a more general concept called mediated reality, in which a view of reality is...
and mixed realities
Mixed reality
Mixed reality refers to the merging of real and virtual worlds to produce new environments and visualisations where physical and digital objects co-exist and interact in real time...
. In virtual environments, users may interact directly with the environment or use tools with specific functionalities to do so. 3D interaction occurs when physical tools are controlled in 3D spatial context to control a corresponding virtual tool.
Users experience a sense of presence when engaged in an immersive virtual world. Enabling the users to interact with this world in 3D allows them to make use of natural and intrinsic knowledge of how information exchange takes place with physical objects in the real world. Texture, sound, and speech can all be used to augment 3D interaction. Currently, users still have difficulty in interpreting 3D space visuals and understanding how interaction occurs. Although it’s a natural way for humans to move around in a three-dimensional world, the difficulty exists because many of the cues present in real environments are missing from virtual environments. Perception and occlusion are the primary perceptual cues used by humans. Also, even though scenes in virtual space appear three-dimensional, they are still displayed on a 2D surface so some inconsistencies in depth perception will still exist.
3D user interfaces
User interfaceUser interface
The user interface, in the industrial design field of human–machine interaction, is the space where interaction between humans and machines occurs. The goal of interaction between a human and a machine at the user interface is effective operation and control of the machine, and feedback from the...
s are the means for communication between users and systems. 3D interfaces include media for 3D representation of system state, and media for 3D user input or manipulation. Using 3D representations is not enough to create 3D interaction. The users must have a way of performing actions in 3D as well. To that effect, special input and output devices have been developed to support this type of interaction. Some, such as the 3D mouse, were developed based on existing devices for 2D interaction.
Input devices
Input Devices are instruments used to manipulate objects, and send control instructions to the computer system. They vary in terms of degrees of freedom available to them and can be classified into standard input devices, trackers, control devices, navigation equipment, and gesture interfaces.Standard input devices include keyboards, tablets and stylus, joysticks, mice, touch screens, knobs, and trackballs.
Trackers detect or monitor head, hand or body movements and send that information to the computer. The computer then translates it and ensures that position and orientation are reflected accurately in the virtual world. Tracking is important in presenting the correct viewpoint, coordinating the spatial and sound information presented to users as well the tasks or functions that they could perform. 3D trackers have been identified as mechanical, magnetic, ultrasonic, optical, and hybrid inertial. Examples of trackers include motion trackers, eye trackers, and data gloves.
A simple 2D mouse may be considered a navigation device if it allows the user to move to a different location in a virtual 3D space. Navigation devices such as the treadmill
Treadmill
A treadmill is an exercise machine for running or walking while staying in one place. The word treadmill traditionally refers to a type of mill which was operated by a person or animal treading steps of a wheel to grind grain...
and bicycle
Bicycle
A bicycle, also known as a bike, pushbike or cycle, is a human-powered, pedal-driven, single-track vehicle, having two wheels attached to a frame, one behind the other. A person who rides a bicycle is called a cyclist, or bicyclist....
make use of the natural ways that humans travel in the real world. Treadmills simulate walking or running and bicycles or similar type equipment simulate vehicular travel. In the case of navigation devices, the information passed on to the machine is the user’s location and movements in virtual space.
Wired glove
Wired glove
A wired glove is an input device for human–computer interaction worn like a glove.Various sensor technologies are used to capture physical data such as bending of fingers. Often a motion tracker, such as a magnetic tracking device or inertial tracking device, is attached to capture the global...
s and bodysuits allow gestural interaction to occur. These send hand or body position and movement information to the computer using sensors.
Output devices
Output devices allow the machine to provide information or feedback to the user. They include visual displays, auditory displays, and haptic displays. Visual displays provide feedback to users in 3D visual form. Head-mounted displays and CAVEs (Cave Automatic Virtual EnvironmentCave Automatic Virtual Environment
A Cave Automatic Virtual Environment is an immersive virtual reality environment where projectors are directed to three, four, five or six of the walls of a room-sized cube...
) are examples of a fully immersive visual display, where the user can see only the virtual world and not the real world. Semi-immersive displays allow users to see both. Monitors and workbenches are examples of semi-immersive displays. Auditory displays provide information in auditory form. This is especially useful when supplying location and spatial information to the users. Adding background audio component to a display adds to the sense of realism. Haptic displays send tactile feedback or feeling back to the user.
3D interaction techniques
3D interaction techniques are methods used in order to execute different types of task in 3D space. Techniques are classified according to the tasks that they support.Selection and manipulation
Users need to be able to manipulate virtual objects. Manipulation tasks involve selecting and moving an object. Sometimes, rotation of the object is involved as well. Direct-hand manipulation is the most natural technique because manipulating physical objects with the hand is intuitive for humans. However, this is not always possible. A virtual hand that can select and re-locate virtual objects will work as well. 3D widgets can be used to put controls on objects. Users can employ these to re-locate or re-orient an object. Other techniques include the Go-Go technique and ray casting, where a virtual ray is used to point to, and select and object. More recently there has been user interface development and research by Richard White in KansasKansas
Kansas is a US state located in the Midwestern United States. It is named after the Kansas River which flows through it, which in turn was named after the Kansa Native American tribe, which inhabited the area. The tribe's name is often said to mean "people of the wind" or "people of the south...
over the past 3 years regarding interactive surfaces & classroom interactive whiteboards, grade school students, and 3D natural user interfaces known as Edusim
Edusim
Edusim is a Cave Automatic Virtual Environment "Immersive Touch" 3D Natural User Interface based concept of lesson driven 3D virtual worlds on the classroom interactive whiteboard or classroom interactive surface...
.
Navigation
The computer needs to provide the user with information regarding location and movement. Navigation tasks have two components. Travel involves moving from the current location to the desired point. Wayfinding refers to finding and setting routes to get to a travel goal within the virtual environment.- Wayfinding : WayfindingWayfindingWayfinding encompasses all of the ways in which people and animals orient themselves in physical space and navigate from place to place.-Historical:...
in virtual space is different and more difficult to do than in the real world because synthetic environments are often missing perceptual cues and movement constraints. It can be supported using user-centred techniques such as using a larger field of view and supplying motion cues, or environment-centred techniques like structural organization and wayfinding principles.
- Travel : Good travel techniques allow the user to easily move through the environment. There are three types of travel tasks namely, exploration, search, and manoeuvring. Travel techniques can be classified into the following five categories:
- Physical movement – user moves through the virtual world
- Manual Viewpoint manipulation – use hand motions to achieve movement
- Steering – direction specification
- Target-based travel – destination specification
- Route planning – path specification
System control
Tasks that involve issuing commands to the application in order to change system mode or activate some functionality fall under the category of system control.Techniques that support system control tasks in three-dimensions are classified as:
- Graphical menus
- Voice commands
- Gestural interaction
- Virtual tools with specific functions
Symbolic input
This task allows the user to enter and/or edit, for example, text, making it possible to annotate 3D scenes or 3D objects.See also
- Interaction techniqueInteraction techniqueAn interaction technique, user interface technique or input technique is a combination of hardware and software elements that provides a way for computer users to accomplish a single task. For example, one can go back to the previously visited page on a Web browser by either clicking a button,...
- Interaction designInteraction designIn design, human–computer interaction, and software development, interaction design, often abbreviated IxD, is "the practice of designing interactive digital products, environments, systems, and services." Like many other design fields interaction design also has an interest in form but its main...
- Human–computer interactionHuman–computer interactionHuman–computer Interaction is the study, planning, and design of the interaction between people and computers. It is often regarded as the intersection of computer science, behavioral sciences, design and several other fields of study...
- Cave Automatic Virtual EnvironmentCave Automatic Virtual EnvironmentA Cave Automatic Virtual Environment is an immersive virtual reality environment where projectors are directed to three, four, five or six of the walls of a room-sized cube...
- Virtual realityVirtual realityVirtual reality , also known as virtuality, is a term that applies to computer-simulated environments that can simulate physical presence in places in the real world, as well as in imaginary worlds...
- 3D Natural User InputEdusimEdusim is a Cave Automatic Virtual Environment "Immersive Touch" 3D Natural User Interface based concept of lesson driven 3D virtual worlds on the classroom interactive whiteboard or classroom interactive surface...