State of the Art

Augmented Reality needs three basic subsystems. Those are the same as in Virtual Reality.

• Scene generator
• Display device
• Tracking and sensing

Rendering is not a major problem of AR because in most application only a few objects have to be drawn. The display device and tracking are very sensitive parts of the system. If the tracking is imprecise or the a low-quality display is used, the illusion of augmention is destroyed. So various techniques to handle these two problems are described in this section.

Display Device

See through HMD

Figure 2: Optical see through HMD

See through HMDs use optical combiners to mix the real world's image, and the virtual image from monitors. Fig. 2.1 shows a simple diagram of a see through HMD. The opaque displays reduce the amount of light from the real world by about 30%.

See through has the advantage, that the resolution of the real world is not limited by the resolution of the displays. In most applications there are only a few virtual objects. It isn't that bad, if only the virtual objects are displayed in low resolution. So the resolution of the displays is not a major problem.

There is a delay between the real and the virtual image. The image of the real world is seen without any delay. But the virtual image has a lag, because it must pass the tracker, the scene generator and the monitor. This end-to-end system delay depends of the used hardware. A 50 ms delay is typical to midrange systems used today. This delay gets only a problem if there is some motion. So this is a dynamic error. One method to reduce this effect, is to predict the users next position[2].

Video see through HMD

Figure 3: Video see through HMD

Video see through HMDs use a closed view HMD. Those closed view HMDs are well known from virtual reality. Two cameras are mounted on the head, and the virtual image from the scene generator is combined with the image delivered by the cameras. Fig. 2.2 shows the concept of the video see through HMDs.

There are no dynamic errors with this device, because the real image can be delayed to have the same lag as the virtual one. But the delay between mechanical motion and the seen motion can cause motion sickness. Closed loop Augmented Reality (tracking and augmention is performed on the same image) can be done rather easily, because no additional camera is needed. A big disadvantage is that the real world image has the same (low) resolution than the display has. But it's easier to control visual behavior like brightness or shadows because the real world's image can be manipulated. And the virtual image can complete overpaint the real image. With through HMDs only bright objects can overpaint only the reality, because 30% of the real worlds image and 70% of the virtual image can bee seen in the display. In bright environments even rather bright objects cannot overpaint the real objects completely.

Monitor based

Monitor based systems are also possible. It's similar to the video see through HMD. But the Monitor is fixed, and the cameras and the tracker are mounted on a robot[3]. The user can control the robot. But this type of system is not very common. Projection walls or Cave systems can also be used for Augmented Reality. The ARGOS[5] project of the University of Toronto uses a monitor based AR system to control a robot.

Tracking

Mechanical

Mechanical trackers have a very high accuracy and reliability. But they are not flexible, and have a very limited range and are limited to one user. Fakespace produces some mechanical trackers.

Magnetic

Magnetic trackers use one fixed transmitter, and sensors. They are used rather often, because they are very robust. But the error increases with larger distances. They are also disturbed by ferro-magnetic objects and other magnetic fields. The tracking systems of Polhemus are magnetic trackers. Another magnetic tracking system is to use the magnetic field of the earth. This system shows very bad accuracy, but is not limited in it's range and very easy to setup.

Optical

Optical tracker aren't that robust but have a very high accuracy. And they need a powerful computer to keep tracking time short. There are two ways of tracking. Mounting a camera on a helmet and tracking fixed features (for example LEDs on the wall). Or mounting features on the helmet tracked by fixed cameras.

Acoustic

Acoustic tracker systems use ultra sonic signals. But to work properly, a large number of transponders is needed on known location. The system delay is rather high. As the optical tracker the acoustic needs line of sight to work. Possible systems are the Intersense IS-600 / IS-900.

GPS

The Global Positioning System (GPS) can be used for outdoor tracking. It should work on every place on earth. But it's sensors need contact to the sattelites. This is a problem in buildings. It's accuracy is very limited to about 10-100 meters. Differential GPS can achieve accuracy of about 0.1-1 meters. But this is by far not enough for most applications.

Inertial

Accelerometers and gyroscopes are used to record users movements. Because the position can not be recorded directly but is integrated over all recorded data. The result tends to drift with time and gets imprecise. This sort of system is well suited for outdoor applications in combination with portable systems.

Hybrid

Hybrid tracker try to combine the strengths of at least two other tracking methods. All combinations are possible. Examples are inertial and optical or magnetic and optical tracking like in Auer's work[1]. Hybrid systems are complex because two systems have to be handled and combined. But they show the best results, and will be the widely used in the future.