Controlling a virtual city fly-through by optical tracking of inline-skaters

4. Optical beacon tracking (OBT)

As already mentioned, beacon tracking can be used to track 3, 5 or 6 degrees of freedom, depending on the number of markers tracked. Due to the actual problem of fixing 3 markers on an inline-skater's head in a way that the markers are appropriately arranged and do not interfere with each other, we chose the approach to track just a single marker per skater. This reduces the number of degrees of freedom to three, which means that we do not have any information about the orientation of the skater, hence we can not use movements like pirouettes automatically for our visualisation process. Up to this point we had already made many compromises, but we were forced to further reduce our high requirements because of the time-constraint.

However, we want to introduce the principles of beacon tracking as outlined in the papers from Franz Madritsch [1], [2].

The central idea of camera based beacon tracking is to determine the image coordinates of passive beacons captured by one or more cameras at video rates. Through centroid calculation the position of the beacon's image projection can be determined with sub-pixel accuracy.

OBT also offers flexibility concerning its working volume, in contrast to magnetic and mechanic tracking techniques. Accuracy, though not so important for us, is obviously determined by the working volume, the camera resolution, and the size of the beacons. Our tracking system was operating in a working volume of approximately 10 by 5 by 2 metres using standard low resolution CCD-cameras with 512 x 492 pixels and a marker size of approximately 10 by 10 cm. The marker was fixed on a cap worn by the inline-skater.

The first consideration in tracking an inline-skater is the question of whether the task can be done in real-time or not. If real-time image processing is achieved this indeed can be considered as a new medium for interactive input to human-computer interfaces. By using two identical tracking processes running on two off-the-shelf workstations (SGI Indy) we were able to do framerates of 20 Hz. This was mainly due to limited network bandwith, but still enough to offer "near" realtime processing.

In the next few paragraphs the different sub-processes of OBT are outlined. First of all the beacons have to be localised in the two images, this yields coordinates in image space. These 2-d coordinates are used to calculate the final 3-d coordinates.

As a marker captured by a CCD or video-camera undergoes a discretization both spatially and radiometrically, a criterion must be found to decide which pixels of the frame-buffer belong to a marker. If beacons differ in brightness or colour from the background a threshold function can be employed for segmentation. For this method to work it is necessary to make sure that the attributes assigned as beacon characteristics do not appear within the background, so the background was subtracted from the images by using a steady image of the empty stage. In our case the beacons consisted of retro-reflective materials plastered onto a cap, which reflect the spotlights from the sources placed next to the cameras. In this way the light is reflected directly back to the sensor and good contrast can be achieved. Once a beacon is extracted, its position has to be computed. To estimate the beacon's center we calculated its centroid.

However, locating the beacons in the image is a time consuming task. Algorithms processing the entire image would probably limit the performance of the system on sequential processors, especially when high resolution cameras are used. Neighbourhood searches are efficient since beacons are not expected to jump randomly within the image between two frames. Our approach towards an optical tracking strategy takes advantage of the correlation between adjacent frames. Initially we search through the whole image until the beacon is extracted. In the next frame we just search a square area of fixed size around the position of the beacon in the previous frame. If the beacon can not be found within this window, the rest of the image is searched [1].

To enable the algorithm to find the markers and to calculate correct positions, the quality of the images had to be sufficiently high. To obtain this necessary level of quality, the cameras had to be aligned and calibrated very carefully.

Alignment was achieved by locating the exact centre of the stage, referring to this point as the world coordinates origin. A lot of tests had to be carried out to calibrate the cameras properly. We had to experiment with different light sources, different colours of reflectors, and we had to change several camera parameters, such as shutter-speed, gain or saturation.

Previous Chapter

Next Chapter