Controlling a virtual city fly-through by optical tracking of inline-skaters

2. Introduction

First of all, we want to introduce the ideas lying behind the TV-show, in order to understand the context of our task. The show, completely conducted by "Haydn Film", Graz, developed from the idea to bring high-end technics and recent scientific research to people, who are not familiar with science but are familiar with the products, which are due to science. This model for a television show was proposed to the committee of the "European Week for Scientific and Technological Culture" in order to obtain funds and support for the show. Finally funds were received and the idea which was already born came true.

The aim of the show was to bring the most recent advances in automobile research to the TV-audience, hence the subtitle of the show: "Cybercar - the car of the future". The whole show was to be a mixture of serious discussion between car manufacturers on one hand and show-, entertainment- and music-blocks on the other hand. Inline-skaters were to be a part of these show-blocks, to demonstrate a new paradigm of easy, free and uncomplicated movement through cities. The show should contain different "types" of inline-skaters, like figure-skaters, aggressive-skaters, and street skaters.

In the context of the inline-skaters, an other idea was born. The director of the show wanted to use a big screen to visualise a virtual city while the inline-skaters were doing their performance. Movements of the inline-skaters should be seen on the screen as movements through the city. Especially during the performance of figure-skaters the arrangement of sound-effects, music, skater movements and the projected city should match together, i.e. heavy sound effects from cars and unsteady, nervous and anxious movements of the skaters should produce a sort of grey, dark and misty city scenario. As the performance continues a kind of soft music starts and the skaters start to loose their anxiety as they begin with "round" movements. The virtual city changes from dark and grey toy-blocks to colourful and textured houses and the fog disappears.

Now we are at a point, where we can start to talk about the project we had to solve in more detail. Again, our task was to visualise a virtual-city fly-through, while inline-skaters were performing on a triangle-shaped stage. The movements of the inline-skaters should have a recognisable impact on the fly-through, such that people watching the show could easily notice the connection between the poses and movements of the skaters and the pictures seen on the big screen on the back end of the stage. This was indeed not an easy task to solve, and we will discuss this issue in a later chapter in more detail.

The thing we had to consider first was the question of how to track the position of an inline-skater. We had several theoretical possibilities, only a few of them were worthwhile discussing, because one big constraint of the show was that our project team had only about 5 weeks of time available to solve the problem. Several tracking techniques proposed in the past could have been used, like active contour tracking, window based feature tracking, or beacon tracking based on image processing. Other tracking techniques like magnetic tracking were not appropriate because the setting of the stage was too big.

Besides the time-constraint we had the other big requirement of real-time tracking. No experience was available on contour and feature tracking, but a well developed area of research at our institute are techniques based on beacon tracking [1], [2]. Former experiments with this technique were conducted in a much smaller setting, i.e. only about a meter of distance between cameras and the beacons to detect. The setting we had to become familiar with, was much bigger, i.e. a stage, about 10 metres in length and cameras mounted at a height of approximately 9 metres (see section 2). This particularly had an impact on the images we received from our cameras. To use cameras for beacon-tracking in such a big setting was definitely quite risky due to the fact that the luminous intensity of the objective is quite poor. When using beacon tracking on the other hand, one does not have to worry too much about good contrast and good luminous intensity because the only thing of interest is the beacon to track.

Another issue to be solved was the question of whether to use active or passive markers for beacon detection. Active markers are easy to detect due to the high light intensity they emit but require some kind of battery pack carried by the skater. Passive markers, e.g. reflectors can be added easily to the clothes or the helmet of the skater but are not easy to detect especially when not enough light is available on stage. After a lot of tests with different materials and active lamps our decision was to use passive markers due to their ease of use. The major drawback of active markers is, as already mentioned, the problem of fixing a number of lamps on an inline-skater, such that the audience does not think of a "Christmas tree" when observing the skater. Simple LED's would have been much too small in order to be detected by cameras which are mounted about 12 metres away.

We also had to think about the question, how many degrees of freedom we will be able to track. 6 DOF require at least 3 independent markers to be tracked, but how could we fix three distinguishable markers onto an inline-skater? Reducing our initial high requirement of 6 DOF tracking to only one marker would have meant that only 3D position tracking remained. Thus all information about skater-orientation would have been lost.

In the forthcoming sections we want to talk about the principles introduced in this section in more detail. Chapter 2 contains information about the hardware, software and network configuration. Chapter 3 covers beacon tracking in more detail. Chapter 4 talks about the software packages used, the development of a "fly-through generator", the conversion of inline-skater movements into reasonable fly-through data and the program to build arbitrary virtual cities.

Previous Chapter

Next Chapter