Image Based Lighting in Augmented Reality

Authors

Peter Supan
Upper Austria University of Applied Sciences, Hagenberg/Austria
Email: dm04027@fh-hagenberg.at
Ines Stuppacher
Upper Austria University of Applied Sciences, Hagenberg/Austria
Email: dm04026@fh-hagenberg.at

Abstract

Our work focuses on using Image Based Lighting in Augmented Reality applications. In our application the real environment has immidiate effects on the virtual objects. It is possible for us to render reflective objects and see our own reflection in the objects, updated 30 times per second. Moreover the virtual objects cast soft shadows on the real floor and on each other. The virtual shadows resemble the real ones because the real light sources are detected in the environment.

Introduction

In this paper we present the use of realtime Image Based Lighting in an Augmented Reality (AR) setup. Augmented Reality is a way to extend the real world with a virtual overlay. This could be simple information about the real world or photorealistic renderings of objects like car prototypes or architectural models.

In many AR applications it is desirable for virtual objects to fit seamlessly into the real environment. This requires the virtual objects to be lit consistently with their neighbouring real objects. We try to enhance the lighting of the virtual objects by using the technique of Image Based Lighting (IBL), where an image or video of the environment is used to compute lighting. IBL is widely used in feature films and impressive filmlets. In these applications lighting is captured offline, requiring a huge amount of time. In contrast, our application captures the lighting in realtime. This permits the real environment to influence the lighting of the virtual objects immidiately. It is thus possible for a real person to see his/her own mirror image in one of the virtual objects.

In this paper we first explain the principles of Image Based Lighting in section 2. Afterwards, we discuss the capturing of realtime environment maps in our setup in section 3. In section 4 the final rendering of the virtual objects is described. Here we also explain how to add shadows to the virtual scene. Finally we discuss the results of our approach and future work in section 5.

Paper

Download the full paper:
ARIBL_Peter_Supan.pdf

Images

Only diffuse lighting on the objects.	Only specular lighting on the objects.	Diffuse and specular lighting with decal textures.
Pipeline of the algorithm.	The red object is visible in the virtual reflections.	Real objects influence the reflections.
Yellow reflection from the apple juice.