Speaker: Christoph Winklhofer
Mixed reality is the idea to merge real and virtual objects in a scene. The visual appearance of such an augmented environment depends on a plausible lighting simulation. Reflective and refractive objects are ubiquitous in the real world but most mixed reality systems neglect them. The reason is that these objects require a global illumination approach. Indeed, such a complex method is hard to embed in a mixed reality system that demands real-time frame rates to handle user interaction.
This thesis describes the integration of reflective and refractive objects in a mixed reality environment. The aim is to create a realistic light distribution that simulates reflection and refraction between real and virtual objects. Another important aspect for a believable perception are caustics, a light focusing due to the scattering from reflective or refractive objects.
The proposed rendering method extends differential instant radiosity with three other image space rendering techniques capable to handle reflection, refraction and caustics in real-time. They link billboard impostors with relief mapping to produce convincing reflections and refractions. Combined with deferred shading and instant radiosity, it is possible to capture indirect-lighted surfaces. For caustics, a buffer stores the photon concentration in screen space and maps them onto objects, analog to a light map. Finally, differential rendering merges real and virtual objects. Therefore, the occurring light paths are analyzed and the differential effect is applied also to reflected and refracted objects.
By combining these techniques, our method successfully simulates the various lighting effects from reflective and refractive objects and is able to handle user interactions at real-time frame rates. This offers a practicable possibility to greatly improve the appearance of a mixed reality environment