Virtual Window Theory and Practice

You may or may not have seen or heard about a 'virtual window' before. The idea is to make your computer monitor act like a window. This means that as you move around your monitor, the view through it changes just like as if you were looking through a window. This obviously means that some sort of tracking of the viewers head or eyes is performed in real time with the rendering of the scene. This is usually done with web cam(s) or Wii controllers and infrared LEDs. There have been many demonstrations of virtual windows on YouTube, but many of them are flawed for various reasons. One of the best videos on the net that explains and demonstrates this is the famous Johnny Lee video below:

Before I continue, I would like to point out that Johnny Lee's program (the one with the targets) is a slight exception to the rules or laws of a virtual window that I will explain in this post. This is because he has deliberately placed the front most target in a position that is supposed to be in front of the display, hence that object should not really be viewable, but this technique would need to be used if you ever want a 3D effect where objects jump out the screen at you.

It is important to keep in mind when programming a virtual window that as the viewer moves around, the perspective of the 'camera' in the virtual world should change and not the position of the camera. An obvious way to tell if this is being performed is if an object is ever able to go from behind the window in view to in front of the window out of view, as shown in this video and in the Johnny Lee one above.

Another key point to keep in mind is that when looking through a real window whilst moving towards it, objects on the other side will appear to get larger to the viewer, but are actually appearing to take up a smaller percentage of the window. This is because the window also appears to get larger to the viewer, at a faster rate than the objects being looked at. This is best shown in the diagram below.

The diagrams above show how as the viewer gets closer to the display, the object being rendered should actually get smaller rather than bigger. The object only looks larger than before to the viewer because they are now closer to the display.

What is happening is that the viewers field of view through the window is getting larger as they get closer to the window, and thus can see more of the surrounding scenery. The display is remaining still, and hence the virtual camera should remain still as well. Many demonstrations of the virtual window show the objects taking up more of the screen as the viewer gets closer. This shows that the camera is actually moving closer to the objects, instead of changing its perspective.

Again, it is important to remember that Johnny Lee's video (first video above), is an exception to this rule because the front most objects are placed in front of instead of behind the window. This means that they should indeed take up more of the window space as the viewer moves closer to the display.