VIRTUAL REALITY

Why am I writing this virtual reality book?

First of all, I love writing books and papers, and giving them away for free on the Internet. I want to reach people all over the world, regardless of their background or economic status. Spreading knowledge and understanding helps us all to advance, while also leveling the playing field for those not fortunate enough to have access to the right people and resources. My 2006 Planning Algorithms book has succeeded in this, with over a million global downloads, use in numerous courses, and references appearing everywhere. I continue to get so many thanks, and invitations to visit people all over the world, that it motivates me to keep going.

Second, I want to write a book on virtual reality (VR) because of my unique background and perspective. I was in the founding family of Oculus VR, starting work for them in September 2012, a few days after the successful Kickstarter campaign. They needed to get head tracking working in a hurry, and Jack McCauley reached out to me because he found material in my Planning Algorithms book that seemed related (e.g., quaternions). At the time, I was on a sabbatical on Oulu, Finland. I flew to California and met Jack, along with Palmer Luckey, Brendan Iribe, and Michael Antonov (all four from the Kickstarter video). They did not even have an office yet. I liked the energetic gang, and I was impressed with the potential of the prototype headset, even though it was not working too well (I even broke its tracking by intentionally hitting a kinematic singularity).

I was so hooked that I was debugging tracking software on the airplane ride back to Finland, while twisting a sensor around that was connected via USB to a laptop that we just bought from Best Buy. In the months that followed, my wife Anna Yershova and I were working for Oculus from our apartment in Finland (see picture). We did things like gyro integration, tilt correction, sensor performance characterization, magnetometer calibration, offset head model, predictive tracking (to overcome latency), and early positional tracking experiments. It was great that Oculus bought us a UR5 robot arm to perform systematic studies (I don't advise anyone to live in a small apartment with a live robot arm).

The company started growing so fast that I wanted to play a larger role. In March 2013, I bailed on the Finnish sabbatical, and moved to Irvine, CA to work full time for Oculus as the chief scientist, while continuing to focus on tracking. I could not stop my desire to teach and write. I started a seminar series, where I gave frequent lectures (see next picture) or invited others to. I also ran a scientific blog (the articles are still here, here, and here) on our progress.

It did not take long before I realized that we are doomed if we do not understand human perception. While working on predictive tracking, Peter Giokaris pointed out that one of my "better" methods was worse in terms of perception. This motivated me to develop perceptually tuned tracking methods, and also to hire perceptual psychologists. By having a small perceptual psychology team, we were able to resolve hypotheses through carefully designed experiments with human subjects, determine problems with optics, write a scientifically backed Best Practices Guide, and start to get a handle on health and safety issues (especially VR sickness).

I had planned to write a series of blogs on perceptual issues, such as vection, depth perception, simulator sickness, and the power of positional tracking, but I got increasingly busy with making the first vision-based positional tracking system work for the Rift DK2, and maintaining growing piles of older SDK code. A few months later, Facebook acquired Oculus, and I had a hard decision to make: Work in a large company, or return to university life.

I was overwhelmed with excitement for the research and educational challenges that lie ahead in VR. I knew that a deep understanding of perceptual psychology and human physiology is crucial to the engineering of VR hardware and software systems and for the development of successful, effective VR content. It is often hard for engineers and developers to realize this because our perceptual systems are mostly invisible to us. They function effortlessly while we have little or no awareness of them.

I therefore came back to university life with a new mission: Contribute to rapidly growing VR ecosystem by teaching people my freshly learned perspective. Therefore, I started a new VR course at the University of Illinois, which my wife and I continue to teach and expand. In July 2015, I taught a two-week version of the course at IIT Madras, and the appear here for free on the Internet through the NPTEL on-line course system of India. Oculus/Facebook has kindly made donations to both of these. The course at the University of Illinois has so far been a successful catalyst for starting interdisciplinary projects all over our campus, in areas such as kinesiology, psychology, avionics, nuclear engineering, foreign language, and anthropology. These activities continue to shape and improve my understanding of VR. If you are an educator, perhaps you can run a similar course at your institution. In that case, I hope you find the free Virtual Reality book, lectures, slides, and other related materials useful.

Best wishes,
Steve LaValle,
Urbana, Illinois, USA, 2015