VIRTUAL REALITY

By Steven M. LaValle. Cambridge University Press, 2023.

This free VR book covers the fundamentals of virtual reality systems, including geometric modeling, transformations, graphical rendering, optics, the human vision, auditory, and vestibular systems, tracking systems, interface design, human factors, developer recommendations, and technological issues.

Anyone is welcome to download, print, use, copy, and/or distribute anything on this page, either electronically or on paper. However, I do not take responsibility for your distribution of any copyrighted images contained in this book draft under fair use exceptions. You do not need to ask my permission for using these materials, although I would appreciate hearing from you if you find this material useful. If you redistribute any of this material, please include a link back to this web page.

Free, related lectures are available on YouTube. These are part of an accompanying MOOC (free on-line course), produced by NPTEL and IIT Madras, 2016. Free-use pictures of the pioneering headset demonstrated at Harvard in 1968 by Sproull, Foster, and Sutherland appear here.

Why did I write this? Click here for some answers.

Download the whole book

[pdf file] -- Two pages per one. Recommended for printing on US Letter paper.
[pdf file] -- Two pages per one. Recommended for printing on A4 paper
[pdf file] -- One page per one (larger print). May be easier for on-line viewing.

Book cover art by Anna Yershova LaValle.

The whole book in HTML

Download chapters

CHAPTER TOPICS

Chapter 1: Introduction
[pdf] Definition of VR, modern experiences, historical perspective.

Chapter 2: Bird's Eye View
[pdf] Hardware, sensors, displays, software, virtual world generator, game engines, human senses, perceptual psychology, psychophysics.

Chapter 3: The Geometry of Virtual Worlds
[pdf] Geometric modeling, transforming rigid bodies, yaw, pitch, roll, axis-angle representation, quaternions, 3D rotation inverses and conversions, homogeneous transforms, transforms to displays, look-at and eye transforms, canonical view and perspective transforms, viewport transforms.

Chapter 4: Light and Optics
[pdf] Light propagation, lenses and images, diopters, spherical aberrations, optical distortion; more lens aberrations; spectral properties; the eye as an optical system; cameras; visual displays.

Chapter 5: The Physiology of Human Vision
[pdf] Parts of the human eye, photoreceptors and densities, scotopic and photopic vision, display resolution requiments, eye movements, neural vision structures, sufficient display resolution, other implications of physiology on VR.

Chapter 6: Visual Perception
[pdf] Depth perception, motion perception, vection, stroboscopic apparent motion, color perception, combining information from multiple cues and senses, implications of perception on VR.

Chapter 7: Visual Rendering
[pdf] Graphical rendering, ray tracing, shading, BRDFs, rasterization, barycentric coordinates, VR rendering problems, anti-aliasing, distortion shading, image warping (time warp), panoramic rendering.

Chapter 8: Motion in Real and Virtual Worlds
[pdf] Velocities, acceleration, vestibular system, virtual world physics, simulation, collision detection, avatar motion, vection.

Chapter 9: Tracking
[pdf] Tracking systems, estimating rotation, IMU integration, drift errors, tilt and yaw correction, estimating position, camera-feature detection model, perspective n-point problem, sensor fusion, lighthouse approach, attached bodies, eye tracking, inverse kinematics, map building, SLAM.

Chapter 10: Interaction
[pdf] Remapping, locomotion, manipulation, social interaction, specialized interaction mechanisms.

Chapter 11: Audio
[pdf] Sound propagation, ear physiology, auditory perception, auditory localization; Fourier analysis; acoustic modeling, HRTFs, rendering, auralization.

Chapter 12: Evaluating VR Systems and Experiences
[pdf] Perceptual training, recommendations for developers, best practices, VR sickness, experimental methods that involve human subjects.

Chapter 13: Frontiers
[pdf] Touch, haptics, taste, smell, robotic interfaces, telepresence, brain-machine interfaces.

Related resources

Hand-written VR lecture notes from UIUC course in Spring 2015, on which the book was based.
Steve LaValle's recorded VR lectures from NPTEL at IIT Madras, July 2015.

CHAPTER	TOPICS
Chapter 1: Introduction [pdf]	Definition of VR, modern experiences, historical perspective.
Chapter 2: Bird's Eye View [pdf]	Hardware, sensors, displays, software, virtual world generator, game engines, human senses, perceptual psychology, psychophysics.
Chapter 3: The Geometry of Virtual Worlds [pdf]	Geometric modeling, transforming rigid bodies, yaw, pitch, roll, axis-angle representation, quaternions, 3D rotation inverses and conversions, homogeneous transforms, transforms to displays, look-at and eye transforms, canonical view and perspective transforms, viewport transforms.
Chapter 4: Light and Optics [pdf]	Light propagation, lenses and images, diopters, spherical aberrations, optical distortion; more lens aberrations; spectral properties; the eye as an optical system; cameras; visual displays.
Chapter 5: The Physiology of Human Vision [pdf]	Parts of the human eye, photoreceptors and densities, scotopic and photopic vision, display resolution requiments, eye movements, neural vision structures, sufficient display resolution, other implications of physiology on VR.
Chapter 6: Visual Perception [pdf]	Depth perception, motion perception, vection, stroboscopic apparent motion, color perception, combining information from multiple cues and senses, implications of perception on VR.
Chapter 7: Visual Rendering [pdf]	Graphical rendering, ray tracing, shading, BRDFs, rasterization, barycentric coordinates, VR rendering problems, anti-aliasing, distortion shading, image warping (time warp), panoramic rendering.
Chapter 8: Motion in Real and Virtual Worlds [pdf]	Velocities, acceleration, vestibular system, virtual world physics, simulation, collision detection, avatar motion, vection.
Chapter 9: Tracking [pdf]	Tracking systems, estimating rotation, IMU integration, drift errors, tilt and yaw correction, estimating position, camera-feature detection model, perspective n-point problem, sensor fusion, lighthouse approach, attached bodies, eye tracking, inverse kinematics, map building, SLAM.
Chapter 10: Interaction [pdf]	Remapping, locomotion, manipulation, social interaction, specialized interaction mechanisms.
Chapter 11: Audio [pdf]	Sound propagation, ear physiology, auditory perception, auditory localization; Fourier analysis; acoustic modeling, HRTFs, rendering, auralization.
Chapter 12: Evaluating VR Systems and Experiences [pdf]	Perceptual training, recommendations for developers, best practices, VR sickness, experimental methods that involve human subjects.
Chapter 13: Frontiers [pdf]	Touch, haptics, taste, smell, robotic interfaces, telepresence, brain-machine interfaces.