Even if the image itself projects onto the image plane it might be distorted at the periphery. Assuming that the lens is radially symmetric, the distortion can be described as a stretching or compression of the image that becomes increasingly severe away from the optical axis. Figure 4.20 shows how this effects the image for two opposite cases: barrel distortion and pincushion distortion. For lenses that have a wide field-of-view, the distortion is stronger, especially in the extreme case of a fish-eyed lens. Figure 4.21 shows an image that has strong barrel distortion. Correcting this distortion is crucial for current VR headsets that have a wide field-of-view; otherwise, the virtual world would appear to be warped.
Steven M LaValle 2020-11-11