Simulation of a Vision Correction Display System

Vidya Sunil
Vehant Technologies
vidyas@vehant.com Renu M Rameshan
Vehant Technologies
renur@vehant.com

Abstract

Eyes serve as our primary sensory organs, responsible for processing up to 80% of our sensory input. However, common visual aberrations like myopia and hyperopia affect a significant portion of the global population. This paper focuses on simulating a Vision Correction Display (VCD) to enhance the visual experience of individuals with various visual impairments. Utilising Blender, we digitally model the functionality of a VCD in correcting refractive errors such as myopia and hyperopia. With these simulations we can see potential improvements in visual acuity and comfort. These simulations provide valuable insights for the design and development of future VCD technologies, ultimately advancing accessibility and usability for individuals with visual challenges.

Figure 1: Output images (1(c), 1(d)) of a simulated vision correction display for a hyperopic eye which sees 1(b) without any correction.

1 Introduction

Almost all optical systems, including the human eye, exhibit optical aberrations. In the past, eyeglasses were the primary means of correcting optical aberrations. However, advancements in refractive surgery and contact lenses have provided additional options for addressing refractive errors. These methods, although effective, can be considered obtrusive since they require the individual to wear eyewear or undergo surgical procedures. This can lead to discomfort and, in rare cases, potential risks.

A recent addition to the options for correcting optical aberrations in the eye is the introduction of computational displays[5]. Unlike traditional methods that physically modify the eye’s refractive power through eyewear or surgery, computational displays[6, 11] present pre-distorted content based on the viewer’s prescription. This approach allows a sharp image to be formed on the retina without the need to alter the eye’s physical optics as the prescription changes. Computational-based vision correction displays [6, 11] do not require direct contact with the observer and can be built using readily available hardware components. Once the display device is constructed, digital correction of refractive errors is possible, eliminating the need for further adjustments to the optical hardware component.

Light field displays[4], mainly used to display glasses-free 3D images, are now being explored for correcting visual aberrations in observers. These displays have the potential to revolutionise visual experiences by offering high-quality and personalised solutions in various modes, including 2D, glasses-free 3D, and vision correction. As technology continues to evolve, we can anticipate further improvements in resolution, contrast, and overall performance of light field displays.

In this paper, we report the simulations of a vision correction display solution that leverages the shearing and refraction of a light field as it propagates from the display to retina through the lens. Our approach involves utilising two methods for creating a light field display: modifying the LCD display with a lenslet array and with a pinhole array. With these techniques, we aim to improve the visual experience for individuals with various refractive errors. This innovative approach holds promise for advancing vision correction technology and enhancing accessibility for those with visual impairments. These ideas are present in [8], and our contribution is in developing a simulation which gives the user the flexibility in designing such a system. In our understanding this is the first such simulator.

1.1 Problem Definition

This paper aims to create a simulation of a vision correction display (VCD) to enhance visual activities for visually impaired individuals. In this work, we only focus on the rectification of hyperopia and myopia.

2 Display Light Field Formation

In this work, we are generating display light field using the Inverse Light Field Projection which is introduced in [8]. Light field[9, 7] is the most general representation of light propagation. The plenoptic function[9], regarded as the most general form of the light field, is a 7D function that describes all characteristics of light rays. However, this 7D plenoptic function can be reduced to a 4D representation using the two-plane model[10]. In this work, we focus on a four-dimensional light field with positions(2D) and directions(2D).

An image formed in the retina can be expressed as the integral of the light field inside the eye [8],

I_{r}(x)=\int_{-\infty}^{\infty}L_{r}(x,u)A(u)\,du\ ,

(1)

where $I_{r}$ is the image formed in the retina, $L_{r}$ is the retinal light ﬁeld, and $A$ is the aperture function, $x$ is a position on retina and $u$ is in aperture.

The retinal light field depends on the light field from the display and can be calculated as explained below[8]. The display light field $L_{d}$ , travels a distance $d_{o}$ to reach the pupil, resulting in a shearing of the light field in the x-direction, where the amount of shear depends on $d_{o}$ . Subsequently, it undergoes refraction by the lens with a focal length $f$ , causing a shearing in the y-direction with a shear dependence of $-1/f$ . The light field then travels an additional distance $d_{e}$ within the eye until it reaches the retina. At the retina, the radiances are integrated to derive the retinal light field. Linear transforms on the light field can be used to depict propagation and refraction. A composite transform M can be employed to establish the relationship between the display light field $L_{d}$ and the retinal light field $L_{r}$ as follows.

$\displaystyle\begin{pmatrix}x\\ u\end{pmatrix}$	$\displaystyle=Q(d_{e})T(d_{e})R(f)T(d_{o})\begin{pmatrix}x^{d}\\ u^{d}\end{pmatrix}$
	$\displaystyle=\begin{pmatrix}1&0\\ 1&-d_{e}\end{pmatrix}\begin{pmatrix}1&d_{e}\\ 0&1\end{pmatrix}\begin{pmatrix}1&0\\ -1/f&1\end{pmatrix}\begin{pmatrix}1&d_{o}\\ 0&1\end{pmatrix}\begin{pmatrix}x^{d}\\ u^{d}\end{pmatrix}$
	$\displaystyle=M\begin{pmatrix}x^{d}\\ u^{d}\end{pmatrix},$	(2)

where $Q(d_{e})$ is the re-parameterization matrix to ensure that angular plane lies on the pupil, $T(d_{e})$ is the shear matrix introduce by the propagation of light with in the eye, $R(f)$ is the refraction matrix, $T(d_{o})$ is the shear due to the propagation of light from display to the eyes and $\begin{pmatrix}x^{d}\\ u^{d}\end{pmatrix}$ are the display light field parameters. $M$ is always invertible. Having established the relation between $L_{r}$ and $L_{d}$ (equation 2), the equation 1 can be rewritten as,

	$\displaystyle I_{r}(x)$	$\displaystyle=\int_{-\infty}^{\infty}L_{r}(x,u)A(u)\,du$
		$\displaystyle=\int_{-\infty}^{\infty}L_{d}(M^{-1}(x,u))A(u)\,du,$		(3)

where the display light field $L_{d}$ is created by inversely transforming the retinal light field $L_{r}$ . This integral is discretized to produce a linear forward model.

I_{r}=PL_{d},

(4)

where $P$ is the prefiltering matrix, $L_{d}$ is display light ﬁeld and $I_{r}$ is the retinal image. The light field $L_{d}$ is essential for generating $I_{r}$ , the retinal image. In section 2.1, we elaborate on the process of generating the light field ( $L_{d}$ ), which is created using the LCD display image $I_{d}$ with either a lenslet array or a pinhole array.

The goal of the vision correction display[8] is to provide the viewer with a 4D light field which produces the required 2D retinal projection. The following objective function can be optimised to determine the emitted light field if viewing distance, pupil size, and other factors are known.

	$\displaystyle\operatorname*{argmin}_{L_{d}}\quad\|\|I_{r}-PL_{d}\|\|^{2}$		(5)
	$\displaystyle\text{subject to}\quad 0\leq L_{d}\leq 1$

The optimized light field $L_{d}$ is refereed as the prefiltered light field which is capable of projecting correct image in the retina.

2.1 Display Image formation

In this section the generation of $L_{d}$ (the light field needed by the eye to perceive correct image) using a LCD with a pinhole array and LCD with a lenslet array are explained.

The pinhole array selectively permits the passage of pixel intensities from the pixels directly beneath each pinhole. Consequently, in order to construct a comprehensive light field using an LCD with a pinhole array, it is imperative to position the pixels of a single image at a distance such that each of them aligns directly under the consecutive pinholes. Failure to do so may result in certain pixel values being obstructed by the pinhole array, leading to the loss of critical information. Resolution of the vision correction display is very less compared to LCD display due to this behaviour of pinhole array. In this case, the comprehensive light field will only be visible when then the optical axis of the eye and the axis of the pinhole array aligns.

The light rays originating from the pixels positioned under a single lens undergo refraction only by that specific lens for a given viewing angle. In the context of a lenslet array, the pixels situated under each lens for a given viewing angle are treated as a unified entity. Pixels under each such unit together forms a single light field segment. Thus, the combination of all such units collectively forms a comprehensive light field. For different viewing angles, the pixels positioned beneath each lens may differ, resulting in the creation of distinct light fields. As the pixels under each lens collectively constitute a single light field segment, this diminishes the resolution of the VCD significantly compared to the LCD used.

For precise visualisation of the comprehensive light field of an image using an LCD with a lenslet array, it’s crucial to position pixels from the same image consistently in same location beneath different lenses. Specifically, when the viewing angle aligns with the optical axis of the lenslet, the pixels belonging to the same image should be positioned at the centre of each respective lens in order to generate a comprehensive light field for that particular image.

Viewing angle significantly influences the accurate generation of the correct light field in both scenarios. Even slight variations in the viewing angle can result in incomplete visualisation of the light field. To address this challenge, shifted images of the same image are pre-filtered and subsequently interlaced.

Figure 3 illustrates the entire system of the vision correction display. The workflow is as follows: Firstly, the image and its shifted images undergo pre-filtering, followed by interlacing to create the display image ( $I_{d}$ ), which is then presented on the LCD (D). A lenslet array or pinhole array is positioned in front of the LCD at a distance equal to its focal length ( $f_{l}$ ) to produce the light field ( $L_{d}$ ). These light field rays propagate towards the eyes, where they are refracted by the eye’s lens, forming an image on the retina. In cases of defocused eyes, a pre-filtered light field will be generated on the display, resulting in the production of a clear image for the eyes.

3 Simulation of primitives

In this study, we replicate the complete vision correction display [8] setup with Blender[1] software. The simulation involves representing the preprocessed image alongside either a pinhole array or a lenslet array, which serves as the vision correction display. Additionally, we employ a defocused camera within the Blender simulation to emulate the characteristics of a defocused human eye.

3.1 Pinhole Array

The pinhole array consists of evenly spaced pinholes arranged in both horizontal and vertical directions. During simulation, it is crucial to ensure that the pinholes are evenly positioned based on the spatio-angular trade-off ratio required. The spatio-angular trade-off ratio is determined by the ratio between the distance separating the centres of the pinholes and the width of the pinhole. In our scenario, we consider a pinhole size of $100\mu m$ and a distance of $500\mu m$ between pinholes, resulting in a spatio-angular trade-off ratio of 5:1. Additionally, the focal length of the array is considered to be $3mm$ . Due to the limited amount of light rays passing through pinholes, it’s essential to maintain high illumination of the display to achieve a bright light field in the VCD system with pinhole. In all simulations utilising a pinhole setup, the display emission strength is maintained at 50.

3.2 Lenslet

A lenslet array consists of a densely packed arrangement of micro lenses. Multiple types of lenslet arrays can be found in the market[2, 3], each showcasing micro lenses with distinct shapes and configurations. In this paper, we are using an array of square shaped micro lens with $3mm$ focal length $500\mu m$ lens diameter and $1mm$ thickness as our lenslet array. We are opting for square lenslet array because of the easiness to interlacing the images and also due to the close packing nature of square lenslets. Standard illumination of the display screen is sufficient for the VCD system with lenslet array.

3.3 Defocused Eye

In our simulation, we model a fixed-focus camera to represent a defocused eye. For hyperopia, we define the fixed focal plane of the camera as the nearest focal plane of the eye. Additionally, we position the display closer to the eye than the fixed focal length to mimic a hyperopic eye. For myopia, we consider the fixed focal plane of the camera as the farthest focal plane of the eye. Furthermore, we place the display image at a greater distance from the eye to simulate a myopic eye.

4 Simulations and Results

We employ a camera equipped with a $50mm$ lens set at $f/8$ for our simulations, representing a 6mm human pupil. And, the display used in all simulations features a pixel density of 254 pixels per inch (PPI). For both hyperopic and myopic systems, we conduct two simulations each, utilising both pinhole arrays and lenslets.

4.1 Hyperopic system

To simulate a hyperopic system, we focus the camera at a distance of $38cm$ and position the display $25cm$ away. The pinhole/lenslet array is placed $3mm$ in front of the display. Detailed results are presented in Figure 4. Figure 4(a) is the original image that is prefiltered for the hyperopic system. Figure 4(b) shows the image formed in retina by a simulated hyperopic system when the original image is displayed in the plain LCD setup, which is blurred as expected. The corrected image produced by the VCD system with pinhole array is shown in Figure 4(c). This is the image formed in the retina when the prefiltered light field is given as input to the VCD system. The output image of the VCD system with pinhole array shows a significant improvement in sharpness compared to the defocused blurred image. However, there is noticeable vignetting effect present at the edges of the output image. The vision corrected image produced by the VCD system with a lenslet array is shown in Figure 4(d). This is the image formed in the retina of the hyperopic eye when the prefiltered light field is given as input to a VCD system with lenslet array. This image exhibits enhanced sharpness and clarity in comparison to the defocused image. Additionally, unlike the output image of the VCD system with pinhole array, the VCD system with lenslet array does not suffer from vignetting effect and appears brighter overall.

4.2 Myopic System

To simulate a myopic system, we focus the camera at a distance of $38cm$ and position the display $25cm$ away. The pinhole/lenslet array is placed $3mm$ in front of the display. Detailed results are presented in Figure 5. Figure 5(a) is the original image used for prefiltering in the myopic system. Figure 5(b) shows the image formed in retina by a simulated myopic system when the original image is displayed in the plain LCD setup, which is blurred as expected. The corrected image produced by the VCD system with pinhole is shown in Figure 5(c). This is the image formed in the retina of a myopic eye when prefiltered light field is given as input to a VCD system with pinhole array. The sharpness of this image is indeed improved in comparison to the defocused image. However, there are some vignetting effects and other artefacts are present in the output of the VCD system with pinhole array. The vision corrected image produced in by VCD system with lenslet array is shown in Figure 5(d). This is the image formed in the retina when prefiltered light field is given as input to a VCD system with lenslet array. The image formed in myopic eye by a VCD with lenslet is sharper than image with plain LCD. But there are some artefacts are present the in the VCD output.

5 Discussion

We have demonstrated a simulation for vision correction display with the purpose of providing a design tool for such displays. The initial results shown for hyperopia and myopia are promising. We are working on improving the myopia results and also on correcting the vignetting and removing other artefacts. We intend to extend these simulations to include other visual abberations too.

References

[1] https://www.blender.org/.
[2] https://www.edmundoptics.in/.
[3] https://www.fresneltech.com/.
[4] Nikhil Balram and Ivana Tošić. Light‐field imaging and display systems. Information Display, 32:6–13, 08 2016.
[5] Attila Barsi, Oliver Staadt, Tariqul Islam, and Stephan Ohl Mary Guindy. State-of-the-art report on display systems. 2021. Ref. Ares(2021)620785 - 26/01/2021.
[6] Huang F.-C., Lanman D., Basrsky B. A., and Raskar R. Correcting for optical aberrations using multilayer displays, 2012. ACM Trans. Graph. (SIGGRAPH Asia) 31, 6, 185:1– 185:12.
[7] Lippmann G. Epreuves reversibles donnant la sensation du relief,, 1908. Journal of Physics 7, 4, 821–825.
[8] F. Huang, G. Wetzstein, B. Barsky, and R. Raskar. Eyeglasses-free Display: Towards Correcting Visual Aberrations with Computational Light Field Displays, 2014.
[9] Ivo Ihrke, John Restrepo, and Lois Mignard-Debise. Principles of light field imaging: Briefly revisiting 25 years of research, 2016. Inf. Disp. 2016,32, 6–13.
[10] Marc Levoy and Pat Hanrahan. Light field rendering, 1996. Proc. ACM SIGGRAPH, 1996, pp. 31–42.
[11] Pamplona V., Oliveira M., Aliaga D., , and Raskar R. Tailored displays to compensate for visual aberrations., 2012. ACM Trans. Graph. (SIGGRAPH) 31.