In the present paper we demonstrate that a focus tunable lens in conjunction with an autofocus algorithm is able to reliably measure distance. The algorithm produces a distance measurement accurate within +/- 0.3 mm in less than one second. It requires no additional hardware apart from the imaging system comprising objective, camera and focus tunable lens. The fast and accurate focus and distance measurement enables and simplifies various applications ranging from robot vision to smart manufacturing control.
Introduction
Many industrial and consumer applications rely on image analysis of objects placed at different distances, e.g. object recognition, QR-code reading and micromachining. In a conventional lens system, optical elements move with respect to each other to adjust the focus. In contrast, an Optotune electronic lens adjusts the focus by directly changing the curvature of a core element containing a liquid trapped between a membrane and glass. This allows for much faster actuation while saving space, as it replaces a series of optical elements.
In the current work we have developed a software which enables automatic focusing of the whole system. The autofocus works by evaluating the contrast values of images taken at sequential focus positions. The relationship between focal power (in diopter) and contrast value follows a Gaussian or Lorentzian distribution. Therefore, we fit a Lorentzian to the data, collected by a sweep through the focal range, to find the sharpest image. The diopter value corresponding to the image in focus is located at the diopter setting at the peak of the fit.
Besides automatic focusing, applications like manufacturing or logistics often require knowledge about the distance from the sensor to the object, too. Therefore, an additional sensor is usually inserted into the setup. This device not only needs more space, but also resources. The new distance sensor solely requires an Optotune focus tunable lens and the autofocus software to find the distance to this object. The distance sensor is included in the hardware and no additional components are needed.
In the following section we explain in detail the hardware and software components and provide an application example.
Software
The autofocus algorithm operates as follows: The camera takes several images at different diopter values. The algorithm then calculates the contrast of a small area close to the image center. It subsequently fits a Lorentzian to the contrast values versus the optical powers as shown in figure 1. The sharpest image corresponds to the peak of the Lorentzian and defines the required diopter setting.
Figure 1 shows the autofocus program flow. The software starts to take an image at the smallest possible diopter value, which is at -2 diopter for the EL-16-40-TC. In the first sweep an image is taken after every 0.4 diopters until the largest possible diopter value is reached, which is at 3 diopter for the EL-16-40 TC. At the same time the contrast value is calculated for each image. Afterwards the diopter setting corresponding to the largest contrast value is localized. To refine the autofocus, firstly a small interval is defined around this diopter setting. Secondly, the lens is set to the smallest possible diopter value within the interval and the second sweep starts, but this time with a much smaller step size of 0.02 diopter. After reaching the largest diopter value within the interval the data of the first and the second sweep are combined and are fit with a Lorentzian distribution
The peak position of the Lorentzian distribution indicates the diopter setting of the highest expected image contrast. The final output is the focal distance of the tunable lens at which the image is sharp.
Figure 1: Program flow of the autofocus algorithm. The software collects the contrast value of the image at different diopter settings. The first sweep through the diopter range has a relatively large step size, the second sweep has a smaller step size and collects the contrast values around the maximum. A Lorentzian distribution is fit to the data points. The peak indicates at which optical power the focus is located.
The distance measurement algorithm (see figure 2) requires calibration. This calibration curve defines the relationship between the distance to the object and the optimal diopter setting for a focused image. As for the autofocus, the object is placed at a certain distance in the center of the image. Then the autofocus algorithm is used to detect the optical power for the sharpest image. The output of the autofocus is inserted into the calibration curve to find the distance to the object.
Figure 2: Program flow of the distance sensor. The software first reads in the calibration file which contains the data to fit a model onto. For the machine vision setup, the optical power scales inverse proportional with the working distance, therefore the model relates to 1/x. Now the autofocus is performed and the focus position obtained is inserted into the distance model. The model provides the working distance for an arbitrary object height. Therefore, in order to perform another distance measurement one can start with a new run of the autofocus directly and skip the first two steps of the program flow.
Every setup requires individual calibration of the distance measurement algorithm (see figure 3). To obtain a calibration curve, several distance/diopter combinations must be obtained by using the autofocus algorithm. After collection of a sufficient number of data points, a model is fit to the data. Without placing other optical components between the sensor and the object, the calibration curve has the shape of 1/x as theoretically expected from the lens equation (see plot in figure 2). An increase of calibration points leads to a more accurate result of the distance measurement with the new sensor.
Figure 3: Program flow of the distance calibration. The software collects the information about the distances from the camera to the object and the respective diopter settings and saves them in a text file. The data serve for the calibration of the distance sensor. An increase of calibration points leads to a more accurate result of the distance measurement, therefore the replacement of the object Δd should be small. The autofocus algorithm is used to find the focal power of the lens leading to a sharp image at a specific distance.
Hardware
Figure 4: The sample machine vision setup with an iDS camera, the fixed focal length lens and the Optotune focus tunable lens in a front lens configuration. The distance sensor is focusing onto the object below.
The distance sensor can be used for many different applications as barcode reading in machine vision, micromachining in laser processing, etc. An example hardware configuration for machine vision is illustrated in figure 4. It consists of a high-resolution sensor (iDS 1.1” 4104x3004), connected to a lens system (Kowa LM50FC, WD=300mm, F# = 1.8) and the Optotune focus tunable lens (EL-16-40-TC). The components are set up in a front lens configuration where the tunable lens is placed in front of the fixed focal length lens.
The working principle of the EL-16-40-TC is based on Optotune’s well-established technology of shape-changing polymer liquid lenses. The optical fluid in the core is sealed off with an elastic polymer membrane as shown in figure 5. An electromagnetic actuator exerts pressure on the container with the fluid and therefore changes the curvature of the lens ranging from concave to convex. By changing the electrical current flowing through the coil of the actuator, the optical power of the lens is controlled.
Figure 5: Scheme of the sealed tunable lens container filled with an optical fluid and embedded in an EL-16-40-TC housing.
Results
Using a liquid lens as an autofocus program has several advantages: no additional hardware is required; the system runs independently and most importantly it is very fast compared to mechanical focusing methods. The autofocus presented here is using the Optotune tunable lens with a large aperture (16 mm) for high resolution images (5 MP). It takes between 0.5 and 1 seconds to obtain the sharpest image. The time varies slightly depending on the distance to the object. Based on the user’s requirements the speed of the autofocus can be adjusted by using different components. E.g. a camera with a larger framerate enhances the speed. The uncertainty of the autofocus is 0.01 diopter for the setup.
Figure 6: a) Calibration curve of the distance sensor provides the corresponding distance for a known optical power.
Figure 6: b) Residuals (data point at xᵢ – model at xᵢ) of the calibration curve. The residuals are more spread for negative diopter values or large distances, proving that the distance sensor is more accurate in the positive diopter range of the lens.
Figure 7: Standard deviation and maximal and minimal deviation from the mean value at four different distances. The standard deviation is smaller in the positive diopter range leading to a higher measurement precision.
The uncertainty of the distance sensor is determined by the calibration curve scaling with 1/x, see figure 6a. This relationship can be derived from the lens equations; thus, the measurement coincides with the theory. Due to this shape the curve is steeper for smaller diopter values. This means that the same diopter interval taken in the negative diopter range corresponds to a larger distance interval than if it was taken in the positive diopter range. Therefore, the uncertainty of the distance value increases with the working distance. This is shown in figure 7 representing the standard deviation for four different distances together with the maximal and minimal change to the mean value. In the positive diopter range the standard deviation is up to one order of magnitude smaller than in the negative diopter range. Another method to visualize this effect is the calculation of the residuals (data point at xᵢ – model at xᵢ), see figure 6b. Since the residuals are more spread for negative diopter values or large distances, the distance sensor is more accurate in the positive diopter range of the lens.
The three parameters: Precision of the measured distance, value of the uncertainty and complete distance range are defined by the individual setup used. For the machine vision setup presented here, the lowest uncertainty of the distance measurement is 0.3 mm for a complete distance range of 620 mm (240 mm - 860 mm). Based on the requirements, the measurement precision and uncertainty can be varied using a different configuration, for instance a lens system with a different focal length or a back-lens configuration instead of a front-lens configuration. It is important to keep in mind that the range and the precision affect each other.
In summary, when requiring high precision and the distance range is smaller than the corresponding full diopter range, working in the positive diopter range is suggested. There the results are most precise and feature the highest repeatability.
Conclusion
With a speed of less than a second the new autofocus program allows quick focusing onto objects at different distances. This enables fast and sharp imaging using integrated components without the need of additional hardware. Optotune tunable lenses focus quickly with a lifetime in excess of 1 billion cycles enabling a reliable autofocus and distance measurement system. Furthermore, the precision with which Optotune focus tunable lenses measure distances is below one millimeter after a one-time calibration. The liquid lens based autofocus and distance measurement system is more compact, repeatable, and accurate than conventional systems on the market. It combines automatic focusing and distance measurement in one device.
For more information, please contact sales@optotune.com.
Authors: Jennifer Studer, Sarah Kurmulis and Mark Ventura