International Journal of Optomechatronics

ISSN: 1559-9612 (Print) 1559-9620 (Online) Journal homepage: http://www.tandfonline.com/loi/uopt20

3-D Microscope System by Using a Liquid Crystal Lens Marenori Kawamura , Eiji Yumoto & Shunsuke Ishikuro To cite this article: Marenori Kawamura , Eiji Yumoto & Shunsuke Ishikuro (2013) 3-D Microscope System by Using a Liquid Crystal Lens, International Journal of Optomechatronics, 7:3, 149-159, DOI: 10.1080/15599612.2013.807526 To link to this article: http://dx.doi.org/10.1080/15599612.2013.807526

Published online: 09 Aug 2013.

Submit your article to this journal

Article views: 43

View related articles

Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=uopt20 Download by: [37.44.207.181]

Date: 26 January 2017, At: 01:35

International Journal of Optomechatronics, 7: 149–159, 2013 Copyright # Taylor & Francis Group, LLC ISSN: 1559-9612 print=1559-9620 online DOI: 10.1080/15599612.2013.807526

3-D MICROSCOPE SYSTEM BY USING A LIQUID CRYSTAL LENS Marenori Kawamura, Eiji Yumoto, and Shunsuke Ishikuro Graduate School of Engineering and Resource Science, Akita University, Akita, Japan The microscope system in combination with a digital image processing is developed for determining all-focused images and depth mapping properties of microscopic objects by using a liquid crystal (LC) lens with variable focal length. Three-dimensional distributions of the microscopic objects are determined by applying voltages to the electrodes of the type and tuning a focal plane in a depth direction. Keywords: component, liquid crystal lens, three-dimensional imaging system, variable focal length

1. INTRODUCTION When microscopic three-dimensional (3-D) objects are observed by using a conventional microscope, the focused and defocused images can be obtained in same area. It is remarkable that the effect of focused and defocused images depends on the magnification of the objective lens (numerical aperture; NA) since a depth of field (DOF) is inversely proportional to the NA of the lens. There are many algorithms for estimating depth properties of arbitrary objects by moving in the depth direction to obtain continuous object images (Nayar 1992; Nayar et al. 1994). When the positions of the camera or the object is moved in the depth direction to focus on the object with a mechanical focusing system, long acquisition times and the associated focusing errors due to the vibration occur. In our proposed LC lens system without any mechanical movements, both the speed and accuracy of the adjustment to focus on the object can be improved. In our group, many types of optical devices using nematic liquid crystal (LC) materials with a large birefringence and dielectric anisotropy have been developed. S. Sato proposed an optical device such as an LC lens with a circularly holepatterned electrode for tuning a focal length without any mechanical movements (Sato 1979). The LC lens with wide variable lens properties from a concave (negative) lens property to convex (positive) lens property has been reported by Wang et al. (2005). When the circularly hole-patterned electrode of the LC lens is divided into four or eight parts and the different voltages are applied to the divided electrodes, Address correspondence to Marenori Kawamura, Graduate School of Engineering and Resource Science, Akita University, 1-1 Tegatagauen-Machi, Akita City 010-8502, Japan. E-mail: kawamura@ gipc.akita-u.ac.jp 149

150

M. KAWAMURA ET AL.

NOMENCLATURE df I(x, y) Ii(x, y) Si(x, y) W De e== e?

distance between the focused and reference planes integrated image with a deep depth of field in objects light intensity at a pixel position (x, y) for the ith image sampling function at a pixel position (x, y) for the ith image window around the pixel of interest dielectric anisotropy permittivity parallel to an LC director permittivity perpendicular to an LC director

Dn no ne neff h V0 V1, V10

birefringence of an LC material ordinary refractive index of an LC material extraordinary refractive index of the an LC material effective refractive index of an LC cell average of the tilt angle of LC molecules applied voltage to the external electrode applied voltages to the circularly hole-patterned electrodes

the effective refractive index distribution can be changed and then the focal plane can be moved; that is, a beam-steering property in addition to the beam-focusing property can also be obtained (Ye et al. 2005; Kawamura et al. 2009; Kawamura et al. 2011). Both the effective diameter of the lens aperture and the lens power strongly depend on not only the thickness of a middle glass substrate, such as an insulating layer and an LC layer, but also on the diameter of the hole-patterned aperture of the LC lens with a single circularly hole-patterned electrode. The other structure of the LC lens with the different circularly hole-patterned electrodes has been already developed for improving positive and=or negative lens properties by controlling the edge of the effective refractive-index-distribution in the hole pattern region (Kawamura et al. 2010; Kawamura et al. 2011). The lens power of the LC lens with the double circularly hole-patterned electrodes can be improved and the LC lens will be expected to be usable in a microscope system. In this article, we propose a three-dimensional microscope imaging system for tuning a focal plane in a depth direction by using a composite objective lens with an LC lens without any mechanical movements. The focal length of the microscope imaging system is controlled by applying the voltage to electrodes of the LC lens with double circularly hole-patterned electrodes and an external flat transparent electrode. The continuous microscopic images are taken by changing the position of the focal plane. The all-focused images and depth mapping properties of the microscopic objects are obtained by processing with our proposed image digital filter from continuous focal images. 2. MULTIFOCUSED IMAGE SEQUENCE AND THE COMPOSED IMAGE BY THE DIGITAL PROCESSING Figure 1 shows an object of unknown shape placed on the translation stage. The reference plane corresponds to the initial position of the stage. The configuration of the objective lens, an LC lens, and imaging sensor defines a single plane. The distance df between the focused and reference planes is determined by measuring the focal length of the LC lens when the voltages are applied to its

LIQUID CRYSTAL LENS 3-D MICROSCOPE SYSTEM

151

Figure 1. Focused plane on an object of unknown shape.

electrodes. The monoscopic images (I1
Ik) of the object can be continuously scanned step by step with a constant change of the focusing distance between consecutive images. The integrated image I(x, y) with a deep depth of field (DOF) in objects may be expressed as the following:

I ðx; yÞ ¼

k X

Ii ðx; yÞ Si ðx; yÞ;

ð1Þ

i¼1

where i ¼ 1
k and k is the number of the images in the sequence. Ii(x, y) is the light intensity at a pixel position (x, y) for the ith image in the sequence and Si(x, y) is a sampling function at the same pixel position. The position of the exactly focused point can be obtained by determining Si(x, y) from consecutive focal images of the objects. The well-known digital image filter (Ito et al. 1989) in this study is given by

Vi ðx; yÞ ¼

Wy Wx X
2 1 X Ii ðx þ p; y þ qÞ  Ii ðx; yÞ ; W p¼Wx q¼Wy

ð2Þ

P PWy where Ii ðx; yÞ ¼ W1 Wx p¼Wx q¼Wy Ii ðx þ p; y þ qÞ and W ¼ (2Wx þ 1)(2Wy þ 1) is a specific region such as a window around the pixel of interest. The position of the focused point can be obtained by the variance at the specific region.

3. MOLECULAR ORIENTATION OF A LIQUID CRYSTAL LENS AND FOCAL LENGTH Figures 2(a)–2(c) show the schematic diagrams of the LC molecular orientation in an LC cell. The LC material has a positive dielectric anisotropy (De ¼ e==  e? > 0) and a birefringence (Dn ¼ ne  no, no and ne are ordinary and extraordinary refractive

152

M. KAWAMURA ET AL.

Figure 2. The distribution of the LC molecular orientation in the LC cell and effective refractive indices.

indices). When a linearly polarized light is normally incident on the LC cell along the x-axis, the effective refractive index of the LC cell can be described by the following formula as ne no neff ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi n2e sin2 h þ n2o cos2 h

ð3Þ

where  h is the average of the tilt angle of LC molecules along z-axis. When the all LC molecules align along x-axis [Figure 2(a)] and the linearly polarized light is incident on the LC cell, the effective refractive index is a constant value (ne) along the x-axis. If the tilt angle of the LC molecules at the center (x ¼ 0) of the LC cell is larger than that of the molecules at the both edges as shown in Figure 2(b), then the effective refractive index at the center will be smaller than that at the edges. The distribution of the effective refractive index becomes a parabolic function and the parabolic distribution opens up. Thus, the distribution of the relative phase difference has the same profile distribution of the effective refractive index and the collimated light wave diverges though the LC cell; that is, the LC cell becomes the concave (negative) lens. On the other hand, when the LC molecules are reoriented as shown in Figure 2(c), the distribution of the phase difference has also the property of the parabolic function and the transmission light converges. The convex (positive) lens property can be obtained.

4. STRUCTURE OF THE LC LENS Figures 3(a) and 3(b) show the side and top views of the LC lens with double circularly hole-patterned electrodes. The sub-circularly hole-patterned electrode at the

LIQUID CRYSTAL LENS 3-D MICROSCOPE SYSTEM

153

Figure 3. Structure of the LC lens with double circularly hole-patterned electrodes. (a) Side view and (b) top view.

different diameter is just attached into a conventional LC lens with a single holepatterned electrode. The LC lens with double circularly hole-patterned electrodes is used for improving the wide range of the variable focal length. The LC lens consists of a top glass substrate with a transparent electrode (1.1 mm, glass substrate 1), glass substrate with the upper sub-circularly hole-patterned electrode (300 mm, glass substrate 2), thin glass substrate for an insulation layer (100 mm, glass substrate 3), glass substrate with the main circularly hole-patterned electrode (1.1 mm, glass substrate 4), LC layer with a parallel alignment along x-direction (110 mm), and lower glass substrate with the transparent electrode (1.1 mm, glass substrate 5). The hole-patterned electrodes are fabricated by a photolithography technique and the diameters of the upper and lower apertures are set to be diameters of 3.0 mm and 4.0 mm, respectively. The surfaces of the glass substrate 4 at the other side of circularly hole-patterned electrode and transparent flat electrode on the glass substrate 5 were coated with polyimide parallel alignment material (JSR, AL1254) by using a spin coater. The glass substrates were baked in an electric oven at the temperature of 180 C for 1 hour. The surfaces of the polyimide film were rubbed along x-axis by using a rubbing machine. For rubbing the polyimide film, a rotating rubbing wheel with a cloth surface was applied to the sample surface coated with the polyimide film using a contact compression. Two substrates were overlapped at anti-parallel rubbing directions. The cell gap was controlled by using glass ball spacers at the diameter of 110 mm. The LC material of MLC6080 (Merck Co., Atsugi, Japan) was injected into the empty LC cell under the room temperature. Sine wave voltages at a frequency of f ¼ 10 kHz were applied to the external flat electrode, and the lower and upper circularly hole-patterned electrodes (V0, V1 and V10 , respectively) by using a function generator (WE7000, Yokogawa Co., Tachikawa, Japan) and amplifier.

154

M. KAWAMURA ET AL.

5. EXPERIMENTAL SETUP OF THE MICROSCOPE SYSTEM Figure 4 shows a schematic diagram of the microscope system by using an LC lens with double circularly hole-patterned electrodes. The imaging system in the microscope system consists of the LC lens, a photo mask with 2.0 mm aperture attached on the one side surface of the LC lens, polarizer, lens adapter and an objective lens (magnification: 4 and numerical aperture: 0.15). The LC lens was inserted between lens adapter and objective lens. The polarizing direction of the polarizer is parallel to the rubbing direction of the LC lens. A Casio EXILIM Pro EX-F1 digital camera (Tokyo, Japan) with 1=1.8-inch high-speed CMOS (6.6 million pixels) is mounted on the microscope to obtain digital microscope images of the object. The images are continuously taken when each voltage is applied to the electrodes of the LC lens. A 36-segment light emitted diode (LED) ring illuminator such as a circular microscope illumination device is used in the microscope system. The LED ring illuminator is positioned above a sample stage. The green light at the wavelength of 540 nm is radiated directly from the sample from 36 different directions with the incident angle of about 65 degrees. The light intensity of the LEDs can be adjusted by applying the direct current to the LED from a direct current stabilized power source equipment. The high precision Ronchi ruling target is used for evaluating the spatial resolution of our microscope system’s ability to distinguish object details. A modulation transfer function (MTF) value, such as the contrast ratio between the dark and bright areas transmitted through the Ronchi ruling target, can be determined by measuring the light images from the equal bars of the spatial square wave targets, which have a very high contrast ratio and edge definition. This measure of the resolution in terms of line-pairs per millimeter (lp=mm) is known as a spatial frequency. The inverse of the frequency yields the spacing in millimeters between two resolved lines. The bar targets with a series of equally spaced, alternating bright and dark lines are ideal for testing system performance.

Figure 4. Microscope system with an LC lens.

LIQUID CRYSTAL LENS 3-D MICROSCOPE SYSTEM

155

The focal length of the LC lens can be estimated by measuring a circular interference fringes of an aperture of the LC lens under cross polarizers and fitting an optical phase difference along y-axis with a quadratic function derived from the interference fringe. The position of the focal plane can be controlled by applying the voltages V0, V1 and V10 to the external flat and double circularly hole-patterned electrodes of the LC lens. The consecutive microscopic images are taken by adjusting the focal length and then the three-dimensional distributions of the focused images are reconstructed by the image filter processing as already described in equation (2). We developed this image filtering process by using a Laboratory Virtual Instrumentation Engineering Workbench (Labview) from National Instruments Co. (Austin, TX, USA) with a vision=image processing development software.

6. RESULTS AND DISCUSSION Figures 5(a) and 5(b) show the interference fringes of the LC lens under crossed polarizers when V0 ¼ 38 V, V1 ¼ 8 V, V10 ¼ 8 V and V0 ¼ 75 V, V1 ¼ 8 V, V10 ¼ 0 at the frequency of 10 kHz are applied to the external transparent electrode and double circularly hole-patterned electrodes. The almost circular fringe increases at the diameter of 4.0 mm as increasing the applied voltage V0 when V1 and V1’ are fixed since the LC molecules at the center of the aperture begin to reorient along the nonuniform electric field caused by the top transparent electrode and the circularly hole-patterned electrode. The optical phase difference of the neighbor interference fringe is 2p (rad). The phase difference at the center of the aperture is smaller than that at the edge of circularly hole-patterned region. Figures 6(a) and 6(b) show the distributions of the relative phase difference of the LC lens with the double circularly hole-patterned electrode at y-axis derived by the interference fringe as shown in Figure 5, where the applied voltage to the external electrodes is same value as described above. The relative phase difference distribution can be estimated by counting the interference fringes at each position because the phase difference of the neighbor interference fringe is 2p. As increasing the applied voltage to the top flat electrode, the LC molecules are orientated at around the edge of the hole-patterned electrode along the nonuniform electric field. Then the

Figure 5. Interference fringes (a) V0 ¼ 38 V, V1 ¼ 8 V, V1 ¼ 8 V, (b) V0 ¼ 75 V, V1 ¼ 8 V, V1 ¼ 0.

156

M. KAWAMURA ET AL.

Figure 6. Phase difference distribution (a) V0 ¼ 38 V, V1 ¼ 8 V, V1 ¼ 8 V, (b) V0 ¼ 75 V, V1 ¼ 8 V, V1 ¼ 0.

relative phase difference increases toward the edge of the circularly hole-patterned electrode and the phase profile seems to be a quadric function. At the region near the edge of the circularly hole-patterned electrode, the phase difference distribution will be deviated from the quadric function as increasing the voltage applied to the circularly hole-patterned electrode. As these results, an optical aberration such as a spherical aberration occurs. The focal length can be estimated by calculating quadric curve fitting from the measured phase differences as shown in Figure 6. The focal length of the composite lens of the objective lens and LC lens can also be estimated by moving the sample stage to focus the high precision Ronchi ruling target (Figure 7). When the voltages of V0 ¼ 25 V and V0 ¼ 60 V are applied to the flat Electrode A of the LC lens and the voltage of V10 ¼ 0 is applied to the Electrode B at the same time, the focal length of the composite lens varies from 7.5 mm to 8.8 mm and from 7.6 mm to 9.7 mm with increasing the voltage V1 applied to the Electrode C. Figure 8 shows the MTF property of the microcope system with the two types of the LC lenses. The dots are the experimental results and dashed and solid lines are the result of the curve fitting. The LC lens of type 1 is a conventional LC lens with a single circularly hole-patterned electrode; the sub-circularly hole-patterned electrode is removed in the schematic diagram as shown in Figure 3. The LC lens of type 2 is the LC lens with double circularly hole-patterned electrodes in this study. When the no voltage is applied to the electrodes of the type 2 LC lens, the focal length of the composite lens is 7.4 mm and the contrast ratio at the focal plane

Figure 7. Focal length properties while keeping the applied voltages of V10 ¼ 0 in addition to V0 ¼ 25 V or V0 ¼ 60 V.

LIQUID CRYSTAL LENS 3-D MICROSCOPE SYSTEM

157

Figure 8. MTF property of the microscope system.

monotonically decreases as incresing the spatial frequency. The contrast ratios of the two types of the LC lenses also decrease as increasing the spatial frequency. The MTF property of the type 2 LC lens can be improved about two-fold higher than that of type 1 LC lens at the spatial frequency of 40 lp=mm.

Figure 9. Microscope images of an IC chip with varying focusing plane. (a) 8.40 mm, (b) 8.66 mm, (c) 8.92 mm, (d) 9.18 mm, (e) 9.44 mm, and (f) 9.70 mm.

158

M. KAWAMURA ET AL.

Figure 10. All-focused images. (a) Stage height adjusment, (b) driving the LC lens.

Figures 9(a)–9(f) show the six microscope images of 16 images of an integrated circuit (IC) chip when the position of the focal plane along the depth direction is changed every about 87 mm. The width and depth of the lead frame of the IC chip are about 2.0 mm and 1.2 mm, respectively. Since the sample IC chip at each focal plane is irradiated uniformly by using the 36-segment LED ring illuminator, there is no shadow at the side of the sample as shown in Figure 9. Both the magnification and reduction of these images cannot be obtained because the transmission light through the objective lens and LC lens with a variable focal length is collimated. It is seen that the reflective brightness in the microscope image drastically changes by the angle of the lead frame of the IC chip and the defocus images with the high contrast are also obtained. Figures 10(a) and 10(b) show all-focused images by using a series of the microscope images when adjusting the height of the sample stage without the LC lens and controlling the focal plane in the depth direction with the voltages applied to the LC lens, respectively. The total of 16 images were taken for this sample and all-focused images were calculated by the images filtering process as described in Equation (2). The calculated result of the all-focused image as shown in Figure 10(a) cannot be obtained sharply due to the large change in magnification of the microscope images. On the other hand, the all-focused image can be obtained clearly when the distance between the composite lens and the sample stage is fixed and the focal length is adjusted with applying the voltage to the LC lens. The shape and depth information of the sample can be derived from our technique by using the LED ring illuminator since the light can be irradiated uniformity. Figure 11 shows the depth profile

Figure 11. Depth mapping profile (color figure available online).

LIQUID CRYSTAL LENS 3-D MICROSCOPE SYSTEM

159

also calculated by using the images filtering process as described in Equation (2). The color bar shows the depth scale. The small step of
0.05 mm in the depth direction was successfully measured. The acquisition times of this LC lens system without any mechanical movements must be faster than those of the conventional all-focused imaging system with mechanical movements such as a piezoelectric motor or stepping motor. This investigation of the interesting topic about the speed and accuracy of two systems is now in progress. 7. CONCLUSION The composite lens of the LC lens with double circularly hole-patterned electrodes and objective lens has been developed for use in a monoscopic optical microscope system with no mechanical movements. The focal length between the objective lens and the monoscopic object can be controlled by applying the voltage to the LC lens. The captured image of each focal length in the depth direction can be taken by using CMOS camera and the images have almost the same properties of the magnification and resolution. Then the all-focused microscope images and depth mapping profile of the object can be estimated by the digital image process from every image of the sequence. Our optical system is used to obtain the depth mapping property and focused images of a variety of industrial products as well as medical samples. ACKNOWLEDGMENT This research was partially supported by the Ministry of Education, Science, Sports and Culture Grant-in-Aid for Scientific Research Young Scientists (B), 90312694, 2012.

REFERENCES Ito, K., A. Hayashi, and Y. Ichioka. 1989. Digitized optical microscopy with extened depth of field. Appl. Opt. 28: 3487–3493. Kawamura, M. and H. Goto. 2011. Elliptical light distribution control by using liquid crystal device. Int. J. Optomechatronics 5: 15–29. Kawamura, M., H. Goto, and E. Yumoto. 2010. Improvement of negative lens property of liquid crystal device. Jpn. J. Appl. Phys. 49: 118002–1–118002–2. Kawamura, M. and Y. Ito. 2011. Liquid crystal device with a variable focal length by using two different ring electrodes. J. Inst. Elect. Engnr. Jpn. 131: 53–58. Kawamura, M., M. Ye, and S. Sato. 2009. Transient properties of a liquid crystal optical device with an anamorphic lens property. Mol. Cryst. Liq. Cryst. 510: 182–190. Nayar, S. K. 1992. Shape from focus system. Proc. IEEE Conf. Comp. Vision Pattern Recognition 302–308. Nayar, S. K. and Y. Nakagawa. 1994. Shape from focus. IEEE Trans. Pattern Anal. Mech. Intell. 16: 824–831. Sato, S. 1979. Liquid-crystal lens-cells with variable focal length. Jpn, J. Appl. Phys. 18: 1679–1684. Wang, B., Y. Mao, and S. Sato. 2005. Liquid crystal negative lens. Japanese J. Appl. Phys. 44: 4979–4983. Ye, M., Y. Yokoyama, and S. Sato. 2005. Liquid crystal anamorphic lens. Jpn. J. Appl. Phys. 44: 235–236.