3D Research (2010) 02: 31-35 10.1007/3DRes.02(2010)06

3DR REVIEW

w

Holographic Display of synthetic 3D dynamic scene Melania Paturzo* • Pasquale Memmolo • Andrea Finizio • Risto Näsänen • Thomas J. Naughton • Pietro Ferraro

Received: 08 June 2010 / Accepted: 17 June 2010 © 3D Research Center and Springer 2010

Abstract A synthetic scene of real-word objects is obtained through the multiplexing of several digital holograms. Moreover, by an opportune numerical hologram deformation, it is possible to synthesize 3D dynamic scenes that can be displayed by means of a spatial light modulator. In fact, the spatial adaptive deformation of digital holograms allows the control of the object position and size in a 3D volume with a high depth of focus. Through this novel technique a 3D dynamic scenes can be projected as an alternative to difficult and heavy computations needed to generate realistic-looking computer generated holograms. Finally we report the result of a pilot experiment to evaluate how viewers perceive depth in a conventional single-view display of these dynamic 3D scenes.

1. Introduction* Three-dimensional (3D) imaging, display, and processing have been investigated frequently1-3. In fact, the synthesis of dynamic 3D scenes can be useful in many fields such as for training and simulation of real-world scenarios (surgery training, 3D object design and visualisation) and for entertainment application (for example, video-games, virtual reality, 3D video)4-5. Holography is one popular technique that can record and reconstruct 3D objects. Recently, digital holography has become viable with the development of megapixel CCD sensors that have Melania Paturzo1 • Pasquale Memmolo1,2 • Andrea Finizio1 • Risto Näsänen3 • Thomas J. Naughton3,4 • Pietro Ferraro1 1 CNR Istituto Nazionale di Ottica – Sezione di Napoli Via Campi Flegrei, 34 80078 Pozzuoli (Napoli) Italy 2 DIET, Università di Napoli “Federico II”, Via Claudio 21, 80125 Napoli, Italy 3 University of Oulu, Oulu Southern Institute, Vierimaantie 5, 84100 Ylivieska, Finland 4 Department of Computer Science, National University of Ireland Maynooth, Ireland e-mail: [email protected]

sufficient dynamic range in each pixel. The reconstruction can be either performed numerically for a 2D screen or for display in 3D by a SLM (spatial light modulator) 6-8. Holographic displays have the unique advantage of representing all possible visual depth cues, autostereoscopically, with both vertical and horizontal parallax, giving an appropriate medium for unlimited simultaneous viewers at arbitrary viewing positions, and without the potentially nausea-inducing accommodationvergence rivalry inherent in modern stereoscopic 3D cinema. Most of the impressive achievements in holographic 3D display that have been reported were obtained through the realization of computer generated holograms (CGHs) 9-13. With CGHs it is possible to synthesize holograms not only of single objects but of full scenes with multiple dynamic objects. However it is an extremely difficult task due to the huge computation time and often the results are usually of poor quality in terms of image resolution, even if recently some progresses have been done in optimizing algorithms for generating CGHs14-16. Most importantly, CGHs contain only computer-generated information, and, therefore, it is difficult to obtain a realistic representation of the real world. Here we propose an original method that consists in recording holograms of each real-world object individually under favorable conditions, and then building-up a dynamic synthetic 3D scene with a process that is analogous to stopmotion video17. However, in our case the movie has all the advantageous attributes associated with holographic display, such as representing all possible visual depth cues without incompatibility. The proposed method is suitable for real-time optical reconstruction of 3D scenes. 2. Experimental Procedure To create a dynamic 3D scene using only one hologram, we consider a digital hologram of a single object recorded at distance d. The in focus image of the object is obtained reconstructing numerically the hologram at a distance d

32

3DRes. (2010) 02: 31-35

Fig. 1 (left) Scheme of the movement (back-and-forth along the z-axis) performed by a single object through subsequent deformations of the same hologram; (right) numerical reconstruction of the hologram after its stretching with a deformation parameter of 0.8, 1, 1.2, respectively, the distance of reconstruction is fixed at the recording distance

from its plane using the diffraction Fresnel propagation integral. If an affine geometric transformation, consisting of a simple stretching, is applied to the original recorded hologram, we obtain a transformed hologram. By reconstructing the deformed hologram, the in-focus distance becomes D = d / α 2 where α is the elongation factor 18,19. The stretching method has been developed to extend the depth of focus in digital holography20. Therefore, through subsequent reconstruction of the same hologram, stretched with a variable elongation factor α, it is possible to create a dynamic 3D scene with the object moving along the z axis. Moreover, a movement of the object in x,y directions can also be added by a simple shift in the reconstruction plane. To synthesize a dynamic 3D scene with more than one object, we construct each frame of the movie combining various digital holograms according to the following procedure: 1) Each hologram is reconstructed in a plane in which it is in focus. 2) To filter out the off-axis conjugated order a spatial mask is applied to the complex field in the image plane. 3) The position of each object in the x-y plane is changed by a simple shift in order to avoid superimposition of different objects in the combined image. 4) The complex wavefield is propagated back to the hologram plane. 5) Each hologram is stretched according to the desired change of the in-focus distance, that is the object position along the z-axis. 6) The complex-valued holograms are added together to yield a multiplexed hologram. 7) The multiplexed hologram is reconstructed at a fixed distance to obtain the multi-object scene.

4. Results & Discussion

For the experimental validation of our method, we consider a digital hologram of a single object, a puppet: “Pulcinella,” recorded with an optical configuration that optimizes recording parameters such as object-to-CCD distance and illumination intensity for a high quality hologram reconstruction. The holograms are acquired by means of a Mach-Zehnder interferometer in off-axis configuration with a plane reference wave with a distance between the objects and the CCD of about 56 cm. If we stretch the hologram, the object is obtained in focus at a distance different to the original recording distance and with different lateral magnification. Therefore we can create a 3D scene in which a single object is moved back-and-forth as depicted in Fig. 1(left). With this aim, the same digital hologram is successively reconstructed with different elongation factors α. Fig. 1(right) shows the numerical reconstruction of Pulcinella’s hologram stretched with elongation factors α of 0.8, 1 and 1.2, respectively. The numerical reconstructions are performed at a single reconstruction distance for all deformed holograms. It is clear that hologram stretching is equivalent to a change in the reconstruction distance and in the image lateral magnification. The results of Fig. 1 demonstrate that the apparent position in the holographic reconstruction of a real object can be changed independently from the original recorded distance. Fig. 2(a) shows four numerical holographic reconstructions of another puppet representing an astronaut from four different angles (The corresponding movie shows the full 360° sequence). The recording process is performed with an optimized optical configuration that admits a high quality hologram. In Fig. 2(b) are shown the corresponding optical reonstructions performed by a SLMLCOS. The real images are projected onto a scattering screen at a certain distance from the SLM and, then, acquired by a camera. Fig. 3(a) shows three numerical holographic reconstructions of “Pulcinella” at three different angles. In this case, each recorded hologram is geometrical

3DRes. (2010) 02: 31-35

33

Fig. 2 Holograms acquired during the object rotation: (a-Movie1) Numerical reconstructions of four different digital holograms recorded while the object rotates by 360° around its vertical axis from a fixed position . (b-Movie2) Four frames of the optical reconstruction projected on a screen using a SLMLCOS.

transformed, by numerical stretching, to create a dynamic 3D scene in which Pulcinella travels backwards and forwards in the 3D volume while performing a “pirouette.” The processed holograms are reconstructed optically by an SLM-LCOS. Fig. 3(b) shows three frames of the 3D scene optically projected on a fixed scattering screen.

sees Pulcinella traveling back and forth in the 3D volume, with a very large depth of focus (over 48 cm) enabled by this simple, but effective, adaptive transformation of digital holograms. Dynamic and more complex 3D scenes can be synthetically constructed using diverse holograms. As explained above, the procedure is based on recording several digital holograms of individual objects, each of which rotates about its vertical axis but in a fixed position, and combining them coherently. Fig. 4 shows the optical reconstruction of a synthetic hologram of Pulcinella and the astronaut obtained by the combination of two different holograms. They were recorded separately with the two puppets at the same distance and with the same optical configuration. The two original holograms are stretched separately, according to the desired position along the z-axis of the corresponding object, before combining them. The position of each object in the x-y plane is also changed by a simple numerical shift in order to avoid superimposition of the objects in the combined image. The advantage of this synthetic procedure is that complex 3D scene can be displayed using holograms that have been recorded in ideal conditions and hence with the same high quality.

Fig. 3. Holograms acquired during the object rotation: (a)Numerical reconstructions of three different digital holograms. (b) Three frames of the optically reconstructed 3D scene, created by the projection of the geometrical deformed holograms

In the optical reconstruction by an SLM, an observer

The geometric transformation can be flexibly adapted to manipulate the object’s position and size in 3D and within a very large depth of field dispensing with the recording of holograms at different distances from the camera. Moreover, the puppet can be recorded with different postures to get 3D scenes containing whatever animated action is desired (walking, speaking, moving arms, etc.). A basic archive of “postures” (a sort of data base of digital holograms) of one or more puppets, recorded in optimal

34

3DRes. (2010) 02: 31-35

Fig. 4 3D scene with more than one object reconstructed optically using a SLM-LCOS: (a) Scheme of the movements performed by the two objects (backand-forth along the z-axis with rotation); (b-Movie3) optically reconstructed scene projected on a fixed screen.

conditions, can allow one to construct 3D dynamic scenes with no limitations in arrangement or dynamic action.

We have shown that it is possible to synthesize a complex 3D scene by using digital holograms recorded at a fixed distance and position in respect to the CCD camera. The dynamic effect is created through a flexible adaptive geometrical transformation of the holograms.

5. Visual perception experiment

We are able to synthesize numerically 3D scenes of realworld objects using the recorded holograms and to display them optically. In fact, the synthetic holograms can be given as input to any SLM array for optical reconstruction. The result is a 3D scene truly observable of 3D real-world objects projected in a volume in front of the SLM. Moreover, it has been demonstrated that monocular depth cues in the synthesized videos are sufficient to produce a realistic perception of depth.

Finally, a pilot perception experiment was conducted to estimate the perceived depth in a video sequence displayed on a conventional computer monitor. The motion parallax can produce strong depth perception and different cues can be used to determine depth of an approaching object increases in size while the retinal image of a departing object decreases in size. Also, the relative lateral movement of objects provides effective depth cues. However, the perception of depth in images is often underestimated21. So, we expect that the full extent of depth will not be perceived. The non-stereo movie used in the perception experiment contains a sequence of Pulcinella and the astronaut dancing with axial movement along the viewing direction (see Movie3). The sequence was shown continuously looping back and forth as a single-view video clip displayed on a stereoscopic display. The task of the observer was to estimate the magnitude of Pulcinella’s movement in depth. The study revealed that all viewers could see depth in the video. On average, Pulcinella’s movement in depth was estimated to be between 2 cm and 16 cm (mean = 5.87, standard deviation = 5.51) by different observers when the viewing distance from the display was 100 cm. The depth estimate produced by different observers was, thus, highly variable. The great variability in perceived depth probably reflects individual differences in perceiving motion parallax, and the lack of other depth cues, in particular the lack of stereoscopic presentation.

6. Conclusion

Acknowledgement The research leading to these results has received funding from Science Foundation Ireland under the National Development Plan, Academy of Finland, and the European Community’s Seventh Framework Programme FP7/2007-2013 under 575 Grant agreement No. 216105 (Real 3D).

References 1.

J. W. Goodman (1996) Introduction to Fourier Optics, 2nd ed. McGraw-Hill, New York

2.

H. J. Caulfield (1979) Handbook of Optical Holography _Academic, New York

3.

F. Okano, J. Arai, H. Hoshino, and I. Yuyama (1999) Threedimensional video system based on integral photography, Opt. Eng. 38:1072–1077

4.

S.A. Benton and V.M. Jr. Bove (2008) Holographic Imaging, Wiley Inter-Science

3DRes. (2010) 02: 31-35

35

5.

H.M. Ozaktas and L. Onural (Eds.) (2008) ThreeDimensional Television Capture, Transmission, Display Series: Signals and Communication Technology , Springer

14. Y. Ichihashi, N. Masuda, M. Tsuge, H. Nakayama, A. Shiraki, T. Shimobaba, T. Ito (2009) One-unit system to reconstruct a 3-D movie at a video-rate via electroholography, Opt. Exp. 17:19691-19697

6.

T. Kreis, Handbook of Holographic Interferometry: Optical and Digital Methods (Wiley-VCH, 2005)

7.

M. R. Chatterjee and S. Chen, Digital Holography and Three-Dimensional Display: Principles and Applications (2006) ed. Poon, T. Springer, New York, Chap:13

15. B. Munjuluri, M. L. Huebschman, H. R. Garner (2005) Rapid hologram updates for real-time volumetric information displays, Appl. Opt. 44: 50765085

8.

9.

S. Fukushina, T. Kurokawa, M. Ohno (1991) Realtime hologram construction and reconstruction using a high-resolution spatial light-modulator, Appl. Phys. Lett. 58:787-789 G. Tricoles (1987) Computer generated holograms: an historical review, Appl. Opt. 26:4351–4357

10. P. Hilaire, M. Lucente, S. A. Benton (1992) Synthetic aperture holography: a novel approach to three dimensional displays, J. Opt. Soc. Am. A 9:1969–1978

16. L. Ahrenberg, P. Benzie, M. Magnor, J. Watson (2008) Computer generated holograms from three dimensional meshes using an analytic light transport model, Appl. Opt. 47:1567-1574 17. M. Paturzo, P. Memmolo, A. Finizio, R. Näsänen, T. J. Naughton, and P. Ferraro (2010) Synthesis and display of dynamic holographic 3D scenes with realworld objects, Opt. Exp. 18:8806-8815 18. P. Ferraro, M. Paturzo, P. Memmolo, and A. Finizio (2009) Controlling depth of focus in 3D image reconstructions by flexible and adaptive deformation of digital holograms, Opt. Lett. 34: 2787-2789

11. C. W. Slinger, et al. (2004) Recent developments in computer-generated holography: toward a practical electroholography system for interactive 3D visualization, Proc. SPIE 5290: 27–41

19. M. Paturzo and P. Ferraro (2009) Creating an extended focus image of a tilted object in Fourier digital holography, Opt. Exp. 17:20546-20552

12. M. L. Huebschman, B. Munjuluri, H. R. Garner (2003) Dynamic holographic 3-D image projection, Opt. Exp. 11:437–445

20. C. P. McElhinney, B. M. Hennelly, and T. J. Naughton (2008) Extended focused imaging for digital holograms of macroscopic three-dimensional objects, Appl. Opt. 47(19): D71–D79

13. S.Ş Tay, P.-A. Blanche, R. Voorakaranam, A. V. Tunç, W. Lin, S. Rokutanda, T. Gu, D. Flores, P. Wang, G. Li, et al. (2008) An updatable holographic three-dimensional display, Nature 451: 694-698

21. I. P. Howard and B. J. Rogers (2002) Seeing in depth, vol. 2, I Porteous, Toronto