Better Pixels in Professional Projectors

White Paper

Better Pixels in Professional Projectors

January, 2016 By

Chris Chinnock Insight Media 3 Morgan Ave., Norwalk, CT 06851 USA 203-831-8464 www.insightmedia.info In collaboration with

Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

1

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

Table of Contents Introduction ................................................................................................................4 What are Better Pixels? ..............................................................................................4 Consumer TV vs. Professional Projection .................................................................................. 4 Higher Brightness ....................................................................................................................... 5 Benefits & Trade-offs ..........................................................................................................................................5

Uniformity .................................................................................................................................. 6 Benefits & Trade-offs ..........................................................................................................................................6

Enhanced Resolution .................................................................................................................. 6 Benefits & Trade-offs ..........................................................................................................................................9

High Dynamic Range and Contrast ............................................................................................ 9 Benefits & Trade-offs ........................................................................................................................................ 11

Wide Color Gamut (WCG) ....................................................................................................... 12 Benefits & Trade-offs ........................................................................................................................................ 13

High Frame Rate (HFR) ........................................................................................................... 14 Benefits & Trade-off .......................................................................................................................................... 15

Bit Depth ................................................................................................................................... 15 Benefits & Trade-offs ........................................................................................................................................ 16

3D.............................................................................................................................................. 17 Benefits & Trade-offs ........................................................................................................................................ 18

Summary ................................................................................................................................... 18

Implications for the Capture, Processing and Distribution of Better Pixel Content18 Better Pixel Capture .................................................................................................................. 18 Better Pixel Processing ............................................................................................................. 19 Better Pixel Distribution ........................................................................................................... 20

Implementing a Better Pixel Projector.....................................................................20 Applications for Better Pixels ..................................................................................21 Cinema ...................................................................................................................................... 22 Design ....................................................................................................................................... 26 Training & Simulation .............................................................................................................. 26 Rental & Staging ....................................................................................................................... 26 Corporate .................................................................................................................................. 27

Conclusion ...............................................................................................................27 Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

2

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

Table of Figures Figure 1: Various TV/Cinema Resolution Standards ..................................................................... 7 Figure 2: Snellen, Simple and Hyper Acuity (Source: Arris) ......................................................... 8 Figure 3: Range of natural and human luminance values (Source: Dolby) .................................. 10 Figure 4: Reduction in dynamic range through the capture-to-display pipeline (Source: Dolby) 10 Figure 5: HDR vs. SDR Images (Source: 20th Century Fox)....................................................... 11 Figure 6: Various Color Standards on the 1931 CIE Chromaticity Diagram and u’v’ color spaces ....................................................................................................................................................... 12 Figure 7: 24 vs. 48 frames per second .......................................................................................... 14 Figure 8: Various EOTFs being Considered for HDR Content and Display................................ 16 Figure 9: System Contrast Formula (Source: RealD, Technology Summit on Cinema) .............. 23 Figure 10: System Contrast Ratio (Source: RealD, Display Summit 2015) ................................. 24 Figure 11: Theater Contrast for Various APLs and Scattering/Reflectivity Parameters (Source: Barco) ............................................................................................................................................ 25

Table of Tables Table 1: Acuity vs. Resolution and Viewing Angle (Source: Arris) .............................................. 8 Table 2: The Need for Better Pixels in Professional Projection Applications .............................. 21

Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

3

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

Int r o du c t i o n This white paper will provide an overview of a range of video enhancements that are emerging now in the consumer TV and professional AV industries. These are collectively called “better pixels” as they will improve the viewing experience – in some cases, quite dramatically. The objective of the paper is to define what these “better pixel” features are, how they will benefit professional users and the implications for the entire content-to-display pipeline. The key professional application for video content today is cinema, so we will mostly focus on this. However, the technology is starting to spread to other advanced professional markets like simulation, medical, military, etc. – which is why this paper is important While the focus of the paper is on professional projection, no discussion of better pixels can take place without an understanding of the consumer television market and how better pixels are expected to be adopted there. In reality, the consumer TV and professional AV markets are pushing each other in the development, standardization and deployment of better pixel hardware and software.

W h at ar e Bette r P i xe l s? Consumer TV vs. Professional Projection The term “Better Pixels” has arisen to describe a basket of advanced imaging and display technologies. In consumer TV’s, these have centered on the development of what is being termed Ultra high Definition (UHD) Phase 2. UHD Phase 1 has focused on increasing the resolution of consumer TVs from 1920 x1080 to 3840 x 2160 – sometimes referred to as Ultra High Definition, UHD or 4K. Content is mastered using the 709 color gamut (like HDTV content), usually with 8-bits per color and 30 frames per second. It is basically HD content with standard dynamic range and color volume but with more pixels. UHD Phase 2 maintains the same UHD resolution, but seeks to add additional better pixel parameters that include: • • •

High Dynamic Range (HDR) Wide Color Gamut (WCG) High Frame Rate (HFR)

UHD Phase 2 is being driven by Hollywood for the creation of new content. The first UHD TVs with some of these advanced features are in the market today, with many more coming. The first cinemas to support these attributes are also established already, including the Dolby Vision cinemas and new IMAX cinemas. The development of these solutions will increase image quality and in turn, will drive adoption of higher image quality video in other professional projection markets like simulation, rental and staging, corporate, theme parks, museums, medical, government and intelligence, military and more. Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

4

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

However, the definition of “better pixels” in professional projection is a goes wider than the UHD Phase 2 definition. Key elements here include: • • • • • • • •

Higher Brightness Enhanced Resolution High Dynamic Range (HDR) and Contrast Wide Color Gamut (WCG) High Frame Rate (HFR) Uniformity Bit Depth 3D

Let’s now take a look at each of these better pixel elements in greater detail to understand what each means and what the benefits are, but also, what are the consequences of implementing the feature.

Higher Brightness Increased brightness is a key feature of a better, more realistic, viewing experience. This is trivial for small, direct view displays; but becomes a challenge when going to large, immersive screen sizes. Higher brightness also allows, as a component of High Dynamic Range for parts of the image to literally be brighter than current images, which has an impact on viewers. For example, in the consumer TV segment, most standard dynamic range (SDR) TVs offer 300-400 nits of peak luminance. The new HDR TVs offer 600-900 nits of luminance. This level can even go higher for small areas of the TV to deliver bright specular highlights, for example. The same thing is happening in cinema projectors, especially for 3D viewing. For example, conventional cinema projectors with Xenon lamps project 3D images in the 3-4 fL (10-14 nits) range compared to 2D movies which are about four times as bright. The light loss is a result of the optics and filters needed to create the left and right eye images for 3D. Many have complained about the noticeably low light levels with 3D. The solution is here today in the form of 6p (six primary) RGB laser projectors. These systems feature laser light sources operating at six primary wavelengths – 2 in the blue, red and green. Each pair of RGB wavelengths can encode the left or right eye image, which are separated at the viewer by glasses with corresponding spectral filters (note that other 3D technologies could also be used). Because the lasers can be ganged together to increase light levels, 3D projection can now occur at 14 fL (48 nits) – the same as with 2D projection.

Benefits & Trade-offs Increasing the brightness for 3D projection is a huge benefit for the consumer and the impact is significant. Why? - Because human perception of brightness is non linear. In the above example, the brightness of the image increased 4X but our perception of this increase is higher as we are in a dark environment in the theater. This effect works the other way in bright ambient environments. Increasing the output of a projector from 4000 to 8000 lumens, for example, is noticeable, but the image is not perceived as twice as bright. Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

5

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

These perceived changes are not only impacted by the absolute brightness levels and ambient illumination, but also by the screen technology and other complex human visual adaptation parameters like the changing of the iris diameter. Also note that the human visual system perceives luminance on the screen in nits (or cd/m²). In the case of a projected image, this needs to be translated back to lumens out of the lens. To go from 3fL to 10fL 3D in cinema does not seem like a big task – but it is as the light source for the projector must be more than 3X as bright. Without a technical evolution like laser illumination, that means effectively installing 3 projectors.

Uniformity Peak brightness and uniformity are tied somewhat. Brightness levels need to be appropriate for the environment. That means very bright elements in a pitch dark room can be painful, while too dim an image causes eyestrain and poor image quality. For a projection system, the projector lumen output and the screen size, gain and material all play significant role in determining that right brightness level. Uniformity is an important parameter for any display, but is often not measured in detail in flat panels, while it is in projectors. The uniformity drop off from center to edges can be noticeable in some lamp-based projection system, rolling off to 50-75% of the peak brightness at the screen center. The advent of RGB laser systems for projectors has really changed the uniformity capability with uniformity of 95% not uncommon now. The uniformity is impacted by the screen characteristics such as the screen material and gain. A matte white screen acts like a Lambertian reflector scattering light uniformly in a hemisphere. But many screens have gain which increases the light reflect near the surface normal and reduces the light in the mid to extreme angles. That is fine for a long narrow theater, but can be problematic for other venues. Gain screens create hot spots that are brighter than the rest of the image – thus appearing as a non-uniform image. Sometimes screens are curved to reduce this effect; but curving has a negative effect on contrast. Screens need to be considered as part of the uniformity consideration, especially for multi-blended projector applications.

Benefits & Trade-offs The lack of uniformity is not always that noticeable until you see a uniform gray or color on the screen. Or, it can be very noticeable for higher gain screens or when viewing from an oblique angle. The hot spots can be very distracting. Moving to laser-based projectors with higher lumen outputs that allow for lower gain screens offers a clear benefit for end users.

Enhanced Resolution Figure 1 shows some important resolutions currently being used (both mainstream and stateof-the-art) in different video applications: • • •

720 x 480 - Standard Definition (SD) or DVD 1920 x 1080 - Full High Definition (FHD) 3840 x 2160 - Quad High Definition (QHD) or Ultra HD (UHD) or 2160p

Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

6

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

•

4096 x 2160 - DCI-4K or 4K

Figure 1: Various TV/Cinema Resolution Standards The first three resolutions are now common in consumer TVs while the DCI-4K resolution is reserved for cinema and cinema-related applications. As noted earlier, UHD Phase 1 is really about ONLY increasing the resolution to UHD, and to date, this represents the vast majority of UHD or 4K content, as well as the available flat panels and projectors in consumer and professional applications. In professional projection, the definition of “enhanced resolution” is less defined. UHD and 4K projectors clearly qualify, but there are also many projectors with resolutions between FHD and UHD or 4K. In addition, some projectors can do “image shifting” to create higher resolution images from lower resolution microdisplay. Finally, many applications feature blending of multiple projectors to create large screens that can have well in excess of 4K pixels. Professional projection systems require a screen (front or rear) and the size of the screen and the viewer distance has a big impact on the “perceived resolution.” For example, the on-screen pixels from a 4K projector are twice the size when moving from a 100” to a 200” screen. And, the perception of resolution is also dependent upon how far away the viewer is from the screen. Can a viewer see the difference between a 100” image viewed at 10 feet if the projector has 4K or FHD resolution? Conventional wisdom suggests that the limit of visual acuity for a person with 20/20 vision is one arc-minute of resolution. This math says that the 20/20 viewer probably can’t tell the difference in resolution in the above example, all other factors being equal. But this idea of Snellen or 20/20 reading acuity is based on the physical layout of the rods and cones in Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

7

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

the eye. When looking at an eye chart, it is measuring the eye’s ability to recognize symbols and their orientation with a human perception level of about 30 cycles (pixels) per visual degree. Beyond this are two other levels of acuity. Simple acuity refers to the retina’s maximum Nyquist limit, which is about 60 cycles/degree. Hyperacuity is the ability to notice even finer details such as the misalignment of line segments (vernier acuity) beyond 60 cycles/degree (Figure 2).

Figure 2: Snellen, Simple and Hyper Acuity (Source: Arris) What we “see” from a display depends upon the native resolution of the display and how far away we are (which determines the viewing and cycles/degree). Table 1 shows various combinations of display resolution and viewing angle along with the corresponding acuity type.

Table 1: Acuity vs. Resolution and Viewing Angle (Source: Arris) What this table means is that increasing the resolution of the display allows you to see fine details even from a long distance (smaller viewing angles). As a result, the benefits of more pixels depend upon the content. Content that has low spatial frequencies (i.e. not a lot of details and edges), or –fast- moving components will not look substantially better with increasing resolution. Typically static content with lots of details and edges (e.g. a spreadsheet) will look noticeably sharper and crisper as you increase the pixel count. Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

8

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

A crucial part of this story is Barten’s Contrast Sensitivity Function: while the above table might hold only for pixel on/pixel off (white vs. black pixels/lines), the sensitivity to detail also depends on the contrast of the detail – so it may be that for detail with lower contrast, the pixel/degree limit may be lower. Perceived resolution will also be impacted by the room ambient levels as this can decrease contrast and impact our acuity. It also depends upon the screen technology. Not all screens support higher resolution images. For example, some light rejecting screens use prismatic elements that limit their use to HD, not 4K resolutions.

Benefits & Trade-offs Having more pixels in the projector allows the possibility to deliver an increase in perceived resolution to the end user. Sometimes these benefits are tangible like the ability to display fine details that may be critical for the task at hand. Sometime the benefits are more subtle like the “feeling” the image is crisper, sharp or focused. Perceived resolution is dependent upon the projector resolution, screen, viewer location, room ambient and the content – and these cannot always be controlled. Developing a definition for “better resolution” is therefore quite difficult. Since most content today is projected in the Snellen acuity range, perhaps a definition of “better resolution” is one where the projection solution allows the delivery of images that are in the Simple acuity range. This is a concept we seek industry feedback on. When does offering a “better resolution” solution make sense? Clearly, if viewers are seated close to the screen, you don’t want them to see pixels, so increasing resolution may make sense in such situations. When viewers are further back, the need for increased resolution become more content and cost dependent. If the increase in crispness is important and there will be high spatial frequency content, than more pixel can be warranted. However, there is always a cost to increasing resolution in terms of complexity, system volume, bandwidth and dollars.

High Dynamic Range and Contrast The real world contains a wide range of luminance values – about 15 orders of magnitude in cd/m2 or nits (Figure 3). The human visual system can cover most of this range, but not simultaneously. Our eyes adapt to the average light level so that our steady state luminance range is more like 3.7 orders of magnitude (103.7= 5011:1). The steady state adaption of the human visual system is complex and not tied to just the average illumination level of a particular scene. For example, if the viewer is focused on a bright object, the eye will adapt to that luminance level quickly, making it harder to see darker details. So what happens when content is captured by a camera, processed, distributed and displayed on a TV or projector? Figure 4 shows how the colors and range of luminance (contrast) is reduced through this process. One of the reasons why a TV or projector doesn’t look exactly like the real world is the fact that the colors, brightness and range of luminance are reduced. Note that 3D parameters like depth perception and parallax are also important parameters to achieve this realistic look and feel. Modern professional cameras can capture 11-14 f-stops of luminance, but this wide dynamic range is not preserved. In the production process, this range is “squeezed down” to a smaller Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

9

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

dynamic range. In television, the current practice specifies that the peak luminance of the mastered content will not exceed 100 nits while the black level is set at around 0.1 nits. In the cinema, the DCI specification calls for a 2D luminance of 48 nits (14 fL) and a 3D luminance of 24 nits (7 fL). Other professional content may have a wide range of peak luminance and black levels.

Figure 3: Range of natural and human luminance values (Source: Dolby)

Figure 4: Reduction in dynamic range through the capture-to-display pipeline (Source: Dolby) Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

10

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

The specifications for standard dynamic range (SDR) content were developed in the CRT days where mastering CRTs had the same luminance range as consumer TVs. From a practical standpoint, a luminance range of 0.1 to 100 nits made sense as it matched the display technology. As LCD TVs and projectors became more prevalent, this mastering range was not changed. Instead, if the display had higher peak luminance, the content was simply scaled from the 100 nits to whatever peak level the display offered. The dynamic range of the display may have improved, but the basic dynamic range of the content remained the same. High Dynamic Range or HDR changes the way content is mastered so that it now has a greater range in luminance values. Then, HDR displays can show this wider range to great effect. This is illustrated in Figure 5.

Figure 5: HDR vs. SDR Images (Source: 20th Century Fox) HDR is an increase in the contrast ratio. This often means an increase in the peak luminance as well as a decrease in the lowest luminance levels. In theatrical projection for example, it is mainly a reduction in the black levels, while for TVs, it is a change in both. There is no definition of an HDR display, but in projection, sequential contrast in excess of perhaps 5,000:1 without a dynamic iris, would probably be considered to be HDR contrast. But intraframe contrast is also important and what checkerboard pattern you use matters as well. These remain non-standardized attributes at this time.

Benefits & Trade-offs The added value of HDR is twofold: more realism and more creativity. Matching the dynamic range of the human visual system better and showing more detail in dark scenes makes the perception of the digital image more realistic. Furthermore, having more degrees of freedom

Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

11

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

adds value to the creative process. Like the painter having a bigger pallet, HDR gives more options to the digital content creator. The challenge with transmissive display technologies (flat panel or projection) and HDR is the desire to increase brightness and reduce black levels simultaneously. That is quite hard to do, and has a direct impact on cost and complexity. Emissive approaches offer greater long term potential for HDR.

Wide Color Gamut (WCG) Wide Color Gamut (WCG) is a term that is applied to content or displays that have a color gamut beyond the existing color standard. For consumer TV and most professional applications, this means beyond Rec.709. For cinema, this means beyond DCI-P3.In both markets, the Rec.2020 gamut is considered the target for WCG.

Figure 6: Various Color Standards on the 1931 CIE Chromaticity Diagram and u’v’ color spaces Rec.2020 is the largest color space and does the best job of showing all of the colors available in nature (Pointer’s gamut), plus many additional colors (neon lights, LED lights, computer generated colors, etc.). The left image of Figure 6 shows how three color standards are overlaid on the horseshoe of visible colors using the 1931 CIE xy method. The right side of the figure shows another way of showing the same information but using the u’v’ coordinates. The u’v’ space is an improvement on the xy space as it is more linear than xy. As can be seen, P3 and 2020 expand the colors visible on the display considerably beyond Rec.709. A typical example is the color cyan, as appearing in nature in sky- and ocean-colors. This is most noticeable in the expanded red and green colors and the saturation of these colors Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

12

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

(closeness to the edge of the horseshoe). The 3 color points (R, G and B) of Rec.2020 are located exactly on the boundary locus of the 1931 CIE chromaticity diagram. This means that they can only be reached by perfectly monochromatic sources.

Benefits & Trade-offs Displaying content in the P3 or 2020 color gamut has clear advantages over content displayed in 709. Not only is it possible to show more colors, but these colors can be more deeply saturated. People clearly see more saturated colors, offering a solid benefit. Deeply saturated color also offer an addition benefit known as the Helmholtz-Kohlrauch (HK) effect. This effect means that deeply saturated colors look brighter than the same color when less saturated – even if both have the same luminance. The strength of the effect varies by color, but can contribute to the perception that the image is up to 30% brighter. That’s a significant benefit to moving to a P3 or 2020 color space. There are several challenges in the move the WCG. One is realizing displays that can achieve the 2020 gamut. Quantum dots are the leading candidate in flat panels and RGB lasers or LED are viable in projection. But the 2020 color gamut has no tolerances so no display current can say it is 2020-compliant. This needs to change. Secondly, there will be a much bigger need for color remapping algorithms. That means taking content that is mastered in one color volume and displaying it in a display with a different color volume. In the cinema market, movies are mastered in the P3 color gamut and shown on projectors that are calibrated to show P3 colors. But the RGB laser projectors can create colors that are outside of the P3 gamut and are close to the full 2020 color gamut. Stretching to colors from the P3 master to the wider colors available in the RGB laser projector will probably not happen in the cinema because of the tight control of the creative process and respect for reproducing the director’s intent. But as 2020-capable projectors and display show up in homes theme parks, museums and other venues, additional issues will arise. For example, if a near-2020-capable display is available and content is mastered in a smaller color volume like P3 or 709, operators may be tempted to stretch the colors to the wider color gamut. This will create an image that is not what the content creator intended, but it will be done nonetheless. Developing algorithms that do this well without distorting other colors and ruining flesh tones will be desirable. Such gamut mapping algorithms should be carefully evaluated for each application. In addition, there will be the need to reduce color volume. This will occur when 2020 master content is played back on a P3 or 709 display or P3 content played back on a 709 display. For these scenarios, there are two types of solutions: clipping or perceptual rendering. Clipping means the saturation of the color, which is beyond the capabilities of the display, are reduced to what the display can natively produce. This is usually done be reducing the saturation by calculating a vector from the color back to D65 and seeing where it intersects the edge of the native color volume. The color is then displayed with this reduced saturation level. A second method to deal with out-of-gamut colors is called perceptual rendering. Here, the idea is to try to select a new color or series of colors that might look similar to the original color. In other words, if a pixel with cyan is out of gamut, maybe a combination of blue and green at Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

13

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

the right saturation and luminance levels within the native gamut of the display can approximate the original cyan color. These colors may be created by spatial dithering (flashing) or temporal dithering. Again, this requires more sophisticated processing, but these techniques provide the freedom for display manufacturers to differentiate their performance.

High Frame Rate (HFR) High frame rate means different things to different groups of people. For those offering content now at 30 fps, going to 60 fps is high frame rate. But the UHD phase 2 specification calls for UHD content to be able to be mastered and delivered at 120 fps.

Figure 7: 24 vs. 48 frames per second In the theatrical world, high frame rate (HFR) means going from 24 to 48 or 60 fps. This has already been pioneered on The Hobbit movies to mixed reviews. The next Avatar movie is probably going to be in HFR but reports suggest this will be in variable frame rate from 24 to 60 fps. From a technology viewpoint, going to 2K at 120fps or 4K at 60fps is perfectly possible, as are variable frame rates. High frame rates need to consider the capture and display techniques. Higher frame rates result in less blurring, but the speed of the object and the capture frame rate determine the amount of blurring. In addition to frame rate, one must consider the shutter angle in capture and playback. This impacts perception of motion blur and judder. A number of groups are working on developing techniques that allow for capture of content at high frame rates of 120 or 240 fps, with processing techniques that can create finished content at various frame rates and shutter angles. Such tools would expand the creative tool box allowing traditional 24 fps 180-degree shutter angle “film like” content with all kinds of other combinations going up from there. This requires more post production, but can be used to create new looks and experiences. Projection systems are particularly adept at being able to adapt to these new capabilities.

Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

14

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

Benefits & Trade-off Reduction of motion blur is generally accepted to be a desired outcome as it increases the clarity of the image and makes viewing of faster moving objects less objectionable. However, for movies, the 24 fps rate is likely to be mainstream for some time. Films have a certain look that includes the motion blur. Movies are often about suspending belief so having content that looks too “real” can be a negative and not a positive. Increasing frame rate has its consequences technically too. Mainly, the bandwidth of the image processing to display pipeline needs to support the higher frame rates. In addition, some solutions may employ frame rate conversion algorithms (like converting 24 fps content to 60 fps content) which can introduce many type of artifacts and change the entire “look” of the content. The viability of these should be carefully assessed for each application.

Bit Depth Bit depth refers to the number of digital bits devoted to image quality. It typically refers to the bit depth per color in an RGB video format, so 8-bits per color is a 24-bit video – standard today. 8 bits per color means that for every color channel (like R, G or B), there are 28=256 distinct video levels between black and white. In total, there will be 28*28*28=16.777.216 different combinations, potentially representing unique colors. Capture systems and some post production processes work with 12- or even 16-bits per color, but most professional HDTV work today is done at 10-bits per color, with the final rendering done at 8 bits for distribution to the end user in many consumer and professional applications. Digital Cinema requires content mastered in 16 bits and delivered to the theaters with 12bits per color. Professional content varies from 8 to 12 bits, depending upon the need and application. The distribution of code values over the luminance ranges is called the “gamma curve”. The name ‘gamma’ comes from the exponent used in representing luminance coding in video levels: L=Vγ. Since in many cases the shape of this curve is not exactly a power law, a more generic term is used: Opto-Electric Transfer Function (OETF) at the side of the camera, that encodes optical properties into signal codes, and Electro-Optical Transfer Function (EOTF) at the display side, that translates video signals into linear-to-light optical parameters. And for HDR in particular, new OETF and EOTFs curves are needed. That’s because the existing curves were designed for standard dynamic range content and simply scaling these code values to an expanded range won’t work. Why? Two reasons. For one, the curves were designed to try to have steps between code values be about the same from a visual perception point of view. Stretching them messes up this relationship. Secondly, expanding the range with only 8 bits per color does not provide enough gradation between steps. The result is “banding” or posterization in content. This may be familiar as discrete bands in a sky scene or shadow details, for example. As a result, HDR content needs at least 10-bits per color for the finished product. But the gamma curve issue is still a bit undecided. Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

15

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

So far, the leading candidate for wide adoption is the so-called “PQ” or Perceptual Quantizer curve. This has been standardized as SMPTE 2084 and is part of the new Blu-ray 4K specification. Figure 8 shows the PQ curve along with two 709 curves, and the BBC HDR proposed curve as well. The shape of the PQ curve was designed to match human perception over a wide luminance range, with the bit depth extendable depending upon the range of luminance values covered. But most importantly, for the first time a code value is tied to a specific luminance level. That means when the content creator wants a certain pixel to be at 100 nits, a properly designed display will reproduce it at 100 nits – if it uses the PQ curve. Multiple other HDR transfer functions have been developed as well. Some of them are for cameras to be used at capture; others (such as the BBC/NHK Hybrid Log Gamma) to resolve issues of backwards compatibility.

Figure 8: Various EOTFs being Considered for HDR Content and Display

Benefits & Trade-offs With more pixels, more colors and greater dynamic range, more bits are needed to keep the image looking smooth and lifelike. The PQ curve for 10 bits and higher content creation and display is desirable, but other new curves may be suitable for certain applications as well. These need to be evaluated carefully for their impact on the ecosystem and their backward compatibility.

Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

16

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

Moving to higher bit depth has ramifications throughout the acquisition, production, distribution and consumption pipeline. Delivering 10-bits per color to the home requires a major upgrade. Delivering it in professional applications will be challenging as well as equipment may have to be upgraded to support this.

3D Stereoscopic 3D is still a big element in theatrical. For consumer TVs, 3D has faded from interest in North America and Europe, but is still popular in Asia. Virtual reality may actually be how 3D is reintroduced to the consumer market going forward. 3D is also used in a range of specific professional applications. That includes CAD design for automobiles, architecture or aircraft, visualization of computer generated models or 360degree video content, medical imaging data and much more. To create a 3D image, the left and right stereo pairs are presented in a time sequential manner. (This is the case for a single projector only; For dual projection, the images can be shown simultaneously) That means the projector has to run at twice the normal rate. This fact, plus the methods used to optically encode and decode the stereo images, leads to significant loss of brightness. RGB laser projectors address this issue as discussed in the Higher Brightness section. 3D projectors also need to have high stereo contrast – that is, excellent separation of the left and right images with little overlap. High contrast projectors with appropriate “black time” between images, is the key to high performance. Are 3D pixels better pixels? If you can visualize or perform your task easier in 3D, then they should be considered a better pixel. But these pixels are only “better” if the content is acquired and processed correctly and the display system can present images with minimal eyestrain or distortions. That is not always the case as 3D can be hard to acquire or create correctly. Stereoscopic 3D only provides one of the cues that humans use to see, interpret and understand a 3D world. What 3D content provides is a single binocular stereo pair (left and right eye images) with horizontal parallax only. In the real world, we can move our heads left and right to acquire multiple horizontal parallax views. We can also move our heads up and down to acquire vertical parallax. The limitations of glasses-based stereo displays also impact the image quality. This is so called accommodation-vergence mismatch issue. In the real world, when we look at an object our eyes focus (accommodate) on the object and our eye “toe-in” or converge at the same time. These fine muscle movements are calibrated by the brain help us understand how far away the object is. In 3D displays, the point of focus is always the display but our vergence can change depending upon whether object is in front of or behind the screen. Accommodation and vergence are only matched when the object is at the screen plane. Otherwise, this mismatch causes eyestrain that can lead to the discomfort some people feel. That is why minimizing this mismatch range is so important, but this must be done in content creation.

Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

17

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

Autostereoscopic displays have been developed too that allow 3D viewing without glasses. This address one deficiency by adding more horizontal views, but in so doing, most techniques today also introduce sweet spots and transition zone, which can be even more disturbing. Light field displays go further and holographic displays will be ideal, but these are still quite experimental at this stage.

Benefits & Trade-offs 3D displays offers clear benefits in entertainment and multiple professional applications – if the content is created well and the display system is high quality. The ability to see spatial relationships that are not possible in 2D displays is extremely powerful, but only if done correctly. Fortunately, many of these applications are now mature enough to understand how to best create and display 3D with high quality. There are drawbacks of course. In addition to the human factors issues described above, 3D requires users to wear glasses, which is not always ideal. Plus, running the display at twice the frame rate and adding additional technology to display the left and right eye images separately, adds complexity and cost, often leading to lower efficiency as well.

Summary The idea of better pixels as an all or nothing proposition is the wrong way to think of it. In reality, professional projection systems will come to market with various combinations of better pixel attributes. These will be designed to fit specific application needs and price points and to provide differentiation in the product line. Some combinations will be more powerful than others, however. HDR with 4K resolution, wide color gamut and 10-bit processing will become an increasingly popular combination, we believe. HFR and 3D will find specialized uses. Improvements in brightness and uniformity will be led by RGB laser and LED projectors and will expand as these products become more mainstream.

Im pl i cati o n s fo r t h e C apt u r e , P r o ce ssi ng an d D i str i bu ti o n o f Bette r P i xe l C o nte nt Better Pixel Capture The capturing of content with better pixel attributes is most mature in the movie segment. Here, large format single-imager cameras with an RGGB pixel arrangement (Bayer pattern) dominate. These cameras offer 12-15 f-stops of dynamic range, a wide color gamut capture and speeds that can go up to 240 fps with 4K, 5K, 6K and even 8K resolution. This is how most better pixel video content is being generated today, but the cameras are costly. Also costly are the storage devices needed to capture large pixel images along with the associated processing and on-set monitoring devices and monitors. In effect, the entire near realtime rendering and display of HDR and WCG content is now becoming possible. This allows content to be de-Bayered into a YCrCb or RGB format and output as RAW data or compressed in common video formats. Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

18

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

The content has a gamma curve applied – either the PQ curve or proprietary log-based curves. On-set HDR monitors can then play back the HDR content. Most of the playback monitors support a native P3 color gamut at this time, but most will allow Rec.2020 content to be input with out-of-gamut pixels perceptually rendered or clipped to the P3 gamut or highlighted to showcase them. Development of better pixels broadcast TV equipment is much further behind. Here, cameras are typically three chip RGB types. The experiments being done today are mostly using modified cinema gear, however, which is not ideal for the lens requirements of broadcast TV and many other professional applications. Development of non-cinema cameras that can support HDR, wide color gamut, expanded bit depth and other better pixel attributes is in the early stages of development. As a result, it is likely that most “better pixels” content in the near term will be generated with cinema-grade equipment.

Better Pixel Processing The workflow upgrades needed to work with better pixels content depends very much upon the application and the better pixel attributes that need to be managed. In the movie and postproduction environment, there is a desire for on-set near realtime playback of just captured content. Canon, for example, has shown solutions that meet this need today and others will follow. Most work is done in non realtime. Here, non-linear editing software and color grading tools need to be updated to support better pixel content. Once most content is coming in at 4K to 6K resolution with wide color gamut and high dynamic range, tools need to be able to support this. Software solutions from a number of companies are now available to enable this workflow. Outside of the cinema and post-production community, development of better pixels processing is being done in specialized segments. For example, in broadcast, the move from 60 fps to 120 fps is a key better pixels area of investigation. HDR is also being looked at for application to UHD-4K as well as HD content as the benefits are not dependent upon the resolution. However, color gamuts are still mostly locked at 709 as this has been the standard for a long time. In the simulation market, high dynamic range content and projectors have been available for some time, although they have not been called that. Here, achieving the darkest black level is critical for good contrast so the entire image generation pipeline has been optimized to support this workflow. So far however, this segment has paid little attention to wider color gamut. But they do like to have 120 fps content to reduce motion blur and the potential for nausea in the simulator. Automotive and aircraft design is clearly interested in lots of pixels, high dynamic range and wide color gamuts to be able to create as realistic simulations of their designs as possible to make design decisions. The pipeline to do this exists today. Beyond these specialized markets, there seems to be little ability to generate and process better pixels content in a standard way and format. This will remain an issue for wider adoption for some time. Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

19

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

Better Pixel Distribution Distribution refers to the movement of better pixels video within a professional facility or environment, the distribution of content from one location to another, and the final connection from the playback device to the TV, monitor or projector. In a professional facility or environment, lightly compressed or uncompressed video is common. Light compression formats like J2000, TICO, VC-2, DSC and others are emerging as next generation solutions in some of these professional markets. Realtime distribution of better pixel content remains an issue. In broadcast for example, quad 3G-SDI is now used for 4K content at 30 fps, but other solutions will be need to move to 60 fps. IP solution, 12G SDI and other approaches are being considered with some advanced facilities doing some initial installations. This is an expensive upgrade which will slow adoption. For non realtime distribution from one place to another, compression is used except in the most critical applications. This may be light compression or medium to heavy compression, depending upon the needs of the application. Since most better pixel content is also 4K resolution today, compression is often needed to squeeze the content into a smaller pipeline. HEVC is the leading new codec for 4K compression for live transport over IP and other applications like Ultra HD Blu-ray, but others are also being seriously considered. This includes’ Google VP-9/VP-10, Cisco’s Thor and Mozilla’s Daala which are proposing royaltyfree options to the royalty-bearing HEVC codec. These codecs should support most better pixel features, but again, will have to be carefully evaluated for an application. Distribution using these codecs can be for cable/satellite TV, over the top TV, or professional distribution over Internet infrastructure. Compression values will vary depending upon the link bandwidth and/or desired video quality. In ProAV applications, distribution is likely over IP networks using fiber or CAT cable, HDMI, HD-BaseT or DisplayPort connections. There are many different configurations possible and many variations in each connection scheme, so extreme caution is urged when choosing a physical connection method. All can support 4K distribution but support for better pixel features may be problematic. These same connection options will also power the final display/projector link. Obviously, the projector or display must support better pixel features if you deliver it to the display, so make sure it does. There is a lot more complexity to these connectivity and processing issues that are beyond the scope of this white paper. As a result, we want to focus a little more closely on the needs of a better pixel projector.

Im pl e m enti ng a Bette r P ixe l P r o je c to r As we have noted already, different markets and applications require/want different better pixel features. Development of better pixel projectors is not without its trade-offs, however. Issues include: •

Size: the most important law of optical design in projectors is étendue. Étendue can only increase through the optical path. That means that as you start increasing light

Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

20

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

•

•

•

•

sources for higher brightness or modulator size for higher resolution and brightness, the projector size quickly scales up too. Efficiency: all the power you are not using to project on the screen is still pulled from the power plug and internally dissipated as heat. Dual modulation designs to achieve HDR, like Dolby Cinema, introduces even more inefficiency. Processing: Image processing done inside or outside the projector is needed for image scaling, color processing, compression decoding, HDR signal decoding and more. This can add latency and synchronization issues for multi-projector applications using blending and/or warping. Lifetime: all physical components have a limited lifetime. The most critical components are those that must manage a big thermal load. When trying to scale up brightness, the internal load increases, as does the need to careful cooling. This is even more critical in applications where the projector is built for professional applications, some of them running 24/7. Cost & Complexity: A better pixel projector offers stunning performance, but with increased cost and complexity.

Let’s now consider some markets where better pixel projection solutions are being implemented.

A ppl i cati o n s fo r Bette r P ixe l s Table 2 shows a number of better pixel attributes for projection cross referenced with a variety of professional markets. An X indicated that that feature is highly desired in that market segment. As can be seen, there is broad desire for improvements with these attributes.

Table 2: The Need for Better Pixels in Professional Projection Applications Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

21

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

But can the industry deliver better pixel projectors? We will explore this and the consequences in the next section.

Cinema In the cinema market, there are several very impressive better pixel offerings. One key trend is the use of red, green and blue lasers as the light source coupled to a 3-chip DLP engine with 4K resolution that can run at 60 fps. In addition to the better pixel features of 4K and high frame rate of 60 fps for the cinema market, the laser source can deliver very close to the full 2020 color gamut. That is because the laser primaries are very near the primaries required in the 2020 specification. This is one on the few projectors that can display this color gamut. The laser sources are also configured to provide a 6p or 6 primary 3D capability as well. That means there are two principal red, green and blue laser wavelengths –one set for the left eye and one set for the right eye. Using spectral separating glasses, the user can then see the 3D content. And, this content can now run at the standard 14 fL of peak brightness due to the high lumen output of the projectors. The other major advantage of the RGB laser projector is the increased uniformity of the light out of the projector. Now, the center to edge drop off can be as little as 10% compared to over 25% in a Xenon lamp cinema projector. Finally, content for the cinema must be mastered to DCI specification, which includes 12bits per color. So, the current class of RGB laser cinema projectors meet most of the checklist items for a better pixel projector. . To create an HDR cinema projector requires a new design to the standard 3-chip DLP prism architecture, or a big trade off in terms of efficiency. The typical Xenon-lamp based DLP 3-chip DLP projector has a contrast ratio of about 1800-2000:1. Moving to an RGB laser source increases the native contrast beyond 2500:1. Further increases are possible by further aperturing of the optical system, but with the loss of a lot of light and the thermal and electrical consequences as well. To solve this problem, two new designs have been developed by IMAX and Dolby which both claim to offer HDR capabilities added to all of the other features mentioned above. The IMAX design throws away the prism and creates a design where the 3 DLP chips are mounted in the engine and aligned without the prism. This is to reduce scattering and other optical issues that reduce contrast. The Dolby design is essentially two 3-chip DLP engines in series. The first engine acts as a normal cinema engine while the second one adds an additional level of modulation. Neither company has published contrast specification, but it is believed the IMAX design achieves about 8000:1 full on/off (sequential) contrast ratio, while the Dolby design should be much higher. Besides having a superior projector, controlling the room environment is critical for achieving superior image quality. Even moderate levels of ambient illumination can degrade high image contrast and desaturate the wide color gamut of the image. Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

22

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

In studies conducted by RealD and Barco of the cinema environment, they noted that there are four main contributors to the final contrast ratio reaching the eyes of the audience. • • • •

Sequential contrast ratio - projector black state Projector lens veiling glare Auditorium ambient light level Auditorium contrast ratio

The sequential contrast ratio is mostly a measure of how dark the black state is in the projector. Anyone who has been to a theater knows there is always that glow even with no content on the screen. Black is not black. This is the area where HDR projectors can really increase the contrast ratio. Projector lens veiling glare happens when stray light reflects off multiple surfaces and shows up as a glow at some other point on the screen, reducing contrast. This is calculated by measuring the MTF from the ANSI checkerboard contrast – a conservative measurement that yields a contrast of about 1300:1. However, measured in-theater contrast varied from 4001700:1.

Figure 9: System Contrast Formula (Source: RealD, Technology Summit on Cinema) The auditorium is not as dark as you might think. Those exit lights add far more light than commonly believed. Measured contrast was in the 850-980:1 range Finally, the theater has a contrast ratio as well. This is minimized by have black light absorbing walls, carpet and seats, but according to Dolby, it can still be 5% or a contrast ratio of 200:1. And that is with no people in the theater. Once you add people, their faces and clothes reflect more light, further degrading the contrast. Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

23

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

These contrasts add up as shown in Figure 10. The theater system contrast is produced by adding up all the contrasts in the environment, which includes the projector, veiling glare, seats, surfaces and people in the room. The “a” represents the audience occupancy percentage.

Figure 10: System Contrast Ratio (Source: RealD, Display Summit 2015) The result for a typical theater using a DLP projector with a Xenon lamp is a real theater system contrast level of about 354:1. This level was verified by measuring 8 different theaters. But look what happens if you use an HDR projector with a contrast of 1,000,000:1. The theater system contrast increases significantly to 698:1. But the same result can be achieved with a projector that has 100,000:1 contrast as well, which may be a lot more easily produced and at a lower cost. This is something projector makers need to carefully consider. At Display Summit 2015, RealD updated the analysis to consider the effect of the average luminance of the content on the system contrast ratio. The company found that 50% of theatrical content is below the 5% average luminance level (5% of full white), and 90% is below 20% brightness. Other studies have shown that the average picture luminance of TV content is 17%, which is what is used to test TV for their energy usage. Figure 10 shows a calculation of system contrast vs. projector sequential contrast ratio. When content is dark (5-10%), system contrast can rise dramatically with increasing projector sequential contrast. But even for very dark content, there are diminishing returns after increasing the projector sequential contrast beyond 10,000:1 or 20,000:1. This number is even lower as the average picture level increases. Therefore, RealD conclude that having a projector with 500,000 or 1,000,000:1 does not produce a noticeably better image than one with 10-20,000:1. Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

24

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

At the Fall SMPTE conference, Barco presented their theatrical contrast model. It uses slightly different parameters but arrives at a similar conclusion. Figure 11 shows the results of this model. The vertical axis corresponds to the white content of the movie – essentially the average picture level in TV terms. The gray box shows the typical range of values of the majority of movie content. The horizontal axis represents the sum of scattering and reflectivity elements. Again, the gray box represents the range of typical values. A projector with a sequential contrast ratio of 1,000,000:1 was model and the resulting curves show the theater contrast level. The circle near the middle of the gray box is supposed to be the median point which shows the theater contrast in the 700:1 range.

Figure 11: Theater Contrast for Various APLs and Scattering/Reflectivity Parameters (Source: Barco) Barco drew three conclusions from this analysis: •

•

Auditorium reflectivity and scattering largely influence the effective theatre contrast for all image content, especially for median image brightness for which the projector choice makes little difference (700:1 for 1M:1 projector vs. 500:1 for a standard DLP projector). For average auditoriums and pretty dark image content (1.5%, defining a 10:90 split of the frame count), a 1M:1 projector only offers a twofold improvement of image contrast. Most of the contrast improvements happen with relatively lower projector contrast improvements due to the hyperbolic nature of the curve.

Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

25

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

•

•

For the very best auditoriums and even darker image content, the difference in contrast becomes more apparent – however this is for a fast decreasing number of movie frames as one moves further down from 1.5% frame brightness. In practice, it turns out that some effort in reducing the auditorium reflectivity and lens glare (by e.g. keeping the lens and port hole clean of dust) can yield the same or better results than moving to HDR projection.

3D is also an important better pixels feature in the cinema. Two approaches dominate: polarization and spectral separation of the left and right eye images. The polarization approach uses low cots glasses, but requires a silver gain screen, which exhibitors don’t really like. Spectral separation can use a white screen, but the glasses are expensive.

Design The design or automobiles, aircraft and other high-value items is increasingly being done using high-fidelity displays to visualize and simulate the design. Sometimes this is done in 3D and sometime using virtual reality headsets. But whatever the display modality, the best image quality is critical to accurately portray the product as important design decisions hinge on these simulations/visualizations. One popular implementation is a rear-projected set-up with multiple blended 4K projectors. Important better pixel parameters here include everything from Table 2 except high frame rate as these models are usually quite static or change slowly. Wide color gamut, high resolution, and high dynamic range are particularly treasured as this increases the realism of the display. Room ambient are typically normal lighting, so attention to the screen materials and brightness of the projector are needed to maintain contrast and color depth.

Training & Simulation The simulation market is similar to the design application, but the environment is much more controlled. These military and commercial simulators are only illuminated by the projected images and cross contamination of light from one part of the screen to another part, can severely impact contrast. Unique screen designs are often needed in this environment along with projectors with extremely low black levels – but not very high lumen outputs. High frame rate is needed in many applications to reduce motion blur and provide a more lifelike training environment. Solutions up to 120 fps are preferred. High bit-depth is also needed to avoid any banding issues which can be detracting. Oddly, color performance is not currently a key value in this market. However, we suspect as wider color gamut projectors become available, they will be adopted and the image pipeline upgraded to support more realistic and accurate colors.

Rental & Staging For the rental and staging market, one of the most important parameters is brightness. This market can take all the brightness projector makers can offer as applications can include outdoor venues. But, companies always worry about the size and weight of these every more brilliant Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

26

www.insightmedia.info Copyright 2016 All rights reserved

Better Pixels in Professional Projectors

projectors. RGB laser projectors offer some flexibility here in being able to deliver lots of lumen power. In the future, these laser sources may be located separately from the projection head, allowing more flexibility in projector rigging. This is already happening in the theme park market. Other key features include reliability, wide color gamut and state-of-the art video processing.

Corporate In the corporate fixed install market, the projectors can be used in meeting rooms or auditoriums. Here, brightness vs. size is always a concern as is the noise the projector makes. Color gamut, resolution and HDR will become increasingly important as the corporate market is influenced by the consumer TV market. As HDR/WCG UHD content becomes more prevalent, it will find its way into corporations. This will likely start in the executive boardroom and migrate down to auditoriums and meeting rooms.

C o n c l u si o n In this white paper, we have attempted to qualify the meaning of “better pixels” in consumer and professional applications. The cinema market is leading in the adoption of these technologies, but various combinations of these better pixel features are and will permeate further into other professional markets. We hope this paper helps you to understand these trends and plan accordingly for your applications or product developments.

Insight Media 3 Morgan Ave. Norwalk, CT 06851 USA

27

www.insightmedia.info Copyright 2016 All rights reserved