Mastering the Adobe Media Encoder Critical Concepts and Definitions

 by Jan Ozer

 Thanks for downloading this e-book:  When I started planning the Udemy course, Mastering the Adobe Media Encoder, I realized that to use the program effectively, you need to know quite a bit about a lot of topics, like codecs and compression, container formats, file characteristics like resolution and data rate, bitrate controls like constant and variable bitrate encoding, and delivery concepts like single file and adaptive streaming.  In my first Udemy course, Video Compression for Web, Disc and PC/ TV/Console Playback, I addressed these topics with several hours of videos. Soon there after, I was introduced to the concept of a “slidedoc” by noted author Nancy Duarte. According to Duarte, a slidedoc “ is a document created using presentation software, where visuals and words unite to illustrate one clear point per page.”  While video is great, I thought addressing the concepts identified above in a slidedoc would be a more accessible approach for many learners. I liked the result so much I decided to give it away for free.  Another motive, of course, was to get you interested in one of my Udemy courses, or my book, Producing Streaming Video for Multiple Screen Delivery, which all go a lot deeper into more advanced topics like producing H.264, and customizing files for different delivery targets. I’ve included more information about these products with discount codes at the end of this e-book. If you find this document useful, and are looking for additional resources, please check them out. © © Jan Ozer. Ozer 2014

2

TABLE OF CONTENTS

Section 1

Section 2

Section 3

Section 4

Section 5

 Codecs and Compression

 Choosing File Parameters

 Bitrate Controls

 Frame Types (I-, P- and BFrames)

 Streaming Delivery Concepts

+  What is compression? +  How does it work? +  What is a codec? +  How do you choose the right codec? +  Container formats

+  Resolution +  Aspect ratio +  Frame rate +  Field order +  TV standard +  Data rate +  Bandwidth

+  VBR and CBR defined

+  Defining the frame types

+  Download and play

+  The quality differential

+  Choosing an Iframe interval

+  Progressive download

+  When to use CBR and VBR

+  Configuring Iframe parameters

+  Streaming

+  Producing CBR

+  Working with Band P-frames

+  Producing VBR +  Constant Quality/ CRF

+  Adaptive streaming +  Customizing encoding for delivery mode

© Jan Ozer 2014

3

ADOBE MEDIA ENCODER

Compression, Codecs, and Container Formats You probably have a pretty good idea about what a codec is and what it does, but take a few moments to scan through this section and get familiar with concepts like a lossy codec, and how you choose the right codec for a specific task. Once you choose the right codec, you also have to choose the right container format. You’ll learn about that, too. © © Jan Ozer. Ozer 2014

4

CHOOSING BASIC FILE PARAMETERS Overview

 Adobe Media Encoder lets you access literally dozens of codecs. What do they all do, and how do you choose the right one? That’s what you’ll learn in this section.  Of course, choosing the right codec is only step 1. You also have to choose the right container format; otherwise your file may not play on your target system or device. We’ll finish this lesson by discussing what a container format is and how to choose the right one.

Agenda 1.  What is compression? 2.  How does compression work? 3.  What is a codec? 4.  How do you choose the right codec? 5.  What’s a container format and why do I care?

© Jan Ozer 2014

5

WHAT IS COMPRESSION? A technology that reduces the size of video, audio or still image files  Video Compression Technologies • 

AVC/H.264

• 

HEVC/H.265

 Audio Compression Technologies • 

•  • 

Advanced Audio Coding (AAC)

• 

Windows Media Video

• 

VP6

• 

JPEG

• 

PNG

• 

TIFF

• 

Targa

MP3

MPEG-2 • 

 Image Compression Technologies

Dolby Digital

© Jan Ozer 2014

6

HOW DOES COMPRESSION WORK? There are two kinds of compression; lossless and lossy  Lossless technologies, like PK Zip, retain all data and restore the original file bit for bit. How, lossless techniques have very limited application in audio and video streaming because they can’t reduce file sizes sufficiently for efficient delivery.  Lossy compression technologies discard data and restore a facsimile of the original file. The more compression that you apply, the more data gets thrown away, and the harder it is for the compression technology to accurately restore the original file. All streaming compression technologies are lossy.

Lossless

Lossy

Keeps all data/ Restores original file exactly

Discards information/ restores facsimile of original file

PK Zip

All streaming audio and video codecs

Very limited utility with video and audio

Effective for streaming, but at a quality cost

© Jan Ozer 2014

7

THE IMPACT OF LOSSY COMPRESSION The more you compress, the more quality you lose. File data rate of 2.1 Mbps on the left, 504 kbps on the right.

© Jan Ozer 2014

8

OUR JOB To produce the best quality file given the reality of our constraints

In some configurations, it’s challenging to produce high quality streaming video, particularly at low target data rates. With all of your encodes, however, you should seek the optimal balance of file resolution and data rate to produce the best quality file. As you’ll learn in the next section (and probably intuitively already know), dropping the resolution of a file while keeping the data rate constant produces higher quality video. Conversely, boosting the data rate also increases file quality. These are the tradeoffs you’ll consider in the next chapter. © Jan Ozer 2014

9

WHAT IS A CODEC? Codecs are compression technologies

 All compression technologies have two components. We encode in our office/ studio/lab; Viewers decode when they watch the video.  EnCOde/DECode= CODEC  Over the years, the encode/decode functions have been condensed into the term codec. Simply stated, a codec is a compression technology, whether audio, video or still image.

© Jan Ozer 2014

10

CHOOSING THE RIGHT CODEC It’s all about the target; but mind the container format Target

Codec

Container Format

Computer playback (Flash/HTML5) – Single File

H.264

MP4

Computer playback – Flash/HTTP Live Streaming – adaptive

H.264

MP4/MPEG-ts/F4F

Mobile devices – single file

H.264

MP4

Mobile devices – adaptive streaming

H.264

MPEG-ts

Set top boxes (Roku/Apple TV)

H.264

MP4/MPEG-ts

Uploading to UGC or OVP

H.264

MP4

MPEG-2

MPEG

H.264/MPEG-2

MPEG

DVD Blu-ray

When choosing a codec, the most important consideration is the capability of the target. Other than when you’re encoding for DVD, H.264 is almost always the right codec, but as you can see in the chart, the required container format varies widely depending upon the target device and whether you’re distributing single files or via adaptive streaming. We cover container formats next. © Jan Ozer 2014

11

WHAT’S A CONTAINER FORMAT? The technical description  Container format: “A meta-file format whose specification describes how data and metadata are stored” (Wikipedia). Players like Flash, iPods, iPhones and iPads, as well as set top boxes like Roku and Google TV, need video delivered in a compatible container format so they can find the relevant bits of audio and video data and identify characteristics like audio and video codecs.  Container formats are very technical, but you don’t need to know the technical particulars. Rather, you just need to choose the correct container format for your target when producing the file. © Jan Ozer 2014

12

WHAT’S A CONTAINER FORMAT? The visual description and transmuxing  The point here is that the container format sits above and is separate from the encoded audio/video content. It’s very easy and efficient to change the container format of a file because you’re not re-encoding the audio and video; you’re just changing the data in the file header.  When delivering the same video to different target platforms, often you must use different container formats, like MPEG-2 transport streams for HTTP Live Streaming to Apple Devices, and the F4F format for delivery to the Flash Player. Servers like the Wowza Media Server can change the container format on the fly to deliver to these different targets, a process called transmuxing.

File Header 01010101010100101010101010

Audio/Video Content © Jan Ozer 2014

13

MEET THE MOST RELEVANT CONTAINER FORMATS Do you H.264 encoding parameters change for each container format?  The short answer is no. You set H.264 encoding parameters identically irrespective of the target container format.  This table contains the file extensions of most the container formats you’ll need when producing for streaming. With the exception of the .m4a format, which is audio only, you can deploy H.264 video in all the container formats shown.

Container

Origin

.mp4

MPEG-4 file specification

.mov

QuickTime file format

.f4v/.flv/ f4f/f4m

Flash file format

.m4v

Video for iTunes, similar to MP4 but can contain DRM

.m4a

Audio only, MPEG-4 file specification

.mpg

MPEG-2 program stream

.ts

MPEG-2 transport stream

© Jan Ozer 2014

14

WHICH CONTAINER FORMAT SHOULD YOU USE? When producing for streaming or device playback, the MP4 container format is almost always the best choice  When producing for streaming or device playback, or for uploading to a user generated site like YouTube, choose the H.264 format in the Adobe Media Encoder, which uses the MP4 container format that you’ll learn about in a future lesson. This container format is universally accepted by virtually all target players and systems.  Note that Adobe is removing the F4V and FLV formats from Adobe Media Encoder in the next major release of the Creative Cloud. This reflects the reality that the MP4 container format is simply a better option, even when streaming to the Flash Player. © Jan Ozer 2014

15

PROGRAM AND ELEMENTARY STREAMS While on the topic of container formats, let’s cover these  Let’s quickly cover a couple of related concepts. When you hear the term program stream, it means a multiplexed file that contains both audio and video components. Or, you can produce elementary streams that separately contain either audio or video.  You’ll see this distinction most frequently when producing files for DVD and Blu-ray, as shown on the right. Which option is best? This depends upon your authoring program. Adobe Encore can input either program or elementary streams, but products like Apple’s DVD Studio Pro can only input elementary streams.

muxed .mpg file

.m2a – audio .m2v – video © Jan Ozer 2014

16

WHAT’S A TRANSPORT STREAM? A container format with error correction  What’s a transport stream? According to Wikipedia, “a transport stream specifies a container format encapsulating packetized elementary streams, with error correction and stream synchronization features for maintaining transmission integrity when the signal is degraded.”  The MPEG-2 transport stream (.ts) is the format used by Apple for delivering HTTP Live Streaming (HLS) technology to Apple devices and compatible players. Adobe Media Encoder can’t output files in the MPEG-2 transport stream format. To produce files for HLS delivery, you’ll need to output MP4 files and perform the MPEG > .ts conversion in a separate tool.

HTTP Live Streaming to Apple Devices uses the MPEG-2 transport stream format, which Adobe Media Encoder can’t produce

© Jan Ozer 2014

17

SECTION END Summary •  Regarding Compression: • 

Compression shrinks audio/video files to necessary sizes

• 

The more you compress, the more quality you lose; Your job is to adjust file configurations to minimize “the loss”

• 

Choose the codec based upon where your video will be watched

•  Regarding Container formats: • 

Container formats describe how data is packed and how metadata is stored

• 

Though H.264 playback is near universal, you’ll have to choose the right container format for your video to play on the target device

•  Next, you’ll learn the basic file parameters configured when compressing a file © © Jan Ozer. Ozer 2014

18

ADOBE MEDIA ENCODER

Choosing Basic Video Settings As shown above, every time you encode a file, you have multiple configuration options to select, including resolution, frame rate, data rate, TV standard, aspect ratio, frame type (progressive or interlaced). You’ll learn (or be refreshed on) all of these in this and future sections. © © Jan Ozer. Ozer 2014

19

CHOOSING BASIC FILE PARAMETERS Overview  Adobe Media Encoder is a fabulous tool for producing encoded files. Like any tool, however, you have to know the basics to use it effectively. Most video producers have used terms like resolution, data rate, aspect ratio and bandwidth, and have a pretty decent notion of what they mean. Understanding how they inter-relate to determine the quality of a compressed file may be a different matter. Take a few moments to scan through this document to refresh your memory on these topics, and it will make all future lessons go a lot more smoothly.

Agenda 1.  Resolution/frame size 2.  Aspect ratio 3.  Frame rate 4.  Field order 5.  TV standard 6.  Data rate 7.  Bandwidth

© Jan Ozer 2014

20

RESOLUTION Simply stated, resolution is the number of pixels in the file.  This file has a resolution of 1280 pixels wide, 720 pixels high, or 1280x720. This resolution is often called 720p.  You can ascertain the size of most video files by loading them into QuickTime and opening the Inspector or by using a different video analysis tool like MediaInfo.

© Jan Ozer 2014

21

RESOLUTION THROUGH THE PRODUCTION CYCLE Acquisition: We shoot video in resolutions ranging from 720x280 up to 4K. Distribution: Resolutions depend upon the target playback platform (detailed in later lessons)

Optical Discs: Video encoded for DVD must be no larger than 720x480; Blu-ray maxes out at 1920x1080.

Devices: Some older iPods can’t play video larger than 320x240; newer devices extend to 1080p.

Web video: Varies according to factors like affordability (bandwidth costs $$$) and deliverability (for viewers on mobile connections). The resolution of streaming video ranges from 320x240 to 4K.

© Jan Ozer 2014

22

THE IMPORTANCE OF RESOLUTION The larger the resolution, the more date to compress. 720p has four times more pixels than 640x360, and four times the raw data.

© Jan Ozer 2014

23

THE IMPORTANCE OF RESOLUTION When encoding a 720p file, as compared to 640x360, you must apply 4 times the compression to achieve the same bitrate (~500 kbps in both cases). This additional compression degrades the quality of the larger file.

© Jan Ozer 2014

24

THE BOTTOM LINE ON RESOLUTION Consider multiple factors when choosing file resolution •  Requirements of the target – exceed the maximum resolution supported by the target and the files won’t load or play.

•  Quality – when encoding at a fixed data rate, dropping resolution will increase quality. Never consider resolution without considering data rate, because their dual impact on quality is directly related.

•  Source resolution – Typically, you shouldn’t scale video to higher resolutions for distribution; distribute at source resolution or smaller. Individual recipes presented in Section 2 detail the resolutions appropriate for various delivery targets. © Jan Ozer 2014

25

ASPECT RATIO The Aspect Ratio is the ratio of the frame width to frame height.

Letter boxes

Distortion

 You know you have an aspect-ratio related issue when you have black bars in your video, or the video looks stretched. Both issues should be obvious in the Export Settings dialog, and you should resolve them before encoding. You can learn how in the lesson in the Adobe Media Encoder Udemy course on Cropping, Scaling and Avoiding Black Bars and Aspect-Ratio related problems.

© Jan Ozer 2014

26

THE SHORT ANSWER ON ASPECT RATIO The correct Aspect configuration depends upon your target Non-disc playback - When producing for streaming, computer-based playback or any nondisc format, follow two simple rules. Your output resolution should match the aspect ratio of your source video. • 

If 4:3 (DV), output at 640x480, 480x360, 320x240

• 

If 16:9 (DV, all HD formats), output at 1920x1080, 1280x720, 640x360

The Aspect should always be Square Pixels (1.0), as shown on the right.

DVD/Blu-ray – The rules are different for Bluray and DVD, and are presented in the individual recipes for these media types above. © Jan Ozer 2014

27

FRAME RATE In most cases, output frame rate should match the source frame rate •  Back story – in the early days of streaming, many producers dropped their frame rate to 15 fps to maximize frame quality. •  Today, when producing full quality files – either as a single file, or the high-quality iterations within an adaptive group, always use source frame rate. In the images on the right, the video was shot and encoded at 30 fps. •  Low bitrate versions in adaptive group – When producing multiple files from the same source for adaptive streaming, most producers drop the frame rate of some of the lower bitrate files to maintain quality. More on this in the section on adaptive streaming. © Jan Ozer 2014

28

FIELD ORDER When producing for web delivery, always use progressive output •  Producing streaming output – When working with interlaced source, note that all streaming output formats require progressive output, which you achieve by choosing Progressive in the Field Order menu. •  Disc formats – You can output DVD/Blu-ray in either progressive or interlaced; more on these options in their respective lessons.

DV and most SD formats is Lower First. HDV and most HD formats are Upper First.

•  Intermediate files or camera formats – When producing intermediate files for further editing, or camera formats for archiving or other uses, you should preserve the interlaced source video. © Jan Ozer 2014

29

TV STANDARD NTSC is the US and Japan; PAL most everywhere else

NTSC: US and Japan

PAL: Rest of world

© Jan Ozer 2014

30

DATA RATE Data rate is the single most important determinant of quality and deliverability

Video bitrate (.8 Mbps, or 800 kbps)

Audio bitrate (128 kbps)

Combined - 928 kbps  Data rate is the amount of audio and video data per second in the file. In the file above, the video data rate is .8 Mbps, or 800 kbps, while audio is 128 kbps for a combined rate of 928 kbps.

 At a constant resolution, a higher data rate means better quality. It also means a higher delivery cost, since higher data rate files are more expensive to transmit.

 Data rate is critical because it (along with resolution) directly controls quality. All video compression technologies are “lossy,” so the more you compress, the more quality you lose.

mediums (cellular, broadband, etc) can retrieve data at different speeds. Overall, data rate is probably the most important decision you make about every file you produce.

 Data rate impacts file deliverability as well, since different

© Jan Ozer 2014

31

WHAT DETERMINES DATA RATE?

Web video: Data rate policies vary by publisher and target. When delivering to broadband viewers, broadcasters like CNN consider bandwidth costs in their data rate (and resolution) decision, because it’s a huge cost factor. Publishers like Apple, who distribute far viewer streams, use much higher data rates and are more concerned with their viewer’s ability to retrieve the streams (more next section).

Optical Discs: Both Blu-ray and DVD have maximum data rates.

Devices: All devices also have maximum data rates.

When delivering to mobile, the data rate decision is all about deliverability, compressing to data rates low enough for cellular delivery.

© Jan Ozer 2014

32

BANDWIDTH The size of the pipe between your video file and the viewer

Size of the smallest pipe between Your video and your viewer Server Viewer  The viewer’s bandwidth determines your ability to deliver video in real time. If the file data rate exceeds viewer bandwidth, the stream stops, which is never good.  Today we have at least two pipes to consider when we produce our files, broadband and mobile.

 As you’ll see in the next slide, broadband bandwidth is quite high today, enabling fabulous quality.  However, as more viewers watch on mobile devices, cellular bandwidth, which trails broadband considerably, is a huge limiting factor for high-quality delivery. © Jan Ozer 2014

33

BROADBAND Average connection speed per Akamai State of the Internet*: CNN streams their broadband files at about 1200 kbps in the US. However, as you can see below, the average connection speed in the US is more than 7 times higher. Average connection speed*: •  US – 8,618 kbps •  Canada – 7,839 kbps •  UK - 7,855 kbps •  France - 5,125 kbps

 Why does CNN stream at 1200 kbps? Because bandwidth is a very significant expense, and they’ve obviously decided that the 640x360@1200 kbps configuration they use delivers the optimum blend of quality and affordability.  Of course, streaming is only one medium for video delivery. The data rate decision changes by medium, and we address this issue for streaming, optical discs, uploading to YouTube and other mediums in specific lessons.

*http://www.akamai.com/stateoftheinternet/

© Jan Ozer 2014

34

SECTION END Summary

•  Defined multiple terms and how they interrelate: •  Resolution, aspect ratio, frame rate, field order, TV standard, data rate and bandwidth •  Most further defined in recipe sections

•  Next, you’re learn about bitrate control techniques like constant bitrate encoding (CBR) and variable bitrate encoding (VBR)

© © Jan Ozer. Ozer 2014

35

ADOBE MEDIA ENCODER

Choosing Bitrate Encoding Settings In this lesson you'll learn the difference between Constant Bit Rate encoding (CBR) and Variable Bitrate Encoding (VBR). You'll also learn how to optimize quality when using both techniques, and most importantly, when to use each technique. You'll also learn about Constant Quality (CQ) or Constant Rate Factor (CRF) which is a useful bitrate control that some encoders offer for archiving and similar purposes. © © Jan Ozer. Ozer 2014

36

CHOOSING BITRATE ENCODING SETTINGS Overview

 Every time you encode a file in the Adobe Media Encoder, you’ll have to choose between two techniques for controlling the bitrate; constant bitrate encoding, or CBR, and variable bitrate encoding, or VBR. In this lesson you’ll learn what they are, when to use them and how to use them.  Even though Adobe Media Encoder doesn’t support Constant Quality/Constant Rate Factor (CQ/CRF) encoding, I touch on that briefly at the end of the lesson so you’ll know what it is.

Agenda 1.  VBR and CBR defined 2.  How much quality difference? 3.  When to use CBR and VBR 4.  Producing CBR 5.  Producing VBR 6.  Constant Quality/Constant Rate Factor

© Jan Ozer 2014

37

VBR AND CBR DEFINED VBR

800 kbps

CBR

500 kbps 300 kbps

Low Motion

Moderate Motion

Low Motion

Moderate Motion

High Motion

 Constant bitrate encoding (the red dotted line)

 Variable bitrate encoding (the black line)

 Mechanically applies a uniform data rate to each scene in the file, even though they represent varying encoding complexity. This sacrifices quality in some scenes for encoding simplicity and stream regularity.

 Varies data rate according to stream complexity. This optimizes quality over all the scenes in the video file, but takes longer to encode (two passes) and sacrifices stream consistency, a problem in some delivery situations. © Jan Ozer 2014

38

BITRATE CONTROL TECHNIQUES In a Nutshell  Constant Bitrate Encoding (CBR)  Description: • 

Applies one bitrate over entire video; irrespective of content

 Pros: •  • 

Computationally simple Can produce in a single pass (which is obviously necessary for live)

 Cons: • 

Doesn’t optimize quality in files with different scenes with varying encoding complexity

 Variable Bitrate Encoding (VBR)  Description: • 

Varies bitrate to match encoding complexity

 Pros: • 

Optimizes quality in files with different scenes with varying encoding complexity

 Cons: •  • 

Needs multiple passes Introduces stream variability that could interrupt delivery over fixed bitrate connections (e.g. cellular)

© Jan Ozer 2014

39

WHAT CONSTANT BITRATE ENCODING REALLY LOOKS LIKE

Average data rate

 This is what a CBR file actually looks like using a video analysis tool called Bitrate Viewer. The faint blue line is the average data rate, which stays pretty constant over the duration of the file. The spikes are individual frames, which do vary significantly.

 This video has low motion, easy to compress scenes 1 and 2, and higher motion scenes that are harder to compress, like 2, 4 and 5. You see that CBR doesn’t take this into account. As you’ll see, in some instances, this does cause suboptimal quality in hard to encode scenes.

© Jan Ozer 2014

40

WHAT VARIABLE BITRATE ENCODING REALLY LOOKS LIKE

Average data rate

 The same file encoded with VBR. The data rate is low during the initial talking head sequence, then increases significantly for the higher motion shot of a woman making a pita.

 The final two scenes, a fiddler in a street festival and a walk with the camera on my chest, are both high motion, and the data rate responds.

 How much quality difference does this make? In  The chip-grinding machine in the middle is also this file, not so much, but in another file you’ll soon see, the difference is very significant. low motion, and the data rate again drops. © Jan Ozer 2014

41

HOW VBR COMPARES TO CBR OVERALL The average data rate (and file size) should be very similar

Constant Bitrate 600 kbps target  These screen grabs are from the upper right side of the two previous slides. Note the average bitrate for the CBR file on the left was 603 kbps, while it was 596 kbps for the VBR on the right. This makes the point that over the duration of the file, both techniques will deliver very close to the same data rate.

Variable Bitrate 600 kbps target  The VBR file will have more stream variability, as evidenced by the Peak bit rate of 1255 as compared to 776 for the CBR file. But both techniques should deliver the target bit rate specified in Adobe Media Encoder.

© Jan Ozer 2014

42

WHAT’S THE MAXIMUM QUALITY DIFFERENCE TO EXPECT?

Constant Bitrate  This frame is immediately after a camera switch, so even though it looks like a low motion sequence, it’s really high motion. CBR on the left looks ugly because the frame is 1KB as compared to 7KB for the VBR frame on the right (this analysis tool is called Inlet Semaphore and it’s no longer available).

Variable Bitrate  This is the most difference I’ve ever seen between CBR and VBR, and should certainly not be considered representative. If anything, 99% of the time I’m shocked at how little difference there is between files encoded using the two techniques. Still, you can easily see why VBR is preferred when circumstances don’t absolutely call for CBR. © Jan Ozer 2014

43

CREATING TOP QUALITY CBR Very simple with AME; choose target data rate, choose CBR

 Some encoding tools enable 2-pass CBR, which can improve quality over 1-pass CBR. However, Adobe Media Encoder doesn’t and offers only 1-pass CBR.

Choose CBR

 Accordingly, to use CBR, choose it from the Bitrate Encoding menu and insert the target data rate in the next field. Insert target bitrate

© Jan Ozer 2014

44

CREATING TOP QUALITY VBR CREATION – H.264 Choose VBR, 2 pass; insert target and maximum bitrates

Choose VBR, 2 pass

 Adobe Media Encoder offers both 1-pass and 2-pass VBR. When producing for streaming, use 2-pass VBR so you can set a maximum bitrate, which is called constrained VBR. Some producers use 1-pass VBR for archiving, or uploading to a UGC site, but I recommend 2-pass VBR for those applications as well.  When producing a single H.264 file for streaming over broadband, the maximum bitrate should be 200% of the target. So in the figure, the Target is .8 Mbps, and the Maximum is 1.6 Mpbs.  When distributing over cellular, or as part of an adaptive group, consider constraining the stream to 125-150% of the target. This gives the encoder some wiggle room to optimize quality, but also limits stream variability, promoting smooth delivery.

Insert target bitrate

Insert max bitrate © Jan Ozer 2014

45

CREATING TOP QUALITY VBR CREATION – OPTICAL DISC Choose VBR, 2 pass; insert minimum, target and maximum bitrates Choose VBR, 2 pass  When producing for DVD and Blu-ray, Adobe Media Encoder also lets you choose a minimum bitrate as well as target and maximum. My rule of thumb here is to set the minimum between 50%-80% of the target.  When encoding for optical disc playback, I set the maximum comfortably below the maximum throughput of the playback system. For DVD, the maximum playback speed is 9 Mbps, but I never exceed 7.5 Mbps, because DVD-recordable technology starts to fail on older players once you exceed that rate.  The maximum rate for Blu-ray players is 40 Mbps, but I never exceed 35 Mbps for the same reason. Note that I cover these details again in the DVD and Blu-ray lessons.

Insert min bitrate

Insert target bitrate

Insert max bitrate

© Jan Ozer 2014

46

WHEN TO USE VBR AND CBR – ONLINE APPLICATIONS Application Single file streaming – broadband Single file streaming – mobile

Technique 200 % Constrained VBR Constrained VBR (125-150%) or CBR

Progressive delivery

200 % Constrained VBR

Uploading to YouTube or Online Video Platform

200% Constrained VBR

Live

CBR

Adaptive

Highly constrained VBR (125-150%) or CBR

Archival

VBR

 Here are the general rules I live by when producing for online delivery. More on why I recommend constrained VBR for adaptive delivery in that lesson.

© Jan Ozer 2014

47

WHEN TO USE VBR AND CBR – ONLINE APPLICATIONS

Application

Technique

DVD

Constrained VBR – to 7.5 Mbps

Blu-ray

Constrained VBR – to 35 Mbps

Devices

Constrained to just below maximum supported data rate

 Here are the general rules I live by when producing for optical disc, or for tethered delivery to devices, as with iTunes. More on these practices in their respective lessons.

© Jan Ozer 2014

48

CONSTANT QUALITY/CONSTANT RATE FACTOR Not available in AME, but a technique you should know about  The Constant Quality (CQ) bitrate control technique lets you set a target quality (called the Constant Rate Factor, or CRF) and varies the data rate of the file as needed to deliver that quality level. While useless for streaming and similar applications, CQ is very useful for applications that require a guaranteed quality level at the lowest possible file size, like archiving or encoding for iBooks Author.

Choose CQ

 Adobe Media Encoder doesn’t offer CQ because Adobe uses the MainConcept H.264 codec, which doesn’t offer this mode. Most tools that support the x.264 codec do offer this mode, including Sorenson Squeeze (shown) and HandBrake, a free, open-source encoder.  Using CQ is simple; just select the mode and the CRF value. Of course, you’ll have to experiment with several encodes to figure out the correct CRF factor, which with x.264, can range from 0, which delivers the highest quality, to 51. In my work with iBooks Author, I found that values between 20-25 delivered the optimal blend of quality and file size.

Choose CRF value © Jan Ozer 2014

49

SECTION END Summary

•  In this lesson, you learned: •  •  •  • 

What CBR/VBR are When to use them How to use them What CQ is and how it works

•  Next, you’ll learn about frame types like I-frame (also called key frames), B-frame and P-frames

© © Jan Ozer. Ozer 2014

50

ADOBE MEDIA ENCODER

Frame Types Used for H.264 and MPEG-2 When encoding using the H.264 and MPEG-2 codecs, you’ll have to make decisions about which key frame intervals to use (for both codecs) and the B-frame interval (for MPEG-2). This lesson discusses what these frame types are and how to configure them.

© © Jan Ozer. Ozer 2014

51

FRAME TYPES USED FOR H.264 AND MPEG-2 Overview

Agenda  You can’t produce high-quality video without knowing how to configure I-frames (called key frames in the Adobe Media Encoder) and Bframes.  Adobe Media Encoder doesn’t provide a lot of options regarding these frame types, but you’ll learn which options it does provide and how to configure them in this lesson.

1.  Defining the frame types 2.  Choosing an I-frame interval 3.  Configuring I-frame parameters 4.  Working with B- and P-frames

© Jan Ozer 2014

52

DEFINING THE FRAME TYPES

Image from http://www.dallmeier.com/

 The figure above shows a “group of pictures,” or GOP, •  which starts with an I-frame and continues to the last frame before the next I-frame, in this case P-frame number 8. So called “long-GOP formats,” like H.264 and •  MPEG-2, use these three frame types during encoding. •   The frame types are differentiated based upon their abilities to search for redundancies in other frames. Note that B-frames are only allowed with H.264 when encoding using the Main and High Profiles.

I-frames - encoded without reference to other frames (also called key frames). Very much like JPEG files. P-frames - look backward for redundancies to I- and P- frames (P stands for predictive) B-frames - look backwards and forwards for redundancies from I and P frames (B stands for bi-directional interpolated)

© Jan Ozer 2014

53

WHAT ARE THEY SEARCHING FOR? It’s all about redundancy

The background doesn’t change

 Talking head videos compress most efficiently because most of the frame doesn’t change. Bframes and P-frames can “borrow” this redundant data from other frames, which is much more efficient than encoding actual video data.  Again, I-frames don’t search at all, and stand alone. This makes them the least efficient frame type from a compression perspective.  P-frames can utilize redundant information from previous I-frames or P-frames.  B-frames can look both ways, borrowing redundant data from I-frames, B-frames and Pframes before and after the frame being encoded. This makes B-frames the most efficient frames in the GOP.

Only the head is moving

© Jan Ozer 2014

54

APPLICATION: WHEN SHOULD YOU ADJUST THESE SETTINGS?  Adobe Media Encoder enables frame-related adjustments in two general places.  H.264 encoding – When producing H.264 files, you’ll adjust the key frame distance (I-frame) depending upon when you’re encoding for single files or for an adaptive group. More on this in the following two slides.  MPEG-2 encoding – When producing MPEG-2 files for DVDs or Blu-ray discs, you can adjust N values (keyframe distance) and M values (number of consecutive B-frames). I recommend always using the setting in the Adobe presets as is because if you adjust these settings, you may produce a file incompatible with your DVD authoring program.

In H.264 Encoding Controls

In MPEG-2 for DVD preset

© Jan Ozer 2014

55

CONFIGURING I-FRAMES FOR SINGLE FILES STREAMING

Has to seek back to here

 I-Frames are the least efficient compressionwise, so you want as few as possible in your encoded streams. That said, all playback has to start on an I-frame since that’s the only frame type with all the information necessary to decode the frame. In the figure, to play the B-frame on the right, the player would have to seek back to the previous I-frame and scan forward from there. This might take too long if your file only had a single key frame.

To play this frame

 For this reason, when producing single files for streaming, you should insert one key frame every ten seconds or so. In a 30 fps file, this would be a key frame every 300 frames.

© Jan Ozer 2014

56

OTHER I-FRAME OPTIONS Insert I-frames at scene changes  Some encoding tools allow you to choose to insert I-frames at scene changes, which improves quality at scene changes by ensuring a high-quality frame as a starting point for all subsequent B- and P-frames. On the top right, Telestream Episode calls these “natural” keyframes, while on the bottom right, Sorenson Squeeze just tells it like it is.  Adobe Media Encoder inserts keyframes at scene changes automatically, which is great when encoding a single file. However, because you can’t disable the insertion of key frames at scene changes, this may cause problems when encoding for adaptive streaming. More on the next slide.

Telestream Episode

Sorenson Squeeze

© Jan Ozer 2014

57

CONFIGURING I-FRAMES FOR ADAPTIVE GROUPS Alternative 1

I

I

I

I

Alternative 2

I

I

I

I

Alternative 3

I

I

I

I

 When you’re producing multiple files for adaptive streaming, all stream switches must occur on an Iframe. For this reason, you should use the same I-frame interval for all files in the adaptive group. To allow the server to quickly adapt to changing conditions, most producers use a key frame interval of 3 seconds for adaptive groups, or 90 frames for 30 fps footage.  As you’ll learn in the lesson about adaptive streaming, the Adobe Media Encoder can only produce MP4 files for RTMP-based adaptive streaming, it can’t produce the segmented formats required for HTTP-based technologies, though you can convert MP4 files produced by the Adobe Media Encoder into segmented files with third party tools.

 As discussed on the preceding slide, the Adobe Media Encoder doesn’t let you disable key frames at scene changes. With RTMP streaming, the server should be able to use the files produced by the Adobe Media Encoder, even if the key frame interval is irregular due to those inserted keyframes.  However, these files could cause problems when segmented using a third party tool, because the irregular key frames could result in a segment which doesn’t start with a key frame, which is a requirement for all HTTPbased adaptive technologies. Though I haven’t personally experienced this problem, or heard about it from others, I’d be concerned using MP4 files encoded with the Adobe Media Encoder with a third party tool.

© Jan Ozer 2014

58

WHAT ABOUT B-FRAMES?

Typical B-frame-related configuration options from Telestream Episode

 The Adobe Media Encoder doesn’t provide any Bframe configuration options for H.264 encoding. Encoders that do, typically provide the options shown above, the ability to specify the number of consecutive B-frames, and the number of reference frames, or the frames a B- or P-frame can search for redundant information.  However, just because the Adobe Media Encoder doesn’t provide controls for these options, doesn’t

 mean that it doesn’t insert B-frames into the encoded streams.  Rather, when you encode using the High Profile, 2 consecutive B-frames are inserted between Iand P-frames. When you encode using the Main Profile, 3 are inserted. Since B-frames aren’t supported in the Baseline mode, none are inserted in these files. In all instances, Adobe Media Encoder searches 3 reference frames.

© Jan Ozer 2014

59

LESSON END Summary

•  In this lesson, you learned: •  What I-frame, B-frame and P-frames are and how they operate •  That you shouldn’t mess with these parameters when encoding for optical discs •  How to choose an I-Frame interval for single file and adaptive streaming •  How Adobe Media Encoder deploys B-frames and reference frames

© © Jan Ozer. Ozer 2014

60

ADOBE MEDIA ENCODER

Streaming Delivery Concepts In this lesson, you’ll learn the techniques that can be used to deliver on-demand video over the web and how your mode of delivery impacts the encoding configuration.

© © Jan Ozer. Ozer 2014

61

STREAMING DELIVERY CONCEPTS Overview

 When you’re delivering video over the web, you have multiple options. In some instances, the technique you use to deliver the video controls the encoding parameters you’ll use to encode the file.  For example, if you’re encoding a file that you’ll deliver to a buddy via a USB thumb drive, you can encode a high quality file at a very generous data rate. Deliver the same file streaming over a cellular connection and you’ll have to configure the file much, much differently.  In this lesson, you’ll learn the techniques that can be used to deliver video over the web and their implications regarding the encoding configuration.

Agenda 1.  Download and play 2.  Progressive download 3.  Streaming 4.  Adaptive streaming 5.  How encoding is different

© Jan Ozer 2014

62

DOWNLOAD AND PLAY How it all began

 1. Viewer clicks link on website  2. File starts downloading  3. Playback doesn’t start until file fully downloaded

 Download and play was the first technique used to deliver video over the Internet. Since the modems used at the time were very slow, and compression technologies very basic, it took a long time to download a file and view a file.  Any web server can deliver video via this technique; you don’t need a streaming server. Basically, the web server delivers the video file just like any other form of content you can download from a website, like text, images, and PDF files.  Today, very few websites deliver video via download and play.

© Jan Ozer 2014

63

PROGRESSIVE DOWNLOAD The first major innovation, but wasteful of bandwidth

 1. Viewer clicks link on website  2. File starts downloading as quickly as possible and is stored on viewer’s hard drive  3. After a short period of buffering, the file starts to play during the download

 Apple debuted progressive download for QuickTime movie trailers. Since the file is stored on the viewer’s hard drive (step 2), even if you had to wait to play the file, playback was perfect after any download delay, and you could try to start watching almost right away. Local storage does make the file easier to pirate, however.  As with download and play, you don’t need a web server to deliver video via progressive download. As you’ll see on the next slide, because web servers deliver video and other data as quickly as possible, the major disadvantage of progressive download is that it wastes bandwidth. Still, progressive download is the predominant technique used for web delivery today.

© Jan Ozer 2014

64

PROGRESSIVE DOWNLOAD

To see a video of this operation, navigate to bit.ly/streamvpd

The first major innovation, but wasteful of bandwidth

1) Here, I’m watching a video file from Vimeo, which delivers via progressive download. Note that I’m only 59 seconds into the playback of this 8:29 minute file.

2) This tool, called Jaksta, saves the file as Vimeo delivers it. Because web servers deliver the file as quickly as possible, the file is completely downloaded (52.5 MB worth, see below), even though I’m only about 10% of the way through actually playing the file. If I stop playback, Vimeo delivered 90% of the file for no reason, but still had to pay the cost of that delivery. That’s why delivering via progressive download can be wasteful.

© Jan Ozer 2014

65

STREAMING DELIVERY Streaming is bandwidth efficient, but requires a streaming server

 1. Viewer clicks link on website  2. Streaming server starts delivery

 Streaming was debuted by RealNetworks and later copied by Microsoft, Apple and Adobe. True streaming requires a streaming server, or a software program usually running on a workstation separate from the web server. The streaming server’s job is to dole out the video as its played. If a viewer stops watching, no more video is sent, so less bandwidth is wasted.

 3. After a short buffer period, play starts

 Because the streaming server distributes video as needed, rather than as quickly as possible, a streaming server can efficiently serve more viewers with a superior quality of service than a web server delivering via progressive download.

 4. Streaming server doles out video as needed to support playback

 Files delivered via streaming are usually not cached locally, so are more secure. Because they’re not stored on disk, however, if the data rate of the file is too high, the viewer may never be able to play it smoothly.

© Jan Ozer 2014

66

STREAMING DELIVERY

To see a video of this operation, navigate to bit.ly/streamvpd

More bandwidth efficient and a better quality of service for more viewers than progressive download

2) Here’s Jaksta again; as you can see, I’m 1) Here I’m watching a video on

only 22 seconds into the download. If I stop watching now, very little bandwidth is wasted.

Bloomberg, which uses a Flash Media server to stream their video. I’m 24 seconds in.

© Jan Ozer 2014

67

ADAPTIVE STREAMING Create multiple files and customize delivery for bandwidth and device capabilities

 Adaptive streaming technologies produce multiple files at different data rates and resolutions and deliver them depending upon connection bandwidth and playback capabilities of the viewer. Files are switched during playback to adjust to changing conditions.

 The first adaptive streaming technologies required a streaming server, which cut off delivery if a viewer stopped watching, which is bandwidth efficient. Later technologies rely on the player to retrieve video data as needed, so they are just as bandwidth efficient and don’t need a streaming server. © Jan Ozer 2014

68

ANALYSIS  Progressive download •  Still the least expensive alternative •  Can deliver to desktops and mobile with a single file via HTML5, though features are limited (no captions, digital right management, etc). •  Used by the vast majority of smaller sites on the Internet where wasted bandwidth isn’t meaningful

 Streaming

 Adaptive Streaming

•  Most sites using streaming servers have switched to adaptive streaming

•  Adaptive is the preferred technique for delivering to mobile and desktop viewers •  Virtually all broadcast sites have switched to adaptive •  Delivery to all platforms is complicated, particularly Android •  Adaptive streaming technologies enable features like closed captions, digital rights management and DVR operation © Jan Ozer 2014

69

HOW THIS IMPACTS ENCODING  Progressive download •  It’s about quality

 Streaming

•  It’s about deliverability

•  Long key frame interval, key •  frames at scene changes •  •  Variable bit rate encoding (VBR) •  Since file is cached, data rate can be higher than a streaming file, because ultimately it will play smoothly

 Adaptive Streaming •  It’s about switching smoothly

CBR and VBR Long key frame interval, key frames at scene changes

•  Since file is typically not cached, producers have to use a lower data rate that ensures smooth delivery and playback

•  Short key frame interval so switching can be responsive •  Key frame interval identical for all files, no key frames at scene changes •  CBR to avoid unnecessary stream switches •  Identical audio streams to minimize popping © Jan Ozer 2014

70

SECTION END Summary

•  You learned the primary techniques for delivering video over the Internet, and how to customize your encoding parameters for each technique

© © Jan Ozer. Ozer 2014

71

THAT’S IT – HERE ARE SOME ADDITIONAL LEARNING RESOURCES Video Compression for Web, Disc and PC/TV/Console Player A general-purpose compression course  This course is designed for two types of users. If you're new to compression, you will learn how to produce high quality video for distributing over the web, watching on your computer or smart TV/console, for inserting into iBooks or to record onto DVDs and Blu-ray discs.  If you know the basics of compression, this course will serve as a valuable reference for questions that inevitably arise when encoding video, like what's the best key frame setting for web video (Lesson 5), how do I work with x264 (Lesson 17) or how do I implement HTML5 (Lessons 31 & 33).  This course includes all the components of the Adobe Media Encoder course discussed next.

 Course details: § 

7+ hours of lessons

§ 

Retail: $59

§ 

Special offer; $29 with discount code AME_Book (navigate to bit.ly/AME_book for more information and to purchase using the discount code) © Jan Ozer 2014

72

LEARNING RESOURCES Mastering the Adobe Media Encoder - An Online Course An encoder-specific compression course  Working efficiently with any encoder requires two levels of knowledge; First, you have to know how to use the program. Second, you have to know the requirements of your target platforms to ensure that your files serve their intended use on those platforms.  In this course, you'll learn both. First, you'll learn how to use the Adobe Media Encoder as efficiently as possible, with multiple tips and tricks to maximize encoding quality and performance. And you'll learn the requirements of multiple target platforms, from general-purpose streaming, mobile playback and uploading to YouTube, to disc-based productions like Blu-ray and DVD.  You'll learn that in some cases, the presets included with Adobe Media Encoder are spot on. In others, they need some adjusting to maximize quality, playback compatibility or both. In some very rare cases, you probably shouldn't use the Adobe Media Encoder to produce your files at all.

 Course details: §  §  § 

80 minutes of lessons Retail: $25 Special offer; $20 with discount code AME_Course (navigate to bit.ly/AME_Course for more information and to purchase using the discount code) © Jan Ozer 2014

73

LEARNING RESOURCES Producing Streaming Video for Multiple Screen Delivery Available in paperback and iBook  Published in 2013, Producing Streaming Video for Multiple Screen Delivery the only compression-related book released after 2010, and it incorporates the latest technologies, including DASH and HTML5 and new devices such as 4G transmitters for mobile delivery.  This book is written for producers seeking to distribute streaming video to the widest possible audience, including computers, smartphones and tablets, and Over the Top (OTT) devices. Written by Jan Ozer, this book delivers the lessons learned from years of producing and consulting on streaming, and serving as a contributing editor to the industry bible, Streaming Media Magazine.

 Book details: §  §  §  §  § 

434 pages Published April 2013 Amazon: $36.37 iBookStore - $19.99 Navigate to bit.ly/Ozer_multi for more information

© Jan Ozer 2014

74

 About the author:  Jan Ozer is a leading expert on H.264 encoding for live and ondemand production. As a contributing editor to Streaming Media Magazine, Ozer has tested most cloud, enterprise, and desktop encoding tools, worked with most online video platforms (OVPs) and live streaming services, and many webcast platforms. Skilled in video production and editing, Ozer also produces live events for streaming or on-demand and disc-based distribution. Ozer consults widely on live and on-demand streaming and encodingrelated topics. Ozer is a frequent speaker on streaming and video production-related topics at industry events, including Streaming Media conferences, NAB, and other conferences worldwide. Ozer's books, Producing Streaming Video for Multiple Screen Delivery, and Video Compression for Flash, Apple Devices and HTML5, have consistently garnered five-star reviews on Amazon and have been adopted as textbooks by multiple colleges and universities. Ozer produces screencam and hybrid video/screencams for training and product marketing for companies like Adobe, NewTek, Matrox, Vislink, DVEO, Winnov, Telestream and VideoGuys. In the Virginia region, Ozer shoots concerts and other events for live or on-demand streaming and for distribution via Blu-ray and DVD.

© © Jan Ozer. Ozer 2014

75

CONTACT INFORMATION

 To contact Jan Ozer:

www.streaminglearningcenter.com [email protected] 276-238-9135

Twitter: @janozer

  LinkedIn: Jan Ozer

© Jan Ozer 2014

76

Thank You

© © Jan Jan Ozer. Ozer 2014

77