CFAT.2014.08

Intro to Video + Audio

 In

This Class

Themes to cover.

● Description of codecs + container formats ● When to use what, including web, SD, HD, 2k and beyond resolutions, comparisons  ● Constraints, strengths and options of cameras

● File + Project Management

Containers

Containers Description and definitions

● File types usually defined/identified by extension ● Not limited video, capable of housing several different types of data: ○ Video ○ Audio ○ Still Images ○ Metadata ● Often contain data which is compressed (codecs) ● Also called "wrapper": Possible to extract video from one container and insert it into another (without changing the contained data): "rewrapping"

Containers Questions to Ask: Why to Use What

1. Adoption: How likely is it that other users will be able to open the container you are using? 2. Codecs: Does the container support the codec you wish to use in the way you intend? 3. Efficiency: How many system resources will the container use to play back?  Does the playback medium have hardware which is specialized in playing back the media? 4. Extras: Does the format have expanded features that you wish to use?

Containers Types - MOV

.MOV: Quicktime, Apple Format, Native to Quicktime player and all Apple media infrastructure (iMovie, Final Cut, etc) ● Multitrack format.  Each track may contain audio, video, effects, or text. ● Each track contains discreet codec IE video and audio may have different codecs ● Instead of codec/media information track might contain a reference to other media ● Can use the same MPEG-4 codecs as MP4. (but many, many more)  

Containers Types - MP4

.MP4 (MPEG-4 part 14):  ● Official container format for MPEG-4 as defined by the MPEG working group.  ● Interchangeable with MOV in some cases. ● .m4a introduced by Apple for aac/alac (audio only) ○ .m4a Can be renamed to .mp4 with no ill effects

Used on YouTube for most HD and some SD content (boo FLV!) Also used on Vimeo and many other video playback sites.

Containers Types - AVI

AVI (Audio Video Interleave): MS Format ● No Standard Timecode ● New compression techniques (MPEG-4) not accounted for, some techniques not supported ● Variable frame rate not supported ● High degree of compatibility

 

Containers Types - MKV

.MKV: Matroska ● Open Standard/Open Source (LGPL and BSD licensing) ● .MKA for audio only, .MKS for subtitle files ● Limited playback support for both hardware and software (but software solutions exist for all major and many minor operating systems) ● Workaround to import into Premiere (not for editing) ● Fast seeking through file / Internet streaming ● Menus/Chapters ● Selectable Subtitles ● Pirates love it.

Containers AVC/AVCHD

AVCHD (Advanced Video Coding High Definition) ● Meant as a recording and distribution container ● Contains MPEG-4 video (is a brand name owned by Panasonic and Sony) ○ NXCAM (Sony Pro) ○ AVCHD Lite (Limited to 720p/30) ○ AVCCAM (Panasonic Pro) ● File Structure is somewhat compatible with BluRay players - some will play AVCHD DVD discs.

Containers To End.

While Editing

Use .MOV (usually) (unless you are using RAW)

Containers The end of the end.

● Apple has designed Final Cut and all other OS X media products to work most efficiently with MOV. ● Apple products such as iTunes and video capable iPods will not recognize non Quicktime formats (nor will they recognize MOVs which are not encoded with H.264 codec). (This is a distribution issue, not an editing issue.) ● In Windows .mov is a crap format for high end systems Limited to 32bit

Containers The end of the end of the end.

● Premiere has a lot more flexibility with containers - can use MOV, AVI, MXF, and unencrypted MPEG variations to name a few.

Codecs

Codecs Basics

The root of codec is code/decode.  Simply put, a codec is a computer program which is capable of encoding and decoding.  Codec can apply to many things, not just media.   Codecs fall into two main categories: lossless and lossy. A way to categorize media codecs is:  acquisition - intermediary - distribution

Codecs Basics - Lossless

● The most common example of a lossless codec is ZIP archiving.  There is no data loss, but file size is smaller ● Transporting video with zero degradation is highly desirable, especially in the post workflow phase ● Lossless video codecs exist - Animation, HuffyUV, and JPEG2000 ● System requirements limit use of these codecs. NO LOSSLESS CODECS EXIST FOR STANDARDIZED FINAL DISTRIBUTION

Codecs Basics - Lossy

● Most common and varied codecs ● Range from highly compressed with degradation to lightly compressed and visually lossless. ○ Generational loss ● Intermediary codecs often used in place of lossless codecs. ○ DV/HDV MOV container ○ Avid DNxHD ○ Apple Prores ○ AVCIntra100 (MXF)

}

Intermediary Codecs Prores: Tried and true MPEG2

● Based on MPEG2 ● Full-width (Full Raster) 1920x1080 and 1280x720 ● Intraframe Codec ● Designed to be efficient and be used in a post environment ● Multiple versions to choose from: 422 (original), 422 HQ (v2.0), 422 LT (v2.0) , 4444 (v2.0), 4444 XQ

Codecs Distribution

● Distribution formats are almost all lossy (even 35mm!) ● Major distribution formats include: ○ H.264/x.264 - Internet (H.265 will replace) ○ DivX/Xvid - Internet ○ MPEG-2 (SD and HD) - DVD/Bluray ○ VP9 - codec to watch for!

JPEG2000 in a MXF DCP

Codecs - Distribution DivX/Xvid: Piracy in 2001

● Early MPEG-4 based codec ● Good hardware adoption (many DVD players will play from disc or USB stick)  ● Largely superseded by more modern MPEG-4 codec called H.264

Codecs - Distribution H.264/AVC: Magic Codec

● A subset of MPEG-4 ● Retains excellent quality at low bitrate ● Very popular for online distribution ● Used in HDTV and BluRay (encrypted) ● Uses a wide range of profiles which define quality and features used. ● Can be Intra or Inter frame compressed ● Often doesn't hold up well over multiple generations, except for high end profiles or implementations such as AVC-Intra100 ● x.264: open source variant

Codecs - Distribution MPEG - A format and a company!

● Not just a distribution codec - High quality MPEG2 encoding is used commonly on ENG video cameras. ●  MPEG-2 was the defacto standard for home distribution via grandpa’s DVDs.

● MPEG-2 is still the standard codec for use in HD BluRay discs (MPEG-4 may also be used). ● Being replaced by MPEG4 for acquisition, editing, and distribution.

Compression

Compression Nuts + Bolts

● Uncompressed video is huge. ● Compression usually trades quality for space savings. ● Different kinds of compression result in different system requirements. ● Basically works by simplifying the image (thus quality loss)    Intraframe codecs use only spatial compression Interframe codecs use spatial AND temporal compression

Spatial Compression  Compression of a Scene

Within a scene adjoining pixels are analysed.  If they are within a threshold they are combined to be the same colour.  This means that colour information is reduced, thus space required is reduced.

Spatial Compression  Compression of an Image

Spatial Compression Formats

● DV ● AVC-Intra ● DVCPro HD ● Prores ● R3D

Formats which use only spatial compression are referred to as intraframe codecs

Temporal Compression  Compression of an Image Over Time

This compression usually follows a spatial pass.  The type of frames used in temporal compression can be divided into in three categories: ● I-frames: contain all the information about the image. Each frame is analyzed individually. ● P-frames: 'predicted picture' frames only record changes from one frame to the next. ● B-frames: 'Bi-predictive picture' frames work like P-frames, but in both directions, sharing pixels across 3 or more frames.  Come between I or P frames

Temporal Compression  Keeping it Together

GOPS   A Group of Pictures is the sequence in which I,P and B frames are assembled to make a motion video. HDV, MPEG2, low end MPEG4 (H.264, AVCHD) all use GOPS and can be difficult to edit properly because of cutting rules (can't separate specific frame types. They can also lend to image degradation in high motion due to high information changes within GOPS.

Temporal Compression

 Comparing Intraframe to Interframe compression

GOPS: Intraframe = all I frames Interframe = mixture of I B P frames  

Temporal Compression

 Comparing Intraframe to Interframe compression

I-Frame

Temporal Compression

 Comparing Intraframe to Interframe compression

P or B Frame

Intra VS. Inter  In Case You Missed It

Blackmagic = ProRes = INTRAFRAME = AWESOME   60D = h.264 = INTERFRAME = LESS AWESOME  

Bitrate

Bitrate Quality Slider

● Definition of how much information is stored per second ● Relates to spatial and temporal compression ● Often higher bitrate = better image (less compression), but it is dependent on the codec used as well. ● So why use MPEG 2 instead of always using MPEG-4? ● No simple rule ● Guidelines for distribution give good starting point: DVD, BluRay and YouTube

Bitrate Quality Slider

Bitrate guidelines:   DVD: Up to 9.8 Mb/s (9,800 Kb/s) for MPEG2 video (audio extra) BluRay: Up to 48 Mb/s (48,000 Kb/s) for MPEG2 or H.264 YouTube: Up to 7.8 Mb/s (7,800 Kb/s) for H.264 (this is a guideline only - YouTube states there is no maximum, only a maximum file size.  Your own testing is recommended)

When to Use What Formats, quality, etc.

Camera File Types

They shoot the same thing! No they don’t!

Blackmagic = ProRes = file.mov  60D = MPEG 4 = file.mov

=  

When To Use What (Staying Native)

●  Do as few conversions as possible. Every conversion involves some loss of quality. ● When in post phase only transcode to a higher quality format (minimize your losses.) ● If possible stay in the native format throughout (BlackMagic Prores). ● If you're going to output into a higher quality format other than the one you shot with, convert to that format as quickly as you can, usually during capture or digitize.  If you can't convert on capture, convert before working with the footage. ● If outputting to lower quality format, IE H.264 for web, wait until end of workflow before converting to that format.

We Tried Telling You Nicely ● Convert your 60D h.264 footage to ProRes early in your project, if you plan to move between software packages. If you are staying within Premiere only, this step is not required (but still good practice). ● Definitely convert 60D footage to ProRes if you are using FCP7 for best compatibility.

Camera Comparison

Pros + Cons 60D Ninja shooter

:)

:(

● Small package ● Batteries last long time ● Common form factor (won’t stand out) ● Media easy to handle (inexpensive) ● Files very small ● Great in low light ● Lots of lenses + decent autofocus

● Poor Audio controls/quality ● Limited to HD ● Latitude okay not great ● h.264 leaves little room to play in post ● Limited to 10 minutes ● No timecode ● Limited frame rates ● Jello!

Pros + Cons Black Magic Pocket Ninjaer shooterer

:)

:(

● Tiny package ● Uses M4/3 lenses (small) ● Image quality ● Great dynamic range ● Shoot RAW or Prores ● Amazing colour quality ● Far more options in post ● No take time limit ● Image quality

● Poor Audio controls/quality ● Limited to HD ● Screen a pain ● Focus can be difficult ● Large file sizes (vs. 60D) ● Poor battery life ● Gets warm?!?! ● Poor UI

Media Management

Archival Strategy More is Betterer

THERE ARE TWO TYPES OF HARD DRIVES: 1) A hard drive that has failed 2) A hard drive that is about to fail

Physical Side of Data Digital Tupperware

● Buy a hard drive. CFAT can assist with tracking down the right model for you. ● Generally a portable desktop style drive is best cost/performance. ○ External power required ○ More susceptible to physical shock ○ Store at CFAT or wherever you are working (if possible) ● Get a external laptop drive if you are moving the drive around a lot, or travelling while creating media. ○ No extra power needed (USB3 or Firewire) ○ Generally slower/smaller/more expensive than desktop drive ● NO THUNDERBOLT CONNECTIVITY AT CFAT LaCie D2 Quadra Interface V3 2TB recommended

($200 Amazon.ca)

Project Management Those with OCD get A+

● Be consistent with file naming. ● Have a clear file structure ● There should be separate folders for original media (video, audio, images should all have different folders), intermediary exports, archival exports, project files, and notes. ● Often collected media will be best filed into folders by date, with notes to accompany ● A program like Post Haste can give you consistent and reproducible folder structures for your projects. If you stick to CFAT’s guidelines getting help when you need it will be much easier.

Dual Sound Keeping it Together

● Clear layout ● Combine both files in Premiere ● Creates new subfile which links to original media - does not create a new file on HDD! ● Will cover usage in Premiere workshop ● Make sure media is clearly labelled in subfolders