CFAT.2014.08
Intro to Video + Audio
In
This Class
Themes to cover.
● Description of codecs + container formats ● When to use what, including web, SD, HD, 2k and beyond resolutions, comparisons ● Constraints, strengths and options of cameras
● File + Project Management
Containers
Containers Description and definitions
● File types usually defined/identified by extension ● Not limited video, capable of housing several different types of data: ○ Video ○ Audio ○ Still Images ○ Metadata ● Often contain data which is compressed (codecs) ● Also called "wrapper": Possible to extract video from one container and insert it into another (without changing the contained data): "rewrapping"
Containers Questions to Ask: Why to Use What
1. Adoption: How likely is it that other users will be able to open the container you are using? 2. Codecs: Does the container support the codec you wish to use in the way you intend? 3. Efficiency: How many system resources will the container use to play back? Does the playback medium have hardware which is specialized in playing back the media? 4. Extras: Does the format have expanded features that you wish to use?
Containers Types - MOV
.MOV: Quicktime, Apple Format, Native to Quicktime player and all Apple media infrastructure (iMovie, Final Cut, etc) ● Multitrack format. Each track may contain audio, video, effects, or text. ● Each track contains discreet codec IE video and audio may have different codecs ● Instead of codec/media information track might contain a reference to other media ● Can use the same MPEG-4 codecs as MP4. (but many, many more)
Containers Types - MP4
.MP4 (MPEG-4 part 14): ● Official container format for MPEG-4 as defined by the MPEG working group. ● Interchangeable with MOV in some cases. ● .m4a introduced by Apple for aac/alac (audio only) ○ .m4a Can be renamed to .mp4 with no ill effects
Used on YouTube for most HD and some SD content (boo FLV!) Also used on Vimeo and many other video playback sites.
Containers Types - AVI
AVI (Audio Video Interleave): MS Format ● No Standard Timecode ● New compression techniques (MPEG-4) not accounted for, some techniques not supported ● Variable frame rate not supported ● High degree of compatibility
Containers Types - MKV
.MKV: Matroska ● Open Standard/Open Source (LGPL and BSD licensing) ● .MKA for audio only, .MKS for subtitle files ● Limited playback support for both hardware and software (but software solutions exist for all major and many minor operating systems) ● Workaround to import into Premiere (not for editing) ● Fast seeking through file / Internet streaming ● Menus/Chapters ● Selectable Subtitles ● Pirates love it.
Containers AVC/AVCHD
AVCHD (Advanced Video Coding High Definition) ● Meant as a recording and distribution container ● Contains MPEG-4 video (is a brand name owned by Panasonic and Sony) ○ NXCAM (Sony Pro) ○ AVCHD Lite (Limited to 720p/30) ○ AVCCAM (Panasonic Pro) ● File Structure is somewhat compatible with BluRay players - some will play AVCHD DVD discs.
Containers To End.
While Editing
Use .MOV (usually) (unless you are using RAW)
Containers The end of the end.
● Apple has designed Final Cut and all other OS X media products to work most efficiently with MOV. ● Apple products such as iTunes and video capable iPods will not recognize non Quicktime formats (nor will they recognize MOVs which are not encoded with H.264 codec). (This is a distribution issue, not an editing issue.) ● In Windows .mov is a crap format for high end systems Limited to 32bit
Containers The end of the end of the end.
● Premiere has a lot more flexibility with containers - can use MOV, AVI, MXF, and unencrypted MPEG variations to name a few.
Codecs
Codecs Basics
The root of codec is code/decode. Simply put, a codec is a computer program which is capable of encoding and decoding. Codec can apply to many things, not just media. Codecs fall into two main categories: lossless and lossy. A way to categorize media codecs is: acquisition - intermediary - distribution
Codecs Basics - Lossless
● The most common example of a lossless codec is ZIP archiving. There is no data loss, but file size is smaller ● Transporting video with zero degradation is highly desirable, especially in the post workflow phase ● Lossless video codecs exist - Animation, HuffyUV, and JPEG2000 ● System requirements limit use of these codecs. NO LOSSLESS CODECS EXIST FOR STANDARDIZED FINAL DISTRIBUTION
Codecs Basics - Lossy
● Most common and varied codecs ● Range from highly compressed with degradation to lightly compressed and visually lossless. ○ Generational loss ● Intermediary codecs often used in place of lossless codecs. ○ DV/HDV MOV container ○ Avid DNxHD ○ Apple Prores ○ AVCIntra100 (MXF)
}
Intermediary Codecs Prores: Tried and true MPEG2
● Based on MPEG2 ● Full-width (Full Raster) 1920x1080 and 1280x720 ● Intraframe Codec ● Designed to be efficient and be used in a post environment ● Multiple versions to choose from: 422 (original), 422 HQ (v2.0), 422 LT (v2.0) , 4444 (v2.0), 4444 XQ
Codecs Distribution
● Distribution formats are almost all lossy (even 35mm!) ● Major distribution formats include: ○ H.264/x.264 - Internet (H.265 will replace) ○ DivX/Xvid - Internet ○ MPEG-2 (SD and HD) - DVD/Bluray ○ VP9 - codec to watch for!
JPEG2000 in a MXF DCP
Codecs - Distribution DivX/Xvid: Piracy in 2001
● Early MPEG-4 based codec ● Good hardware adoption (many DVD players will play from disc or USB stick) ● Largely superseded by more modern MPEG-4 codec called H.264
Codecs - Distribution H.264/AVC: Magic Codec
● A subset of MPEG-4 ● Retains excellent quality at low bitrate ● Very popular for online distribution ● Used in HDTV and BluRay (encrypted) ● Uses a wide range of profiles which define quality and features used. ● Can be Intra or Inter frame compressed ● Often doesn't hold up well over multiple generations, except for high end profiles or implementations such as AVC-Intra100 ● x.264: open source variant
Codecs - Distribution MPEG - A format and a company!
● Not just a distribution codec - High quality MPEG2 encoding is used commonly on ENG video cameras. ● MPEG-2 was the defacto standard for home distribution via grandpa’s DVDs.
● MPEG-2 is still the standard codec for use in HD BluRay discs (MPEG-4 may also be used). ● Being replaced by MPEG4 for acquisition, editing, and distribution.
Compression
Compression Nuts + Bolts
● Uncompressed video is huge. ● Compression usually trades quality for space savings. ● Different kinds of compression result in different system requirements. ● Basically works by simplifying the image (thus quality loss) Intraframe codecs use only spatial compression Interframe codecs use spatial AND temporal compression
Spatial Compression Compression of a Scene
Within a scene adjoining pixels are analysed. If they are within a threshold they are combined to be the same colour. This means that colour information is reduced, thus space required is reduced.
Spatial Compression Compression of an Image
Spatial Compression Formats
● DV ● AVC-Intra ● DVCPro HD ● Prores ● R3D
Formats which use only spatial compression are referred to as intraframe codecs
Temporal Compression Compression of an Image Over Time
This compression usually follows a spatial pass. The type of frames used in temporal compression can be divided into in three categories: ● I-frames: contain all the information about the image. Each frame is analyzed individually. ● P-frames: 'predicted picture' frames only record changes from one frame to the next. ● B-frames: 'Bi-predictive picture' frames work like P-frames, but in both directions, sharing pixels across 3 or more frames. Come between I or P frames
Temporal Compression Keeping it Together
GOPS A Group of Pictures is the sequence in which I,P and B frames are assembled to make a motion video. HDV, MPEG2, low end MPEG4 (H.264, AVCHD) all use GOPS and can be difficult to edit properly because of cutting rules (can't separate specific frame types. They can also lend to image degradation in high motion due to high information changes within GOPS.
Temporal Compression
Comparing Intraframe to Interframe compression
GOPS: Intraframe = all I frames Interframe = mixture of I B P frames
Temporal Compression
Comparing Intraframe to Interframe compression
I-Frame
Temporal Compression
Comparing Intraframe to Interframe compression
P or B Frame
Intra VS. Inter In Case You Missed It
Blackmagic = ProRes = INTRAFRAME = AWESOME 60D = h.264 = INTERFRAME = LESS AWESOME
Bitrate
Bitrate Quality Slider
● Definition of how much information is stored per second ● Relates to spatial and temporal compression ● Often higher bitrate = better image (less compression), but it is dependent on the codec used as well. ● So why use MPEG 2 instead of always using MPEG-4? ● No simple rule ● Guidelines for distribution give good starting point: DVD, BluRay and YouTube
Bitrate Quality Slider
Bitrate guidelines: DVD: Up to 9.8 Mb/s (9,800 Kb/s) for MPEG2 video (audio extra) BluRay: Up to 48 Mb/s (48,000 Kb/s) for MPEG2 or H.264 YouTube: Up to 7.8 Mb/s (7,800 Kb/s) for H.264 (this is a guideline only - YouTube states there is no maximum, only a maximum file size. Your own testing is recommended)
When to Use What Formats, quality, etc.
Camera File Types
They shoot the same thing! No they don’t!
Blackmagic = ProRes = file.mov 60D = MPEG 4 = file.mov
=
When To Use What (Staying Native)
● Do as few conversions as possible. Every conversion involves some loss of quality. ● When in post phase only transcode to a higher quality format (minimize your losses.) ● If possible stay in the native format throughout (BlackMagic Prores). ● If you're going to output into a higher quality format other than the one you shot with, convert to that format as quickly as you can, usually during capture or digitize. If you can't convert on capture, convert before working with the footage. ● If outputting to lower quality format, IE H.264 for web, wait until end of workflow before converting to that format.
We Tried Telling You Nicely ● Convert your 60D h.264 footage to ProRes early in your project, if you plan to move between software packages. If you are staying within Premiere only, this step is not required (but still good practice). ● Definitely convert 60D footage to ProRes if you are using FCP7 for best compatibility.
Camera Comparison
Pros + Cons 60D Ninja shooter
:)
:(
● Small package ● Batteries last long time ● Common form factor (won’t stand out) ● Media easy to handle (inexpensive) ● Files very small ● Great in low light ● Lots of lenses + decent autofocus
● Poor Audio controls/quality ● Limited to HD ● Latitude okay not great ● h.264 leaves little room to play in post ● Limited to 10 minutes ● No timecode ● Limited frame rates ● Jello!
Pros + Cons Black Magic Pocket Ninjaer shooterer
:)
:(
● Tiny package ● Uses M4/3 lenses (small) ● Image quality ● Great dynamic range ● Shoot RAW or Prores ● Amazing colour quality ● Far more options in post ● No take time limit ● Image quality
● Poor Audio controls/quality ● Limited to HD ● Screen a pain ● Focus can be difficult ● Large file sizes (vs. 60D) ● Poor battery life ● Gets warm?!?! ● Poor UI
Media Management
Archival Strategy More is Betterer
THERE ARE TWO TYPES OF HARD DRIVES: 1) A hard drive that has failed 2) A hard drive that is about to fail
Physical Side of Data Digital Tupperware
● Buy a hard drive. CFAT can assist with tracking down the right model for you. ● Generally a portable desktop style drive is best cost/performance. ○ External power required ○ More susceptible to physical shock ○ Store at CFAT or wherever you are working (if possible) ● Get a external laptop drive if you are moving the drive around a lot, or travelling while creating media. ○ No extra power needed (USB3 or Firewire) ○ Generally slower/smaller/more expensive than desktop drive ● NO THUNDERBOLT CONNECTIVITY AT CFAT LaCie D2 Quadra Interface V3 2TB recommended
($200 Amazon.ca)
Project Management Those with OCD get A+
● Be consistent with file naming. ● Have a clear file structure ● There should be separate folders for original media (video, audio, images should all have different folders), intermediary exports, archival exports, project files, and notes. ● Often collected media will be best filed into folders by date, with notes to accompany ● A program like Post Haste can give you consistent and reproducible folder structures for your projects. If you stick to CFAT’s guidelines getting help when you need it will be much easier.
Dual Sound Keeping it Together
● Clear layout ● Combine both files in Premiere ● Creates new subfile which links to original media - does not create a new file on HDD! ● Will cover usage in Premiere workshop ● Make sure media is clearly labelled in subfolders