Computer Network Architectures and Multimedia Guy Leduc Chapter 4 Multimedia Applications & Transport

Sections 7.1 to 7.4 from Computer Networking: A Top Down Approach, 6th edition. Jim Kurose, Keith Ross Addison-Wesley, March 2012. Also 7.4.2 and 7.4.7 from Computer Networks - 4th edition Andrew S. Tanenbaum Prentice-Hall International, 2003

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp.

4-1

Multimedia networking: outline 4.1 multimedia networking applications 4.2 streaming stored video 4.3 voice-over-IP 4.4 protocols for real-time conversational applications

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp.

4-2

1

Multimedia: audio

! 

! 

PCM (Pulse Code Modulation): analog audio signal sampled at constant rate "  telephone: 8,000 samples/sec "  CD music: 44,100 samples/sec each sample quantized, i.e., rounded "  each quantized value represented by bits, "  e.g., rounded to one of 28=256 values "  8 bits/sample receiver converts bits back to analog signal: "  some quality reduction

quantization error audio signal amplitude

! 

quantized value of analog value analog signal

time sampling rate (N sample/sec)

4: Multimedia App. & Transp.

©From Computer Networking, by Kurose&Ross

4-3

Multimedia: audio Examples:

! 

Telephony: !  8,000 samples/sec, 8 bits/sample: 64 kbps CD music: !  44,100 samples/sec, 16 bits/sample: 705.6 kbps !  Stereo: 1.411 Mbps

Other example rates !  ! 

MP3: 96, 128, 160 kbps Internet telephony: 5.3 kbps and up

©From Computer Networking, by Kurose&Ross

quantization error audio signal amplitude

! 

quantized value of analog value analog signal

time sampling rate (N sample/sec)

4: Multimedia App. & Transp.

4-4

2

More on Audio Compression

MP3 (MPEG 1 audio layer 3) takes masking effects into account and does not encode masked signals. Can compress stereo CD down to 96-128 kbps. 4: Multimedia App. & Transp.

From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

Multimedia: video ❒  video: sequence of images

displayed at constant rate ❍  e.g. 25 images/sec ❒  digital image: array of pixels ❍  each pixel represented by bits ❒  coding: use redundancy within and between images to decrease # bits used to encode image ❍  spatial (within image) ❍  temporal (from one image to next)

spatial coding example: instead of sending N values of same color (all purple), send only two values: color value (purple) and number of repeated values (N)

……………………...… ……………………...…

frame i

temporal coding example: instead of sending complete frame at i+1, send only differences from frame i

frame i+1

©From Computer Networking, by Kurose&Ross

4-5

4: Multimedia App. & Transp.

4-6

3

Multimedia: video ! 

! 

! 

CBR: (constant bit rate): video encoding rate fixed VBR: (variable bit rate): video encoding rate changes as amount of spatial, temporal coding changes examples: "  MPEG 1 (CD-ROM) 1.5 Mbps "  MPEG2 (DVD) 3-6 Mbps "  MPEG4 (often used in Internet, < 1 Mbps)

spatial coding example: instead of sending N values of same color (all purple), send only two values: color value (purple) and number of repeated values (N)

……………………...… ……………………...…

frame i

temporal coding example: instead of sending complete frame at i+1, send only differences from frame i

frame i+1

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp.

4-7

Video - Digital Systems ❒  Consider a rectangular 4:3 grid of pixels, such as ❍  VGA: 640 x 480 ❍  XGA: 1024 x 768 ❒  Pixel = 8 bits for each of the RGB colours ❒  25 frames per sec ❒  With XGA : ❍  24 bits/pixel x 1024 x 768 x 25 frames/sec = 472 Mbps! ❒  Needs compression!

From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp.

4-8

4

Data compression ❒  Encoding/decoding schemes ❒  Video on Demand (VoD) ❍  ❍  ❍ 

Encoding can be slow (done once) Decoding must be fast (done many times) Asymmetrical schemes

❒  Real-time multimedia (e.g. videoconference) ❍ 

Symmetrical schemes

❒  Lossy compression ❍  ❍ 

Encode/decode is not neutral When acceptable, leads to better compression ratios

❒  Two main compression schemes: ❍  ❍ 

Entropy encoding (lossless) Source encoding (lossy)

From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp.

4-9

Entropy encoding ❒  Lossless ❒  Three typical examples: ❍  Run-length encoding •  repeated symbols are encoded as “Special symbol + number of occurrences” ❍ 

Statistical encoding

❍ 

Look up table

•  short codes for frequent symbols (Huffman encoding) •  e.g. CLUT (Colour Look Up Table) •  define the table of the colours actually used •  send table index instead of a 24-bit colour value

From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-10

5

Source encoding ❒  Lossy

❒  Three main examples: ❍  Differential encoding

•  sequence of values are encoded by representing the differences from the previous values •  makes sense if differences are encoded with less bits •  lossy when there are large jumps between two values and a fixed number of bits per difference •  lossless if variable-length encoding is used

❍ 

Transformation

•  e.g. Fourier or DCT Transform •  lossy since only the first amplitudes are sent

❍ 

Variant of Look Up Table with approximations to closest value

4: Multimedia App. & Transp.

From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

4-11

JPEG ❒  Joint Photographic Experts Group

❒  ISO/IEC and ITU standard for compressing still pictures ❒  Compression ratio 20:1 is typical

❒  Roughly symmetrical scheme (decoding as long as encoding) ❒  Lossy sequential mode: ❍ 

6 steps

Block preparation

Discrete Cosine transform

From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

Quantization

Differential quantization

Run-length encoding

Statistical Output encoding

4: Multimedia App. & Transp. 4-12

6

JPEG - Step 1 Block preparation

Discrete Cosine transform

Quantization

Differential quantization

Run-length encoding

Statistical Output encoding

❒  Step 1: block preparation ❍ 

Translate RGB into luminance (Y) and 2 chrominance (I,Q) values •  gives better compression •  we get 3 matrices of pixels

❍ 

Average square blocks of 4 pixels for I and Q •  lossy but unnoticeable

❍  ❍ 

Subtract 128 from each element (0 is middle) Divide up frame into 8x8 blocks

4: Multimedia App. & Transp. 4-13

From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

JPEG - Step 2 Block preparation

Discrete Cosine transform

Quantization

Differential quantization

Run-length encoding

Statistical Output encoding

❒  Step 2: DCT (Discrete Cosine Transformation) to each block ❍ 

Sort of 2 dimensional Discrete Fourier Transform, but more compact •  Advantage: most of the spectral power in the first few terms

❍  ❍ 

Output: block of 8x8 elements (coefficient of DCT) Slightly lossy in practice (round-off errors)

From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-14

7

JPEG - Step 3 Block preparation

Discrete Cosine transform

Quantization

Differential quantization

Run-length encoding

Statistical Output encoding

❒  Step 3: Quantization ❍  Apply sort of low pass filter to coefficients (lossy)

From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-15

JPEG - Steps 4, 5 and 6 ❒  Step 4: Differential quantization ❍ 

Replace upper-left (DC) coefficient by its difference with corresponding element of previous block

❒  Step 5: Run-length encoding ❍ 

Applied to a zig-zag scanning pattern

❒  Step 6: Statistical output encoding (Huffman) From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-16

8

MPEG ❒  Motion Picture Experts Group - ISO standard ❒  Audio and video

❒  MPEG-1 ❍  Video-recorder quality (CD-ROM) ❍  1.2 Mbps output ❒  MPEG-2 ❍  Broadcast quality ❍  4-6 Mbps output is typical but higher for HDTV

❒  MPEG-4 ❍  Medium-resolution videoconferencing with low frame rate •  10 frames/sec

4: Multimedia App. & Transp. 4-17

From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

MPEG-1 Audio signal

Audio encoder System multiplexer

Clock Video signal

MPEG-1 output

Video encoder

❒  Audio and video encoders work independently ❒  Timestamps included in both flows for

synchronization at receiver ❒  Audio compression (MP3) ❍ 

Also, exploitation of redundancy in the 2 channels of a stereo stream

From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-18

9

MPEG-1 - Video compression ❒  Exploit spatial and temporal redundancies ❒  Spatial redundancy: like JPEG ❒  But adds temporal redundancy ❍ 

Many common parts in the following three consecutive frames!

From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-19

MPEG-1 - Video compression (2) ❒  Temporal redundancy: four kinds of frames: I, P, B, D ❍  I frames (Intracoded)

•  self-contained JPEG-encoded still pictures •  should appear periodically in the output (initial synch, resynch on error, fast forward or rewind)

❍ 

P frames (Predictive)

•  block-by-block difference with the previous frame •  search for a macroblock (Y,I,Q) in previous frame which is equal or slightly different •  encode the offset in position and difference

❍  ❍ 

B frames (Bidirectional): same as P but search also in next I or P frame D frames (DC-coded): block averages for fast forward (low resolution)

❒  Example of part of an MPEG sequence ❍  I B B P B B I From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-20

10

MPEG-2 ❒  Similar to MPEG-1 ❒  Better quality (10 x 10 DCT coefficients instead

of 8 x 8) ❒  Several resolution levels (lowest one is comparable to MPEG-1) ❒  Several profiles (e.g. no B frames to simplify encoding) ❒  Usually 3-4 Mbps, but can go up to 100 Mbps (HDTV)

From Computer Networks, by Tanenbaum © Prentice Hall ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-21

Multimedia networking: 3 application types ❒ 

streaming, stored audio, video ❍ 

❍  ❍ 

❒ 

conversational voice/video over IP ❍ 

❍ 

❒ 

streaming: can begin playout before downloading entire file stored (at server): can transmit faster than audio/video will be rendered (implies storing/buffering at client) e.g., YouTube, Netflix, Hulu interactive nature of human-to-human conversation limits delay tolerance e.g., Skype

streaming live audio, video ❍ 

e.g., live sporting event

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-22

11

MM Networking Applications Fundamental characteristics: ❒  typically delay sensitive ❍  ❍ 

end-to-end delay delay jitter

Jitter is the variability of packet delays within the same packet stream

❒  loss tolerant: infrequent losses cause

minor glitches ❒  antithesis of data, which are loss intolerant but delay tolerant

QoS (Quality of Service) refers to performance metrics such as delay, bandwidth, jitter and loss 4: Multimedia App. & Transp. 4-23

©From Computer Networking, by Kurose&Ross

Multimedia Over Today’s Internet TCP/UDP/IP: “best-effort service” ❒ 

no guarantees on delay, bandwidth, jitter, loss (if UDP)

? ?

?

?

?

?

?

But you said multimedia apps require ? QoS and level of performance to be ? effective! ?

?

Today’s Internet multimedia applications use application-level techniques to mitigate (as best possible) effects of delay, loss ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-24

12

Chapter 4: outline 4.1 multimedia networking applications 4.2 streaming stored video 4.3 voice-over-IP 4.4 protocols for real-time conversational applications

4: Multimedia App. & Transp. 4-25

©From Computer Networking, by Kurose&Ross

Internet multimedia: simplest approach Media player ❒  jitter removal ❒  decompression ❒  error concealment ❒  graphical user interface with controls for interactivity ❒  audio or video stored in file

❒  files transferred as HTTP object

received in entirety at client then passed to player audio, video not streamed in this scenario: ❒  no, “pipelining,” long delays until playout! ❍  ❍ 

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-26

13

Internet multimedia: streaming approach

❒  browser GETs metafile ❒  browser launches player, passing metafile ❒  player contacts server ❒  server streams audio/video to player ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-27

Streaming from a streaming server

❒  allows for non-HTTP protocol between server and media player ❒  UDP or TCP for step (3), more shortly

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-28

14

Streaming Multimedia: client rate(s) 1.5 Mbps encoding

28.8 Kbps encoding

Q: how to handle different client receive rate capabilities? A: server stores, transmits multiple copies of video, encoded at different rates (info found in meta-file) 4: Multimedia App. & Transp. 4-29

©From Computer Networking, by Kurose&Ross

Cumulative data

Streaming stored video:

1.  video Recorded (e.g., 30 frames/ sec)

©From Computer Networking, by Kurose&Ross

2. video sent network delay (fixed in this example)

3. video received, played out at client (30 frames/sec) time

streaming: at this time, client playing out early part of video, while server still sending later part of video 4: Multimedia App. & Transp. 4-30

15

Streaming stored video: challenges !  continuous

playout constraint: once client playout begins, playback must match original timing "  … but network delays are variable (jitter), so will need client-side buffer to match playout requirements !  other challenges: "  client interactivity: pause, fast-forward, rewind, jump through video "  video packets may be lost, retransmitted 4: Multimedia App. & Transp. 4-31

©From Computer Networking, by Kurose&Ross

Streaming stored video: revisited client video reception

variable network delay

client playout delay

❒ 

constant bit rate video playout at client

buffered video

Cumulative data

constant bit rate video transmission

time

client-side buffering and playout delay: compensate for network-added delay, delay jitter

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-32

16

Client-side buffering, playout buffer fill level, Q(t)

playout rate, e.g., CBR r

variable fill rate, x(t) client application buffer, size B

video server

client

CBR = Constant Bit Rate VBR = Variable Bit Rate

4: Multimedia App. & Transp. 4-33

©From Computer Networking, by Kurose&Ross

Client-side buffering, playout buffer fill level, Q(t)

playout rate, e.g., CBR r

variable fill rate, x(t)

video server

client application buffer, size B

client

1. initial fill of buffer until playout begins at tp 2. playout begins at tp, 3. buffer fill level varies over time as fill rate x(t) varies and playout rate r is constant ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-34

17

Client-side buffering, playout buffer fill level, Q(t)

playout rate, e.g., CBR r

variable fill rate, x(t) client application buffer, size B

video server

x < r: buffer may empty, causing freezing of video playout until buffer again fills

initial playout delay tradeoff: buffer starvation less likely with larger delay, but larger delay until user begins watching ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-35

Streaming multimedia: UDP ❒  server sends at rate appropriate for client

often: send rate = encoding rate = constant rate ❍  transmission rate can be unaware of network congestion level ❒  short playout delay (2-5 seconds) to remove network jitter ❒  error recovery: application-level, time permitting ❒  encapsulation of audio/video chunks in RTP (Real-Time Transport Protocol, RFC 3550) and then in UDP ❍ 

❍ 

see later for details

❒  needs a control connection in parallel to pause, resume reposition,

etc: ❍ 

Real-Time Streaming Protocol (RTSP, RFC 2326)

❒  issue: UDP may

not go through firewalls

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-36

18

User Control of Streaming Media: RTSP HTTP ❒  does not target multimedia content ❒  no commands for fast forward, etc. RTSP ❒  Real-Time Streaming Protocol ❒  client-server application layer protocol ❒  user control: rewind, fast forward, pause, resume, repositioning, etc.

What it doesn’t do: ❒  doesn’t define how audio/video is encapsulated for streaming over network ❒  doesn’t restrict how streamed media is transported (UDP or TCP possible) ❒  doesn’t specify how media player buffers audio/video 4: Multimedia App. & Transp. 4-37

©From Computer Networking, by Kurose&Ross

RTSP: out-of-band control FTP uses an “out-ofband” control channel: ❒  file transferred over one TCP connection ❒  control info (directory changes, file deletion, rename) sent over separate TCP connection ❒  “out-of-band”, “inband” channels use different port numbers

©From Computer Networking, by Kurose&Ross

RTSP messages also sent out-of-band: ❒  RTSP control messages use different port numbers than media stream: out-of-band ❍  port 554 ❒  media stream is considered “in-band”

4: Multimedia App. & Transp. 4-38

19

RTSP example Scenario: ❒  metafile communicated to web browser (1) ❒  browser launches player (2) ❒  player sets up an RTSP control connection, data connection to

streaming server (3)

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-39

Metafile Example Twister ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-40

20

RTSP Operation

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-41

RTSP exchange example C: SETUP rtsp://audio.example.com/twister/audio RTSP/1.0 Transport: rtp/udp; compression; port=3056; mode=PLAY S: RTSP/1.0 200 1 OK Session 4231 C: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=0C: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=37 C: TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 S: 200 3 OK ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-42

21

Streaming multimedia: TCP ❒  multimedia file retrieved via HTTP GET ❒  send at maximum possible rate under TCP variable rate, x(t) video file

TCP send buffer

server

TCP receive buffer

application playout buffer

client

❒  issue: fill rate fluctuates much due to TCP

congestion control and TCP retransmissions (inorder delivery) ❒  larger playout delay: smooth TCP delivery rate ❒  HTTP/TCP passes more easily through firewalls ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-43

Streaming multimedia: DASH DASH: Dynamic, Adaptive Streaming over HTTP ❒  server: ❒ 

❍  ❍  ❍ 

❒ 

divides video file into multiple chunks each chunk stored, encoded at different rates manifest file: provides URLs for different chunks

client: ❍  ❍ 

periodically measures server-to-client bandwidth consulting manifest, requests one chunk at a time •  chooses maximum coding rate sustainable given current bandwidth •  can choose different coding rates at different points in time (depending on available bandwidth at time)

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-44

22

Streaming multimedia: DASH ❒  DASH:

Dynamic, Adaptive Streaming over

HTTP ❒  “intelligence” at client: client determines ❍  when

to request chunk (so that buffer starvation, or overflow does not occur) ❍  what encoding rate to request (higher quality when more bandwidth available) ❍  where to request chunk (can request from URL server that is “close” to client or has high available bandwidth)

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-45

Content distribution networks ❒ 

❒ 

challenge: how to stream content (selected from millions of videos) to hundreds of thousands of simultaneous users? option 1: single, large “mega-server” ❍  ❍  ❍  ❍ 

single point of failure point of network congestion long path to distant clients multiple copies of video sent over outgoing link

… quite simply: this solution doesn’t scale

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-46

23

Content distribution networks ❒ 

❒ 

challenge: how to stream content (selected from millions of videos) to hundreds of thousands of simultaneous users? option 2: store/serve multiple copies of videos at multiple geographically distributed sites (CDN) ❍ 

enter deep: push CDN servers deep into many access networks •  close to users •  used by Akamai, 1700 locations

❍ 

bring home: smaller number (10’s) of larger clusters in POPs near (but not within) access networks

❍ 

Google uses both, in addition to its “mega data centers” responsible for serving dynamic content

•  used by Limelight

4: Multimedia App. & Transp. 4-47

©From Computer Networking, by Kurose&Ross

CDN: “simple” content access scenario Bob (client) requests video http://video.netcinema.com/6Y7B23V actually stored in a KingCDN content distribution server 1. Bob gets URL for video http:// video.netcinema.com/6Y7B23V from netcinema.com 2 web page 1 6. request video from 5 KingCDN server, streamed via HTTP 3. netcinema’s DNS returns netcinema.com a1105.kingcdn.com 3

netcinema authoritative DNS

KingCDN content distribution server

©From Computer Networking, by Kurose&Ross

2. resolve video.netcinema.com via Bob’s local DNS that relays to netcinema’s authoritative DNS server

4

4&5. Resolve a1105.kingcdn.com via KingCDN’s authoritative DNS, which returns IP address of KingCDN distribution server with video

KingCDN authoritative DNS 4: Multimedia App. & Transp. 4-48

24

CDN cluster selection strategy ❒ 

challenge: how does CDN DNS select “good” CDN node to stream to client ❍  ❍ 

CDN learns the IP address of the client’s local DNS via the client’s DNS lookup CDN can then implement a selection strategy to dynamically direct clients to a “suitable” server cluster or data center

❒  Possible strategies: ❍  ❍ 

❍ 

1. pick IP address of CDN node geographically closest to client 2. pick IP address of CDN node with shortest delay (or min # hops) to client (CDN nodes periodically ping access ISPs, reporting results to CDN DNS) 3. always pick the same IP address, but make sure this IP address is an IP anycast address associated with all CDN nodes •  see next slide

❒ 

alternative: let client decide! ❍  ❍  ❍ 

give client a list of several CDN servers client pings servers, picks “best” Netflix approach 4: Multimedia App. & Transp. 4-49

©From Computer Networking, by Kurose&Ross

IP anycast ❒  An IP anycast address is an IP unicast address, but this address is assigned to

multiple devices that are geographically distributed ❍ 

Like a private address, but an anycast address is routable, i.e. reachable from anywhere in the Internet

❒  When a source sends a packet to an IP anycast address, the packet reaches

one of the devices associated with this address ❍  ❍  ❍  ❍  ❍ 

The choice is made “by the network”, not by the source, because it results from a decision taken at the interdomain routing protocol (BGP) level In other words, BGP decides by applying its usual routing rules based on preferences, AS-Path length, Hot Potato criterion, etc. So, don’t confuse anycast and multicast! An anycast address is a unicast address And nothing can tell you if a unicast address is actually anycast or not! B CDN cluster (IP anycast)

A

Source 1

D

Source 2

C CDN cluster (same IP anycast)

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-50

25

IP anycast in a CDN – Role of BGP ❒  The CDN assigns the same IP address range to each of its clusters ❍ 

Therefore, it’s an anycast address

The CDN relies on standard BGP to advertise this IP address range from each of the different cluster locations ❒  When a BGP router receives multiple route advertisements for this same IP address range, it treats them as providing several paths to the same physical location and picks the “best” ❒ 

1. BGP: AS-PATH “A” to CDN prefix 2. BGP: AS-PATH “BA” to CDN prefix 3. BGP: AS-PATH “C” to CDN prefix CDN cluster (IP prefix)

1

B

2

A

Client

D

1

©From Computer Networking, by Kurose&Ross

In D, BGP selects the “best” path between “BA” and “C” to reach the CDN prefix

C

3 CDN cluster (same IP prefix)

A, B, C are Autonomous Systems (AS)

4: Multimedia App. & Transp. 4-51

Case study: Netflix ❒  30% downstream US traffic in 2011 ❒  owns very little infrastructure, uses 3rd party

services: ❍  own registration, payment servers ❍  Amazon (3rd party) cloud services:

•  Netflix uploads studio master to Amazon cloud •  create multiple version of movie (different encodings) in cloud •  upload versions from cloud to CDNs •  Cloud hosts Netflix web pages for user browsing

❍  three

3rd party CDNs host/stream Netflix content: Akamai, Limelight, Level-3

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-52

26

Case study: Netflix Amazon cloud

Netflix registration, accounting servers 2. Bob browses Netflix video 2

upload copies of multiple versions of video to CDNs

3. Manifest file returned for requested video

Akamai CDN

Limelight CDN

3

1 1. Bob manages Netflix account

Level-3 CDN

4. DASH streaming

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-53

Chapter 4: outline 4.1 multimedia networking applications 4.2 streaming stored video 4.3 voice-over-IP 4.4 protocols for real-time conversational applications

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-54

27

Voice-over-IP (VoIP) ❒ 

VoIP end-end-delay requirement: needed to maintain “conversational” aspect ❍  ❍  ❍  ❍ 

higher delays noticeable, impair interactivity < 150 msec: good > 400 msec: bad includes application-level (packetization,playout), network delays

session initialization: how does callee advertise IP address, port number, encoding algorithms? ❒  value-added services: call forwarding, screening, recording ❒  emergency services: 112 (Europe), 911 (North America) ❒ 

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-55

VoIP characteristics ❒  speaker’s audio: alternating talk spurts, silent

periods. ❍ 

64 kbps during talk spurt

❍ 

pkts generated only during talk spurts

❍ 

20 msec chunks at 8 Kbytes/sec: 160 bytes of data

❍ 

so, 20 msec of packetization delay

❒  application-layer header added to each chunk ❒  chunk+header encapsulated into UDP (or TCP)

segment

❒  application sends segment into socket every 20 msec

during talkspurt

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-56

28

VoIP: packet loss, delay ❒  network

loss: IP datagram lost due to network congestion (router buffer overflow) ❒  delay loss: IP datagram arrives too late for playout at receiver ❍  delays:

processing, queueing in network; end-system (sender, receiver) delays ❍  typical maximum tolerable delay: 400 ms ❒  loss

tolerance: depending on voice encoding, loss concealment, packet loss rates between 1% and 10% can be tolerated 4: Multimedia App. & Transp. 4-57

©From Computer Networking, by Kurose&Ross

constant bit rate transmission variable network delay (jitter)

client reception

constant bit rate playout at client

buffered data

Cumulative data

Delay jitter

time

client playout delay

❒  end-to-end delays of two consecutive packets:

difference can be more or less than 20 msec (transmission time difference)

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-58

29

VoIP: fixed playout delay ❒  receiver attempts to playout each chunk exactly

q msecs after chunk was generated ❍  chunk has timestamp t: play out chunk at t+q ❍  chunk arrives after t+q: data arrives too late for playout, data “lost” ❒  tradeoff in choosing q: ❍  large q: less packet loss ❍  small q: better interactive experience

4: Multimedia App. & Transp. 4-59

©From Computer Networking, by Kurose&Ross

VoIP: fixed playout delay •  sender generates packets every 20 msec during talk spurt

•  first packet received at time r •  first playout schedule: begins at p •  second playout schedule: begins at p’ packets

loss

packets generated packets received

playout schedule p-r playout schedule p’ - r

time r

©From Computer Networking, by Kurose&Ross

p

p'

4: Multimedia App. & Transp. 4-60

30

Adaptive playout delay (1) goal: low playout delay, low late loss rate ❒  approach: adaptive playout delay adjustment: ❒ 

❍ 

❍  ❍ 

estimate network delay, adjust playout delay at beginning of each talk spurt silent periods compressed and elongated chunks still played out every 20 msec during talk spurt

❒  adaptively estimate packet delay: (EWMA - exponentially weighted moving average, recall TCP RTT estimate):

di = (1-α)di-1 + α (ri – ti)

delay estimate after ith packet

small constant, e.g. 0.1

time received - time sent (timestamp) measured delay of ith packet

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-61

Adaptive playout delay (2) also useful to estimate average deviation of delay, vi : vi = (1-β)vi-1 + β |ri – ti – di| ❒ 

❒ 

estimates di, vi calculated for every received packet, but used only at start of talk spurt for first packet in talk spurt, playout time is:

playout-timei = ti + di + Kvi

❒ 

remaining packets in talkspurt are played out periodically Q: does it require clock synchronization?

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-62

31

Adaptive playout delay (3) Q: How does receiver determine whether packet is first in a talk spurt? ❒  if no loss, receiver looks at successive timestamps ❍  difference of successive stamps > 20 msec -> talk spurt begins ❒  with loss possible, receiver must look at both time

stamps and sequence numbers ❍ 

difference of successive stamps > 20 msec and sequence numbers without gaps -> talk spurt begins

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-63

VoIP: recovery from packet loss (1) Challenge: recover from packet loss given small tolerable delay between original transmission and playout ❒  each ACK/NAK takes ~ one RTT

Forward Error Correction (FEC) send enough bits to allow recovery without retransmission (recall two-dimensional parity)

❒  alternative: ❍ 

simple FEC

n chunks, create redundant chunk by exclusive OR-ing n original chunks ❒  send n+1 chunks, increasing throughput by factor 1/n ❒  can reconstruct original n chunks if at most one lost chunk from n+1 chunks, with playout delay ❒  called “erasure” code ❒  for every group of

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-64

32

VoIP: recovery from packet loss (2) ❒  increasing throughput: ❍  by factor 1/n ❒  increasing playout delay: ❍  need enough time to receive all n+1 packets ❒  tradeoff: ❍  increase

n, less bandwidth waste ❍  increase n, longer playout delay ❍  increase n, higher probability that 2 or more chunks will be lost

4: Multimedia App. & Transp. 4-65

©From Computer Networking, by Kurose&Ross

VoIP: recovery from packet loss (3) FEC: Reed-Solomon (RS) scheme ❒  RS is a more sophisticated error correcting code, which can be used as erasure code ❒  An (n,k) RS code encodes k source packets into n > k packets ❒  Systematic code: the n transmitted packets contain verbatim copies of the k source packets ❍  ❍ 

+ n-k new packets no decoding if no source packet loss!

❒  Linear code: coding/decoding

represented by matrix operations: ❍  ❍  ❍  ❍ 

❒  Decoding: ❍  ❍ 

❒  Optimal code: Original k

packets can be recovered provided that any k packets among n are received

©From Computer Networking, by Kurose&Ross

x is the vector of k source packets G is a n x k matrix y is the vector of n transmitted packets y=Gx

❍ 

y’ vector of any k received packets G’ is the k x k submatrix of G with rows corresponding to these packets x = G’-1 y’

4: Multimedia App. & Transp. 4-66

33

VoIP: recovery from packet loss (4) 2nd FEC scheme #  “piggyback lower quality stream” #  send lower resolution audio stream as redundant information #  e.g., nominal stream PCM at 64 kbps and redundant stream GSM at 13 kbps. #  non-consecutive loss, receiver can conceal the loss #  generalization: can also append (n-1)st and (n-2)nd low-bit

chunks

©From Computer Networking, by Kurose&Ross

rate

4: Multimedia App. & Transp. 4-67

VoIP: recovery from packet loss (5)

Interleaving to conceal loss ❒  audio chunks divided into smaller units ❒  for example, four 5 msec units per 20 ms audio chunk ❒  packet contains small units from different chunks ©From Computer Networking, by Kurose&Ross

most of every chunk ❒  no redundancy overhead, but increases playout delay ❒  if packet lost, still have

4: Multimedia App. & Transp. 4-68

34

Voice-over-IP: Skype Skype clients (SC)

❒  proprietary application-

layer protocol (inferred via reverse engineering) ❍  encrypted msgs ❒  P2P components: "  clients: skype peers connect directly to each other for VoIP call "  super nodes (SN): skype peers with special functions "  overlay network: among SNs to locate SCs

Skype login server

supernode (SN) supernode overlay network

"  login server 4: Multimedia App. & Transp. 4-69

©From Computer Networking, by Kurose&Ross

P2P voice-over-IP: skype skype client operation: 1. joins skype network by contacting SN (IP address cached) using TCP 2. logs-in (username, password) to centralized skype login server

Skype login server

3. obtains IP address for callee from SN, SN overlay "  or client buddy list 4. initiate call directly to callee

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-70

35

Skype: peers as relays Problem: both Alice, Bob are behind “NATs” ❍ 

❍ 

NAT prevents outside peer from initiating connection to insider peer inside peer can initiate connection to outside

relay solution: Alice, Bob maintain open connection to their SNs

"  Alice signals her SN to connect to Bob "  Alice’s SN connects to Bob’s SN "  Bob’s SN connects to Bob over open connection Bob initially initiated to his SN

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-71

Chapter 4: outline 4.1 multimedia networking applications 4.2 streaming stored video 4.3 voice-over-IP 4.4 protocols for real-time conversational applications: RTP/RTCP, SIP

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-72

36

Real-Time Protocol (RTP) ❒  RTP specifies packet

structure for packets carrying audio, video data ❒  RFC 3550 ❒  RTP packet provides ❍  payload type identification ❍  packet sequence numbering ❍  time stamping

❒  RTP runs in end systems ❒  RTP packets

encapsulated in UDP segments ❒  interoperability: if two Internet phone applications run RTP, then they may be able to work together

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-73

RTP runs on top of UDP RTP libraries provide transport-layer interface that extends UDP: •  port numbers, IP addresses •  payload type identification •  packet sequence numbering •  time-stamping

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-74

37

RTP Example ❒  consider sending 64

kbps PCM-encoded voice over RTP ❒  application collects encoded data in chunks, e.g., every 20 msec = 160 bytes in a chunk ❒  audio chunk + RTP header form RTP packet, which is encapsulated in UDP segment

©From Computer Networking, by Kurose&Ross

❒  RTP header indicates

type of audio encoding in each packet ❍ 

sender can change encoding during conference

❒  RTP header also

contains sequence numbers, timestamps

4: Multimedia App. & Transp. 4-75

RTP and QoS ❒  RTP does not provide any mechanism to ensure

timely data delivery or other QoS guarantees ❒  RTP encapsulation is only seen at end systems (not by intermediate routers) ❍  routers provide best-effort service, making no special effort to ensure that RTP packets arrive at destination in timely manner

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-76

38

RTP entities Different encodings

Multiple streams

End System SSRC = 53 Translator

Mixer SSRC = 19

Single stream

SSRC = 19 CSRC = 53 77

End System SSRC = 77 ❒  End system: application that actually generates/consumes the content

carried in RTP packets ❍ 

SSRC: Synchronisation Source identifier

❒  Translator: intermediate system that changes the encoding scheme without

altering the timing. It may also convert multicast into multiple unicast streams ❒  Mixer: intermediate system that receives multiple streams and combines them in some manner. The new stream has its own timing (new SSRC) ❍ 

CSRC: Contributing Source identifier 4: Multimedia App. & Transp. 4-77

©From Computer Networking, by Kurose&Ross

RTP Header payload type

sequence number type

time stamp

Synchronization Source ID

Miscellaneous fields

Payload Type (7 bits): Indicates type of encoding currently being used. If sender changes encoding in middle of conference, sender informs receiver via payload type field • Payload type 0: PCM µ-law, 64 kbps • Payload type 3: GSM, 13 kbps • Payload type 7: LPC, 2.4 kbps • Payload type 26: Motion JPEG • Payload type 31: H.261 • Payload type 33: MPEG2 video

Sequence Number (16 bits): incremented by one for each RTP packet sent, detects packet loss and restores packet sequence ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-78

39

RTP Header (2) payload type

❒ 

Synchronization Source ID

time stamp

Miscellaneous fields

Timestamp field (32 bytes): sampling instant of first byte in this RTP data packet ❍ 

❍ 

❒ 

sequence number type

for audio, timestamp clock typically increments by one for each sampling period (for example, each 125 µsecs for 8 KHz sampling clock) if application generates chunks of 160 encoded samples, then timestamp increases by 160 for each RTP packet when source is active. Timestamp clock continues to increase at constant rate when source is inactive

SSRC field (32 bits): identifies source of RTP stream. Each stream in RTP session should have distinct SSRC.

4: Multimedia App. & Transp. 4-79

©From Computer Networking, by Kurose&Ross

Real-Time Control Protocol (RTCP) ❒  works in conjunction

with RTP ❒  each participant in RTP session periodically transmits RTCP control packets to all other participants

©From Computer Networking, by Kurose&Ross

❒  each RTCP packet

contains sender and/or receiver reports ❍ 

report statistics useful to application: # packets sent, # packets lost, interarrival jitter, etc.

❒  feedback can be used

to control performance ❍ 

sender may modify its transmissions based on feedback

4: Multimedia App. & Transp. 4-80

40

RTCP: multiple multicast senders sender

RTP RTCP

RTCP

RTCP

receivers ! each

RTP session: typically a single multicast address; all RTP / RTCP packets belonging to session use multicast address ! RTP, RTCP packets distinguished from each other via distinct port numbers ! to limit traffic, each participant reduces RTCP traffic as number of conference participants increases ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-81

RTCP: packet types Receiver Report (RR) packets: ❒  fraction of packets lost, last sequence number, average interarrival jitter Sender Report (SR) packets: ❒  SSRC of RTP stream, current time, number of packets sent, number of bytes sent ©From Computer Networking, by Kurose&Ross

Source Description (SDES) packets: ❒  e-mail address of sender, sender's name, SSRC of associated RTP stream ❒  provide mapping between the SSRC and the user/host name

4: Multimedia App. & Transp. 4-82

41

RTCP: stream synchronization ❒  RTCP can synchronize

different media streams within a RTP session ❒  e.g., videoconferencing app: each sender generates one RTP stream for video, one for audio ❒  timestamps in RTP packets tied to the video, audio sampling clocks ❍  not tied to wall-clock time

❒  each RTCP sender-report

packet contains (for most recently generated packet in associated RTP stream): ❍  ❍ 

timestamp of RTP packet wall-clock time for when packet was created

❒  receivers use association to

synchronize playout of audio, video

4: Multimedia App. & Transp. 4-83

©From Computer Networking, by Kurose&Ross

RTCP: bandwidth scaling ❒  RTCP attempts to limit its

traffic to 5% of session bandwidth Example ❒  one sender, sending video at 2 Mbps ❒  RTCP attempts to limit its traffic to 100 kbps ❒  RTCP gives 75% of rate to receivers; remaining 25% to sender

©From Computer Networking, by Kurose&Ross

❒  75 kbps is equally shared

among receivers: ❍ 

with R receivers, each receiver gets to send RTCP traffic at 75/R kbps

❒  sender gets to send RTCP

traffic at 25 kbps ❒  participant determines RTCP packet transmission period by calculating average RTCP packet size (across entire session) and dividing by allocated rate

4: Multimedia App. & Transp. 4-84

42

SIP: Session Initiation Protocol [RFC 3261] SIP long-term vision: ❒  all telephone calls, video conference calls take

place over Internet ❒  people are identified by names or e-mail addresses, rather than by phone numbers ❒  you can reach callee (if callee so desires), no matter where callee roams, no matter what IP device callee is currently using

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-85

SIP Services ❒  SIP provides

mechanisms for call setup: ❍  for caller to let callee know she wants to establish a call ❍  so caller and callee can agree on media type, encoding ❍  to end call

©From Computer Networking, by Kurose&Ross

❒  determine current IP

address of callee: ❍  maps

mnemonic identifier to current IP address

❒  call management: ❍  add new media streams during call ❍  change encoding during call ❍  invite others ❍  transfer, hold calls

4: Multimedia App. & Transp. 4-86

43

Example: setting up a call to known IP address Bob

Alice

167.180.112.24

193.64.210.89 INVITE bo [email protected] .2 c=IN IP4 167.180.11 10.89 2.24 m=audio 38060 RT P/AVP 0

port 5060

port 5060

Bob's terminal rings

200 OK .210.89 c=IN IP4 193.64 RTP/AVP 3 m=audio 48753 ACK

port 5060

port 38060

200 OK message indicates his port number, IP address, preferred encoding (GSM)

#  default

SIP port number is 5060

port 48753

time

#  Bob’s

messages can be sent over TCP or UDP; here sent over RTP/UDP

µ Law audio

©From Computer Networking, by Kurose&Ross

SIP invite message indicates her port number, IP address, encoding she prefers to receive (PCM µlaw)

#  SIP

µ

GSM

#  Alice’s

time

4: Multimedia App. & Transp. 4-87

Setting up a call (more) ❒  codec negotiation: ❍  suppose

Bob doesn’t have PCM µlaw encoder ❍  Bob will instead reply with 606 Not Acceptable Reply, listing his encoders ❍  Alice can then send new INVITE message, advertising different encoder

©From Computer Networking, by Kurose&Ross

❒  rejecting a call ❍  Bob

can reject with replies “busy,” “gone,” “payment required,” “forbidden” ❒  media can be sent over RTP or some other protocol

4: Multimedia App. & Transp. 4-88

44

Example of SIP message INVITE sip:[email protected] SIP/2.0 Via: SIP/2.0/UDP 167.180.112.24 From: sip:[email protected] To: sip:[email protected] Call-ID: [email protected] Content-Type: application/sdp Content-Length: 885 c=IN IP4 167.180.112.24 m=audio 38060 RTP/AVP 0

we don’t know Bob’s IP address. -> Intermediate SIP servers needed #  Alice

sends, receives SIP messages using SIP default port 5060 #  Alice

Notes: ❒  HTTP message syntax ❒  sdp = session description protocol ❒  Call-ID is unique for every call ©From Computer Networking, by Kurose&Ross

#  Here

specifies in “Via:” header that SIP client sends, receives SIP messages over UDP 4: Multimedia App. & Transp. 4-89

Name translation and user location ❒  caller wants to call

callee, but only has callee’s name or e-mail address ❒  need to get IP address of callee’s current host: ❍  ❍  ❍ 

user moves around DHCP protocol user has different IP devices (PC, smartphone, car device)

©From Computer Networking, by Kurose&Ross

❒  result can be based on: ❍  time of day (work, home) ❍  caller (don’t want boss to call you at home) ❍  status of callee (calls sent to voicemail when callee is already talking to someone)

Service provided by SIP servers

4: Multimedia App. & Transp. 4-90

45

SIP Registrar registrar ❒  when Bob starts SIP client, client sends SIP REGISTER message to Bob’s registrar server ❒  one function of SIP server:

Register Message: REGISTER sip:domain.com SIP/2.0 Via: SIP/2.0/UDP 193.64.210.89 From: sip:[email protected] To: sip:[email protected] Expires: 3600

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-91

SIP Proxy proxy ❒  Alice sends invite message to her proxy server ❒  another function of SIP server: ❍ 

contains address sip:[email protected]

❒  proxy responsible for routing SIP messages to

callee Bob ❍ 

possibly through multiple proxies

❒  Bob sends response back through the same set of

proxies ❒  proxy returns Bob’s SIP response message to Alice ❍ 

contains Bob’s IP address

❒  SIP proxy analogous to local DNS server plus TCP

setup

©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-92

46

SIP example:

[email protected] calls [email protected]

2. UMass proxy forwards request to Poly registrar server 2 3 UMass SIP proxy

Poly SIP registrar 3. Poly server returns redirect response, indicating that it should try [email protected]

4. Umass proxy forwards request to Eurecom registrar server 4

1. Jim sends INVITE 8 message to UMass SIP proxy. 1

128.119.40.186

7 6-8. SIP response returned to Jim

9 9. Data flows between clients

Eurecom SIP registrar 5. eurecom 5 registrar 6 forwards INVITE to 197.87.54.21, which is running Keith’s SIP client 197.87.54.21

Note: also a SIP ack message from Jim, which is not shown ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-93

Comparison with former H.323 ❒  H.323 is another signaling

protocol for real-time, interactive applications ❒  H.323 is a complete, vertically integrated suite of protocols for multimedia conferencing: signaling, registration, admission control, transport, codecs ❒  SIP is a single component. Works with RTP, but does not mandate it. Can be combined with other protocols, services

©From Computer Networking, by Kurose&Ross

❒  H.323 comes from the ITU

(telephony) ❒  SIP comes from IETF: borrows much of its concepts from HTTP ❍  SIP has Web flavor, whereas H.323 has telephony flavor ❒  SIP uses the KISS principle: Keep It Simple Stupid

4: Multimedia App. & Transp. 4-94

47

Chapter 4: Summary Principles ❒  audio and video coding ❒  multimedia applications types over IP ❍ 

streaming stored audio video, real-time conversational voice/video

❒  UDP versus TCP streaming ❒  making the best of best effort service ❍  ❍  ❍  ❍ 

DASH: Dynamic, Adaptive Streaming over HTTP CDN: Content Distribution Networks adaptive playout delay loss recovery (FEC, retransmissions) and concealment

Protocols ❒  RTSP ❒  RTP/RTCP ❒  SIP ©From Computer Networking, by Kurose&Ross

4: Multimedia App. & Transp. 4-95

How should the Internet evolve to better support multimedia? Laissez-faire Differentiated services philosophy: ❒  just put more capacity where needed ❒  fewer changes to Internet ❒  no major changes in network, infrastructure, yet provide let apps handle it 1st and 2nd class service ❒  content distribution networks, application-layer multicast Integrated services philosophy: ❒  fundamental changes in Internet so that apps can reserve end-to-end bandwidth ❒  requires new, complex software in hosts & routers ©From Computer Networking, by Kurose&Ross

What’s your opinion? 4: Multimedia App. & Transp. 4-96

48