A quick tutorial on IP Router design

SH w i i t c hg i n hg a nP d R e o u tr i n f g

o

r

m

a

n

c

e

1

Outline • • • •

Where IP routers sit in the network What IP routers look like What do IP routers do? Some details:

– The internals of a “best-effort” router • Lookup, buffering and switching

– The internals of a “QoS” router

• Can optics help?

2

1

Outline (next time) • • • •

The way routers are really built. Evolution of their internal workings. What limits their performance. The effect that DWDM is having on switch/router design. • The way the network is built today. • Discussion: The scope for optics 3

Outline • • • •

Where IP routers sit in the network What IP routers look like What do IP routers do? Some details:

– The internals of a “best-effort” router • Lookup, buffering and switching

– The internals of a “QoS” router

• Can optics help?

4

2

The Internet is a mesh of routers (in theory) IP Core router

The Internet Core IP Edge Router

5

What do they look like?

Access routers e.g. ISDN, ADSL Core router e.g. OC48c POS

Core ATM switch

6

3

Basic Architectural Components of an IP Router Routing Protocols Routing Table

Forwarding Switching Table

Control Plane

Datapath” per-packet processing

7

Per-packet processing in an IP Router 1. Accept packet arriving on an incoming link. 2. Lookup packet destination address in the forwarding table, to identify outgoing port(s). 3. Manipulate packet header: e.g., decrement TTL, update header checksum. 4. Send packet to the outgoing port(s). 5. Buffer packet in the queue. 6. Transmit packet onto outgoing link. 8

4

Outline • • • •

Where IP routers sit in the network What IP routers look like What do IP routers do? Some details:

– The internals of a “best-effort” router • Lookup, buffering and switching

– The internals of a “QoS” router

• Can optics help?

9

Basic Architectural Components Datapath: per-packet processing

1. Ingress

2. Interconnect

3. Egress

Forwarding Table

Forwarding Decision Forwarding Table

Forwarding Decision Forwarding Table

Forwarding Decision

10

5

Forwarding Engine Packet payload

header

Router Routing Lookup Data Structure

Destination Address

Outgoing Port

Forwarding Table Dest-network

Port

65.0.0.0/8

3

128.9.0.0/16

1

149.12.0.0/19

7 11

The Search Operation is not a Direct Lookup

Memory

Data

Address

(Incoming port, label)

(Outgoing port, label)

IP addresses: 32 bits long ⇒ 4G entries

12

6

The Search Operation is also not an Exact Match Search Exact match search: search for a key in a collection of keys of the same length. Relatively well studied data structures: • Hashing • Balanced binary search trees

13

Example Forwarding Table

Destination IP Prefix

Outgoing Port

65.0.0.0/8

3

128.9.0.0/16

Prefix length

142.12.0.0/19

1 7

IP prefix: 0-32 bits

65.0.0.0/8

0 65.0.0.0

128.9.0.0/16

128.9.16.14 224 65.255.255.255

142.12.0.0/19

232-1 14

7

Prefixes can Overlap Longest matching prefix

128.9.176.0/24

128.9.16.0/21 128.9.172.0/21 65.0.0.0/8

0

128.9.0.0/16

142.12.0.0/19

128.9.16.14

232-1

Routing lookup: Find the longest matching prefix (aka the most specific route) among all prefixes that match the destination address. 15

Prefix Length

Difficulty of Longest Prefix Match 32

24

128.9.176.0/24 128.9.16.0/21

128.9.172.0/21 142.12.0.0/19 128.9.0.0/16

8

65.0.0.0/8

128.9.16.14

Prefixes 16

8

Lookup Rate Required Year

Line

Line-rate 40B (Gbps) packets (Mpps)

1998-99 OC12c 1999-00 OC48c 2000-01 OC192c

0.622 2.5 10.0

1.94 7.81 31.25

2002-03 OC768c

40.0

125

31.25 Mpps ⇒ 33 ns DRAM: 50-80 ns, SRAM: 5-10 ns 17

Number of Prefixes

Size of the Forwarding Table 100000 90000 80000 70000 60000 50000

10,000/year

40000 30000 20000 10000 0

95

96

97

Year

98

Source: http://www.telstra.net/ops/bgptable.html

99

00 18

9

Basic Architectural Components Datapath: per-packet processing

1. Ingress

2. Interconnect

3. Egress

Forwarding Table

Forwarding Decision Forwarding Table

Forwarding Decision Forwarding Table

Forwarding Decision

19

Interconnects Two basic techniques

Input Queueing

Usually a non-blocking switch fabric (e.g. crossbar)

Output Queueing

Usually a fast bus 20

10

Interconnects Output Queueing

Individual Output Queues

Centralized Shared Memory Memory b/w = 2N.R

1 2 N

1 2 Memory b/w = (N+1).R

N 21

Interconnects

Centralized Shared Memory Input 1

Input 2

Input N

Numerous1work has proven and Output made possible:

– Fairness – Delay Guarantees – Delay Variation Control – Loss Guarantees Output 2 – Statistical Guarantees

Large, single dynamically allocated memory buffer: Output N per “cell” time N writes N reads per “cell” time. Limited by memory bandwidth. 22

11

Output Queueing

How fast can we make centralized shared memory? 5ns SRAM Shared Memory

• 5ns per memory operation • Two memory operations per packet • Therefore, up to 160Gb/s • In practice, closer to 80Gb/s

1 2 N 200 byte bus

23

Interconnects

Input Queueing with Crossbar Memory b/w = 2R

Data In

Arbiter

configuration

Data Out

24

12

Input Queueing

Delay

Head of Line Blocking

Load

58.6%

100%

25

Head of Line Blocking

26

13

Input Queueing

Virtual output queues

27

Input Queueing

Delay

Virtual Output Queues

Load

100%

28

14

Input Queueing Virtual output queues

Memory b/w = 2R Arbiter

Complex!

29

Other Non-Blocking Fabrics Clos Network

30

15

Other Non-Blocking Fabrics Clos Network

Expansion factor required = 2-1/N

31

Other Non-Blocking Fabrics Self-Routing Networks

000 001

000 001

010 011

010 011

100 101

100 101

110 111

110 111

32

16

Other Non-Blocking Fabrics Self-Routing Networks

The Non-blocking Batcher Banyan Network Bitonic Sorter

Self-Routing Network

3

7

7

7

7

7

7

7

2

5

0

4

6

6

5

3

2

5

5

4

5

2

5

3

1

6

5

4

6

6

1

3

0

3

3

0

1

0

4

3

2

2

1

0

6

2

1

0

1

4

4

4

6

2

2

0

000 001 010 011 100 101 110 111

• Fabric can be used as scheduler. •Batcher-Banyan network is blocking for multicast. 33

Outline • • • •

Where IP routers sit in the network What IP routers look like What do IP routers do? Some details:

– The internals of a “best-effort” router • Lookup, buffering and switching

– The internals of a “QoS” router

• Can optics help?

34

17

Basic Architectural Components Routing Protocols

Admission Control

Reservation

Routing Table

Packet Classification

Forwarding Switching Table

Policing & Access Control

Control Plane

Output Scheduling

Ingress

Interconnect

Egress

1.

2.

3.

Datapath” per-packet processing

35

Basic Architectural Components Datapath: per-packet processing

1. Ingress Classifier Table

Forwarding Table

Policing & Access Control

Forwarding Decision

Classifier Table

Forwarding Table

Policing & Access Control

Forwarding Decision

Classifier Table

Forwarding Table

Policing & Access Control

Forwarding Decision

Limitation: Memory b/w

2. Interconnect

Limitation: Interconnect b/w Power & Arbitration

3. Egress

Limitation: Memory b/w 36

18

Outline • • • •

Where IP routers sit in the network What IP routers look like What do IP routers do? Some details:

– The internals of a “best-effort” router • Lookup, buffering and switching

– The internals of a “QoS” router

• Can optics help?

37

Can optics help? Cynical view: 1. A packet switch (e.g. an IP router) must have buffering. 2. Optical buffering is not feasible. 3. Therefore, optical routers are not feasible. 4. Hence, “optical switches” are circuit switches (e.g. TDM, space or Lambda switches). 38

19

Can optics help? Open-minded view: • Optics seem ill-suited to processing intensive functions, or where random access memory is required. • Optics seems well-suited to bufferless, reconfigurable datapaths.

39

Can optics help? Typical IP Router Linecard

Physical Layer

Framing & Maintenance

Lookup Tables

Buffer & State Memory

Packet Processing

Buffer Mgmt & Scheduling

Buffer Mgmt & Scheduling Buffer & State Memory

Buffered or Bufferless Fabric

Arbitration

OC192c linecard: ~10-30M gates ~2Gbits of memory ~2 square feet >$10k cost

Backplane 40

20

Can optics help? • Linecard? – The linecard is processing & memory intensive.

• Interconnect? – Arbitration is very processing intensive. – The fabric can be a bufferless datapath… – How fast can an optical datapath be reconfigured?

41

Outline for next time… • • • •

The way IP routers are really built. Evolution of their internal workings. What limits their performance. The effect that DWDM is having on switch/router design. • The way the network is built today. • Discussion: The scope for optics 42

21