A Custom Fair Usage System for RESNET: Design and Implementation Nick Burd, Head of Technical Infrastructure, Lancaster University [email protected] Matthew Weaver, Network Specialist, Lancaster University [email protected]

Background • RESNET – Approximately 6800 bedrooms – 100M Ethernet connectivity – Exists as a separate network with connectivity to the campus network through a border router

• Achieve balance between providing secure access to campus services, the web, while deterring use of peer-to-peer applications. • In September 2008, we modified our firewall policy from ‘default deny’ to ‘default allow’ to enable applications such as Skype – Big spike in download volumes – Big increase in copyright violations

Introducing a Fair Usage Policy • April 2010 – a new Fair Usage Policy for RESNET is approved – https://resnet.lancs.ac.uk/fair_usage

• The University’s recommendation was to implement the policy without needing any capital investment – integrate a fair usage system into our existing systems and network

• Key points from the policy: – access to university resources and general web browsing should be unlimited – a monthly per-user quota allowance should be introduced to ensure a fair share of the available Internet bandwidth amongst all users

Network topology LU Border Router Juniper M320

Campus Core Network

3G

10G

10G

CANLMAN JANET Internet

10G

Campus Core Routers Juniper MX-960s

148.88.0.0/16

RESNET Border Router Juniper M7i

1G 10G NAT

 A new 10G link was provisioned directly between RESNET and Campus Core Routers

1G

Default route sends all other traffic to RESNET Border Router via NAT RESNET Core Router Brocade BigIron RX4

 Link brought into OSPF as point-to-point /30 10G

10G

 Traffic from RESNET clients to campus servers prefers the more specific route  Juniper firewall policy implemented on both sides of the link to control traffic flow

1G

ResNet Aggregation Routers Brocade BigIron RX4s Brocade FastIron X-Series

10.34.0.0/16

RESNET

Deciding on a fair quota • RESNET traffic data was gathered from April 2010 onwards • Fair usage policies from comparable institutions were evaluated – lack of clarity as to exactly what traffic was governed by a quota – varying quota sizes and representation, e.g. monthly, weekly

• Analysis of data showed a 30GB monthly quota for combined upload and download would only affect about 8% of users • •

Over the period April to June 2010 a 30GB quota was exceed 1566 times Of these, ‒ 185 (12%) used 30 – 35 GB, ‒ 411 (26%) used 35 – 50GB and ‒ 970 (62%) exceeded 50GB

Month

Total Users

Over Quota

%

April 2010 May 2010 June 2010 Average

6455 6573 6339 6456

454 628 484 522

7.00% 9.60% 7.60% 8.10%

Design features and decisions • Easy for users to track their quota usage – online portal with clear graphs showing cumulative usage – helpful e-mail warnings at key usage milestones

• Easy for Service Desk staff to see when users are over or near their quota limit – for efficient handling of related queries at 1st line – tight integration with existing helpdesk tools

• Flexible quota amounts – temporary per-user increases for academic use when appropriate

• Rate-limiting should be non-prohibitive – important for voice/video chat and the student experience

Enforcing the policy/quota 1

Netflow data collected

RESNET Border Router

RESNET NAT

Flows Server

2

Hourly bandwidth usage exported to RESNET database and merged with user records

3

RESNET Database Server

RESNET Network Access Control Servers

IP addresses of users who are over quota are added to a rate-limiting firewall

Enforcing the policy/quota • Per-user traffic data for upload and download is aggregated each hour and added to a running total for the current month • Once users exceed 30GB, their IP addresses are automatically applied to a firewall rule on the RESNET Border Router – – – –

line speed is reduced to 500Kbps download, 100Kbps upload e-mail sent to notify user that their connection has been limited connections remain limited for the remainder of the month all devices validated against the user are limited, so the system cannot be bypassed by simply validating a new computer

• Only traffic going via NAT to off-site is limited – so 100Mbps line speed is maintained to vital university services, including browsing via the web proxy, filestore access and VLE

Encouraging fair use • Users can track their quota usage in near real-time via a web portal • E-mails are sent to users to notify them of their quota usage – At 25% and 50% utilisation, but only if they are likely to exceed the 30GB quota during the month (i.e. they have used more than 7/14GB in the first 7/14 days) – At 75%, 90% and 100% utilisation – Each night if their utilisation for the past 24 hours is greater than 1GB, and if more than 50% of that was used for uploading

Results – traffic reduction Term 1

Term 2

Term 3

Term 1

1G link NAT link from RESNET Border Router

10G bypass link provisioned

Fair usage system went live

• 45% reduction in download traffic • 75% reduction in upload traffic

Term 2

Results – quota usage analysis • March 2011 vs. February 2012 March 2011

February 2012

% of Quota Used

Number of Users

% of Users

Number of Users

% of Users

< 25% 25% to 50% 50% to 75% 75% to 90% 90% to 100% > 100%

5979 73 26 159 94 427

88.5% 1.0% 0.4% 2.4% 1.4% 6.3%

4808 654 478 188 116 404

72.3% 9.8% 7.2% 2.8% 1.7% 6.1%

• Most users actually use less than 25% of their quota • % of users who use more than 30GB is still only 6% – 2% less compared to the estimated 8% from initial research – But, this number is rising…

So, what did it cost? • Nothing….. – well, not exactly

• Engineer resource time = approx. 50 man days – network planning and maintenance to relieve congestion – analysis of traffic data to determine a fair quota amount – software development to design and integrate the fair usage system into our existing RESNET systems

• Network cost – associated interface cost of provisioning a new 10G link

• Software license for ChartDirector graphing software = £75

Conclusions • Some users just do not care about the quota limitation – they repeatedly go over quota on 1st of every month – suggests that heavy users leave their peer-to-peer clients running

• The system has been very effective in increasing our bandwidth capacity to RESNET – both from internal University servers and the Internet

• Monitoring quota usage allows us to better understand how our users are using their connections – an important tool to aid future capacity planning for RESNET and LU

• Little or no effect on the number of copyright infringement notices received by the University – a possible reduction in copyright notices was anticipated – in reality this has not been achieved as a result of introducing fair usage