A Custom Fair Usage System for RESNET: Design and Implementation Nick Burd, Head of Technical Infrastructure, Lancaster University
[email protected] Matthew Weaver, Network Specialist, Lancaster University
[email protected]
Background • RESNET – Approximately 6800 bedrooms – 100M Ethernet connectivity – Exists as a separate network with connectivity to the campus network through a border router
• Achieve balance between providing secure access to campus services, the web, while deterring use of peer-to-peer applications. • In September 2008, we modified our firewall policy from ‘default deny’ to ‘default allow’ to enable applications such as Skype – Big spike in download volumes – Big increase in copyright violations
Introducing a Fair Usage Policy • April 2010 – a new Fair Usage Policy for RESNET is approved – https://resnet.lancs.ac.uk/fair_usage
• The University’s recommendation was to implement the policy without needing any capital investment – integrate a fair usage system into our existing systems and network
• Key points from the policy: – access to university resources and general web browsing should be unlimited – a monthly per-user quota allowance should be introduced to ensure a fair share of the available Internet bandwidth amongst all users
Network topology LU Border Router Juniper M320
Campus Core Network
3G
10G
10G
CANLMAN JANET Internet
10G
Campus Core Routers Juniper MX-960s
148.88.0.0/16
RESNET Border Router Juniper M7i
1G 10G NAT
A new 10G link was provisioned directly between RESNET and Campus Core Routers
1G
Default route sends all other traffic to RESNET Border Router via NAT RESNET Core Router Brocade BigIron RX4
Link brought into OSPF as point-to-point /30 10G
10G
Traffic from RESNET clients to campus servers prefers the more specific route Juniper firewall policy implemented on both sides of the link to control traffic flow
1G
ResNet Aggregation Routers Brocade BigIron RX4s Brocade FastIron X-Series
10.34.0.0/16
RESNET
Deciding on a fair quota • RESNET traffic data was gathered from April 2010 onwards • Fair usage policies from comparable institutions were evaluated – lack of clarity as to exactly what traffic was governed by a quota – varying quota sizes and representation, e.g. monthly, weekly
• Analysis of data showed a 30GB monthly quota for combined upload and download would only affect about 8% of users • •
Over the period April to June 2010 a 30GB quota was exceed 1566 times Of these, ‒ 185 (12%) used 30 – 35 GB, ‒ 411 (26%) used 35 – 50GB and ‒ 970 (62%) exceeded 50GB
Month
Total Users
Over Quota
%
April 2010 May 2010 June 2010 Average
6455 6573 6339 6456
454 628 484 522
7.00% 9.60% 7.60% 8.10%
Design features and decisions • Easy for users to track their quota usage – online portal with clear graphs showing cumulative usage – helpful e-mail warnings at key usage milestones
• Easy for Service Desk staff to see when users are over or near their quota limit – for efficient handling of related queries at 1st line – tight integration with existing helpdesk tools
• Flexible quota amounts – temporary per-user increases for academic use when appropriate
• Rate-limiting should be non-prohibitive – important for voice/video chat and the student experience
Enforcing the policy/quota 1
Netflow data collected
RESNET Border Router
RESNET NAT
Flows Server
2
Hourly bandwidth usage exported to RESNET database and merged with user records
3
RESNET Database Server
RESNET Network Access Control Servers
IP addresses of users who are over quota are added to a rate-limiting firewall
Enforcing the policy/quota • Per-user traffic data for upload and download is aggregated each hour and added to a running total for the current month • Once users exceed 30GB, their IP addresses are automatically applied to a firewall rule on the RESNET Border Router – – – –
line speed is reduced to 500Kbps download, 100Kbps upload e-mail sent to notify user that their connection has been limited connections remain limited for the remainder of the month all devices validated against the user are limited, so the system cannot be bypassed by simply validating a new computer
• Only traffic going via NAT to off-site is limited – so 100Mbps line speed is maintained to vital university services, including browsing via the web proxy, filestore access and VLE
Encouraging fair use • Users can track their quota usage in near real-time via a web portal • E-mails are sent to users to notify them of their quota usage – At 25% and 50% utilisation, but only if they are likely to exceed the 30GB quota during the month (i.e. they have used more than 7/14GB in the first 7/14 days) – At 75%, 90% and 100% utilisation – Each night if their utilisation for the past 24 hours is greater than 1GB, and if more than 50% of that was used for uploading
Results – traffic reduction Term 1
Term 2
Term 3
Term 1
1G link NAT link from RESNET Border Router
10G bypass link provisioned
Fair usage system went live
• 45% reduction in download traffic • 75% reduction in upload traffic
Term 2
Results – quota usage analysis • March 2011 vs. February 2012 March 2011
February 2012
% of Quota Used
Number of Users
% of Users
Number of Users
% of Users
< 25% 25% to 50% 50% to 75% 75% to 90% 90% to 100% > 100%
5979 73 26 159 94 427
88.5% 1.0% 0.4% 2.4% 1.4% 6.3%
4808 654 478 188 116 404
72.3% 9.8% 7.2% 2.8% 1.7% 6.1%
• Most users actually use less than 25% of their quota • % of users who use more than 30GB is still only 6% – 2% less compared to the estimated 8% from initial research – But, this number is rising…
So, what did it cost? • Nothing….. – well, not exactly
• Engineer resource time = approx. 50 man days – network planning and maintenance to relieve congestion – analysis of traffic data to determine a fair quota amount – software development to design and integrate the fair usage system into our existing RESNET systems
• Network cost – associated interface cost of provisioning a new 10G link
• Software license for ChartDirector graphing software = £75
Conclusions • Some users just do not care about the quota limitation – they repeatedly go over quota on 1st of every month – suggests that heavy users leave their peer-to-peer clients running
• The system has been very effective in increasing our bandwidth capacity to RESNET – both from internal University servers and the Internet
• Monitoring quota usage allows us to better understand how our users are using their connections – an important tool to aid future capacity planning for RESNET and LU
• Little or no effect on the number of copyright infringement notices received by the University – a possible reduction in copyright notices was anticipated – in reality this has not been achieved as a result of introducing fair usage