Resource Usage Service Performance Analysis Author: John Ainsworth, Manchester Computing, The University of Manchester. Revision: 2 Date: 9th March 2004

Introduction We have characterised the performance and scaling properties of the WS-I Resource Usage Service that has been developed as part of the DTI e-Science project “Market for Computational Services”. This document captures the results of that analysis, and is intended to serve as a reference for decision makers who are considering deploying the Resource Usage Service as part of an accounting system.

Test Setup The Resource Usage Service (RUS) is a web service. It uses the Xindice XML database as its underlying persistent storage, which is also a web service. Both web services were running in the same container for these tests. The RUS is secured using SSL, which is configured for mutual authentication. The container used was the Sun Application Server, which is available as part of J2EE. Two machines with different specifications were used to run the same test so that a performance comparison can be made. The specification of each ofthese machines is shown in Table 1. Table 1 Test Server Specifications Processor type and speed No of processors RAM Operating system Disks Container

XML Database

Test Server 1 Intel 3.06GHz 2 4GB Redhat Enterprise 3 2x120GB Sun Java System Application Server Platform Edition 8.0.0_01 Xindice 1.1b4

Test Server 2 Intel 3.00 GHz 1 512MB Fedora Core 1 1x25GB Sun Java System Application Server Platform Edition 8.0.0_01 Xindice 1.1b4

The RUS was fed Usage Records, which had been produced by the CSAR machines at Manchester, by a spooler written in Perl. A base set of 3000 records were used, with the urwg:GlobalJobId appended with the count of the number of iterations over this set, and the ds:X509SubjectName was appended as a random string1. This guarantees that each record will be unique, and so will not be rejected by the RUS as a duplicate. The spooler inserts 100 records per insert request. The time taken between sending the request and receiving the response is measured by the spooler, and the average time for a single record is determined from this elapsed time. The spooler and the RUS run on different machines. The networking latency is assumed to be constant. 1

The records produced by CSAR currently do not contain this element, as it can not be passed through the scheduler.

The spooler used in these tests is a modified version of the spooler developed for production use. The differences are that it does not delete the records when they have been inserted and it iterates over the record set. It also modifies the records as described above to guarantee uniqueness.

Tests Insertion time – Test Server 1 The purpose of this test was to characterise how the average insertion time for a record changes as the database increases in size. The spooler was run and the insertion time was recorded at intervals of 1000 records. The resulting plot is shown in Figure 1 for Test Server 1, and again in Figure 2, but using only a subset of the data.

Figure 1 Test Server 1 - Record Insertion Time Against Database Size

Figure 2 Test Server 1 - Record Insertion Time Against Database Size (Restricted Data Set) The insertion time for the RUS shows a slow increase in comparison with the number of records in the database. For a database of 1 million records the insert time is 0.064s, and a database of 2 million records it is to 0.071s. To put this in context, CSAR produces at most 500 records per day. Table 2 Memory Usage Number of Records Average Record Size Size of database file Average size of each index file

1.5 million 1.2 Kb 9.3 Gb 1.9 Gb

2.6 million 1.2 Kb 14.0 Gb 4.2 Gb

Observations During this test the CPU usage on Test Server 1 was observed to be approximately 40-45% on each of the two processors. The points on the plot in Figure 1 which show an insertion time >0.1s are an artefact of the RUS’s optimised insertion procedure, and have been removed from the data set used to produce the plot shown in Figure 2. When a record is inserted, a hash code is computed from the fields which are mandatory for a record (in this case ds:X509SubjetcName, urwg:GlobalJobId, urwg:MachineName, urwg:SubmitHost). The hash code is not unique – multiple records different records may have the same hash code. This hash code is stored with the record. To test if a duplicate record exists, the database is queried to see if a record exists with the hash code of the record

to be inserted. If the hash exists, then the RUS must determine if two different records have the same hash code or whether it is a duplicate record. This process incurs the extra processing overhead. As can be seen from the plot, this occurs infrequently, approximately 0.03% of the time in the test we ran.

Insertion Time – Test Server 2 The results of the insertion time test for Test Server 2 are shown in Figure 3, and again in Figure 4, but with only a subset of data, where all points with an insertion time > 0.3s have been removed from the data set. These plots show a reduced RUS performance in comparison with Test Server 1, and reflect the fact that Test Server 2 is a much lower specification machine than Test Server 1. This test was ended prematurely as Test Server 2 ran out of disk space.

Figure 3 Test Server 2 – Record Insertion Time Against Database Size

Figure 4 Test Server 2 - Record Insertion Time Against Database Size (Restricted Data Set)

Multiple Clients The purpose of this test is to verify the operation of simultaneous readers and writers operating on the RUS. We have successfully used 10 insertion clients operating with 2 extraction clients and 2 deletion clients. The insertion clients are as used in the insertion time tests. The extraction client queries the RUS for 10 records by rus:RUSId, with values chosen at random. The deletion client deletes 10 records per request by specific rus:RUSId, with values chosen at random.