Guidelines for Evaluating Information Technology Solutions for Customs Luc De Wulf Monday, May 02, 2005 Information technology (IT) plays a key role in operating a modern customs administration. Its efficiency depends on the close alignment and integration between the functions and services performed by the customs administration and the information systems relied upon to undertake these responsibilities. When the IT system is expected to integrate all members of the trading community into one network, the objectives are broader still and the partners multiple, as is the case in a truly “One Stop” Border post approach. While the focus of this Note is on the IT solution designed for a traditional stand alone customs administration the approach presented here can easily be used for the broader concept of trade management ICT systems. The choice for an IT solution for Customs should be driven by the idea that it supports modern customs practices and controls based on a realistic and practical set of transition targets and objectives. Clearly IT solutions should serve the modernization effort and not drive it. Several Customs administrations are seeking WB assistance in the evaluation of their IT solutions and in their search for the most appropriate package to replace the IT support in place that is not adequately supportive of modern effective and efficient custom operations. During the preparation of IT Chapter of the Customs Modernization Handbook (CMH) considerable though was given to the preparation of such an evaluation methodology.1 This Note is inspired by that earlier work. The IT chapter in the CMH (Chapter 13) spells out the advantages and disadvantages of a bespoken IT solution for Customs and briefly presented the off the shelf systems available at t he time of writing. However, the chapter did not dwell on the selection and evaluation criteria that could guide this choice, nor it provide clear guidelines on how to select an off-the shelf system. Yet, the evaluation and selection of the appropriate IT is a recurring issue in customs modernization projects. Bank staff is repeatedly asked to provide guidance. We hope that this Note provides such guidance.

1. Proposed Methodology At present the Customs rely on a number of different IT solutions to assist them and the traders in clearing goods for imports and exports and for managing the national customs procedures. Some rely on a bespoken system, i.e. a system that was developed specifically for roll out and adoption within a given national Customs organization. Others have acquired systems from service providers that make their IT solution available for a consideration. UNCTAD has developed its ASYCUDA customs management system that is provides free to UNCTAD 1

Luc De Wulf and Jose B. Sokol, Editors, Customs Modernization Handbook, World Bank, Washington D.C. 2005. 1

member countries; UNCTAD will support the introduction with dedicated technical assistance and can advise on the acquisition of the necessary hardware. These services however come at a cost. Other vendors sell their software and can advise on hardware, and have programs to assist with the roll out and implementation. As part of the background research and analysis performed in development Chapter 13 of the CMH, a baseline survey was conducted of the application and technology providers active in support of customs administrations in developing nations. The objective of the baseline survey was to develop a comprehensive and objective view of the specific information system platforms for customs administrations currently offered by software and service providers in the commercial market. The baseline survey also included several internally developed systems implemented by national customs administrations in selected countries including Senegal and Morocco. In addition, the survey included several national efforts to deploy commercial systems where significant customization was required to meet local needs. This survey was based on a especially designed methodology, which will be explained in this Note. The methodology rated the various solutions surveyed. Basically, a scoring model of 0-100 points-was used to assess each component of the IT solution surveyed. For each survey component the score 100 was used as a benchmark of being fully consistent and supportive of best practices for customs modernization as defined by World Customs administration and Chamber of Commerce Guidelines, and as inspired by the assessment of leading practitioners in both customs modernization and information technology planning (this norm is referred to later in the Note as the WCO Norm). The results obtained were meant to illustrate the relative capabilities of the providers and their products and services as a snapshot of the market and the solutions/options available. These ratings have no hard scientific basis, as they were derived from interviews with the service providers and only a narrow sample of Customs IT users. The details of these surveys therefore were tentative and preliminary and are not available for dissemination. This treatment of the data is also consistent with WB practice of not disclosing evaluations of commercially available solutions. Figure 1 Customs Information Technology Provider Baseline Survey Provider Strategy Functional Architecture Provider Rating

Application/Solution Architecture Solution Provider

Baseline Survey

Software Infrastructure

Baseline Rating

Baseline Average WCO Best Practice

re Technology Infrastructure Implementation & Deployment Deployment Resources Deployment & Delivery Costs

2

Figure 1 describes the approach adopted. Each IT solution was subjected to survey questions that can be summarized under eight headings: Provider strategy, Functional Architecture, Application/Solution Architecture, Software, Technology Infrastructure, Implementation and Deployment, Deployment resources and Costs. Each of these eight main elements of the IT solution then is rated on the score 1-100. These ratings are then consolidated across the various components and compared with the average of all providers surveyed, with the WCO Best Practice Norm, and amongst each other. 2. Detailed discussion of the Survey Components 2.1. Provider Strategy These component seeks the primary reasons behind the decision, objectives and goals to commit the resources and effort needed to offer a customs information technology services to the international marketplace, his experience of designing and building implementation capacity of the product and his customer base. These issues are detailed in Figure 2. Figure 2. Baseline Survey: Provider Strategy Vision & Principles

Vision

Solutions & Services

Solutions

Capacity Bldg

Capacity Building & Assistance Provider Strategy

Baseline Survey Value & Impact

Experience & Expertise

Stability & Investment

Customers & Installed Base

2.2. Provider Functional Architecture:

3

Baseline Rating

Value

Experience

Stability

Customers

This section reviews whether the proposed solution deals with the various functional requirements of a modern customs management system. These functions are derived from the WCO revised Kyoto Convention. Some of the solutions reviewed provided most of the functionalities of a modern customs management system, while others provided partial solution. The review needs to assess to what extend the various functionalities are fully develop and correspond to best practices. Also, the survey can assess to what extend the solution can be complemented with other modules provided by other providers (e.g.. risk analysis, MIS, temporary admission and bonded warehouses). Only a detailed review of these functionalities will permit to fill in the scores. A partial list of the functionalities is provided in Figure 3. . Figure 3 Baseline Survey Process: Functional Architecture Data & Information Management

Functional Architecture

Baseline Survey

Data Mgmt

User Functions & Tools

Users

Passenger Processing

Passengers

Goods Declaration & Processing

Declarations

Import-Export-Transit Processing

Import/Export

Cargo Management & Control

Cargo Mgmt

Pre-Shipment Inspection Port/Border Administration

Process Management & Workflow

Payment Processing

Revenue Accounting Valuation/Tariff/Duties & Tax

Risk Management Audit Based Controls

4

Baseline Rating

Pre-Shipment Port Mgmt

Workflow Payments

Accounting Valuation

Risk Controls

2.3. Application and Software Architecture The application and software architecture is the description of how the basic building blocks of the system (i.e. user interfaces, application program modules and databases) have been developed and assembled to execute the functions and processes of the Customs ICT system. The application and software architecture questions can be derived from the WCO Kyoto guidelines for Customs ICT deployment along with established best practices for current standards for distributed information system architectures and software infrastructures. Figures 4 and 5 provide the details of these survey elements.

Figure 4. Baseline Survey Process: Application Architecture

Application/Solution Architecture

Baseline Survey

Application/Solution Components

Components

Application/Solution Complexity

Complexity

Application/Solution Scalability

Scalability

Application/Solution Performance

Baseline Rating

Performance

Security

Application/Solution Security

Volatility

Application/Solution Volatility

Figure 5 Baseline Survey Process: Software Infrastructure

Software Infrastructure

Baseline Survey

Application/User Interface

Application Client

Application/Process Server

Application Server Database

Data Management Platform Application/Process Integration Application/Process Customization

2.4. Provider Technology Infrastructure

5

Baseline Rating

Integration Customization

This covers the hardware Platform and Network infrastructure of the customs ICT system. The hardware and technology platform infrastructure is the description of the computer systems (i.e. personal computers, wireless computing devices, application and data servers, and operating systems) used to implement and operate the customs ICT system (Figure 6).

Figure 6. Baseline Survey Process: Technology Infrastructure

Technology Infrastructure

Baseline Survey

User/Client Hardware Platform

User Platform

Application/Data Server Platform

Server Platform Network

Data Communications Platform Platform Configuration Mgmt

Baseline Rating

Configuration System Mgmt

System Management & Security

2.5. Project Implementation and Deployment While it important that the product is technically sound and able to support to the customs processes, it is equally important that the vendor or product originator has the capacity to implement and deploy the product in a timely manner, within budget and with the necessary on the ground support for implementation by customs staff and stakeholders. The issues to be covered in this part of the survey include the process for project initiation, stakeholder participation, and governance. Specific questions cover resource management, project planning, impact on customs administration and operations, the methodologies used to document the functional and process requirements for the ICT solution and the review process to measure alignment with customs administration needs. For details see Figure 7.

Figure 7 Baseline Survey Process: Project Implementation and Deployment

Project management (initiation support, ongoing support, stakeholders participation 6 Design, development and delivery

Project management

Development and delivery Documentation

2.6, Cost: Design, Deployment and Implementation

This section aims at providing a realistic view of the funding required for analysis, design, development, implementation, and operation of ICT systems for customs administration and operations. In addition to cost data for the system, it is also important to understand the role and commitment of funding sources and the nature of the funding provided (i.e. loans/grants from donor organizations, ICT service/technology provider investment, government sources, etc.). Fugure 8.

Figure 8 Baseline Survey Process: Cost: Design, Deployment and Implementation Software development and delivery costs

Software development

Technology Infrastructure

Technology Infrastructure Costs Cost: Design, Deployment and Implementation

Baseline Survey

Support and training costs

Baseline Rating

Training

Operational funding and cost options

Operations

System lifecycle costs

System lifecycle

3. Conclusion

7

The exercise described above substantially relies on recourse to sound judgment. Similarly the interpretation requires weighing of one dimension against the other before an informed decision can be made. Hence it should be clear that the exercise described in his Note does not lend itself to mathematical precision. Yet, combined with the procurement guidelines discussed in Chapter 13 of the CMH, the Guidelines can streamline the evaluation of the various IT solutions under investigation and make the exercise more transparent and rational.

8

Annex 1. n Example When all systems included in the survey are scored it is possible to present the results in a cobweb figure as below. Such a visual presentation can be presented for each vendor and for the home build solutions. It portrays (i) the eight dimensions of the survey, (ii) the results of adhering to the WCO best practice, (iii) the scores ofr Provide 1, and (iv) the average scores for the group of service providers. The cobweb presentation below is only an illustration, but is suggestive of what the exercise can provide. It clearly shows that providers can be much better that the average on certain dimensions of the comparison, while lacking on others

Figure 9. Customs ICT Baseline: Provider 1

Provider 100.00

80.00

Costs

Provider 1

Functions

60.00 40.00

WCO Best Practice

20.00

Resources

0.00

Architecture

Baseline Avg.

Software

Implementation

Technology

Source: DAP Strategic Solutions © 2003

9