Guidelines for Electronic Data Interchange
September, 2016
The views expressed in this report do not necessarily reflect the views of the United States Agency for International Development or of the United States Government.
1
Table of Contents Abbreviations..................................................................................................................................................... 3 1.
Preface ....................................................................................................................................................... 5
2.
General project approach for BiH.............................................................................................................. 6
3.
Objectives of this report ............................................................................................................................ 7
4.
EDI Framework .......................................................................................................................................... 8 4.1
Introduction ....................................................................................................................................... 8
4.2
Identification of roles and domains................................................................................................... 9
4.2.1
DSO Identification directly issued by FERC, RSERC and SERC .................................................... 9
4.2.2
Supplier Identification ............................................................................................................. 11
4.2.3
Metering point Identification .................................................................................................. 12
4.2.4
BRP Identifications................................................................................................................... 14
4.2.5
Check digit ............................................................................................................................... 14
4.3
Communication means and file names ........................................................................................... 14
4.3.1
File Transfer Protocol (FTP) ..................................................................................................... 14
4.3.2
Web services ............................................................................................................................ 14
4.3.3
Simple Mail Transfer Protocol (SMTP)..................................................................................... 14
4.3.4
Recommendation .................................................................................................................... 15
4.3.5
File names ................................................................................................................................ 15
4.4
Data exchange ................................................................................................................................. 15
4.4.1
Acknowledgement and error messages .................................................................................. 17
4.4.2
Message types ......................................................................................................................... 17
4.4.3
Message time, date and time period ...................................................................................... 17
4.4.4
Message structure and content............................................................................................... 19
4.5
DSO Metering Point Database ......................................................................................................... 23
4.5.1 5.
Brief functional description of the database ........................................................................... 24
Roadmap for development of EDI in BiH................................................................................................. 26
2
Abbreviations BiH
Bosnia and Herzegovina
BRP
Balance Responsible Party
CET
Central European Time
CEST
Central European Summer Time
CSV
Comma-Separated Values
DSO
Distribution System Operator
EBIX
European forum for energy Business Information eXchange
EDI
Electronic Data Interchange
EIC
Energy Identification Coding
ENTSO-E
European Network of Transmission System Operators
EIA
Energy Investment Activity
EU
European Union
F-GCS
General Conditions for Electricity Supply
FERC
Regulatory Commission for Energy in the Federation of Bosnia and Herzegovina
FTP
File Transfer Protocol
FTPS
FTP Secure
GIAI
Global Individual Asset Identifier
GLN
Global Location Number
GMT
Greenwich Mean Time
GSRN
Global Service Relation Number
ID
Identifier
INVOIC
Invoice message
ISO
Independent System Operator
IT
Information Technology
LIO
Local issuing office
kW
Kilowatt
kWh
Kilowatt Hour
3
MIME
Multipurpose Internet Mail Extensions
MSCONS
Metered Services Consumption Report
MWG
Market Working Group
OBIS
OBject Identification System
R-GCS
General Conditions for Delivery and Supply of Electricity
RSERC
Republic of Srpska Regulatory Commission for Energy
SERC
State Electricity Regulatory Commission
SFTP
Secure (or SSH) File Transfer Protocol
SMTP
Simple Mail Transfer Protocol
USAID
U.S. Agency for International Development
UN/EDIFACT
United Nations Electronic Data Interchange for Administration, Commerce and Transport
UTC
Coordinated Universal Time
UTILMD
Utility master data message
XML
EXtensible Markup Language
4
Preface USAID’s Energy Investment Activity project is supporting Bosnia and Herzegovina (BiH) in its efforts to increase investments in the energy sector and to advance the accession process related to energy sector requirements. In particular, the Energy Investment Activity project aims to address impediments to increased investments in the energy sector, to address retail market deficiencies, to achieve energy savings in BiH using regulatory incentives and to advance EU accession requirements in the energy sector. In order to assist the electricity sector in BiH to make further necessary steps toward the establishment of a fully transparent retail electricity market, USAID funded the technical assistance project, Energy Investment Activity (EIA), with component 2 “Address retail Market Deficiencies in BiH.” EIA proposed the constitution of a Market Working Group (MWG) and pertinent subgroups:
Subgroup 1 – deals with activities related to development of DSO rules and processes Subgroup 2 – deals with activities related to the development of Electronic Data Interchange
Assistance to work of Subgroup 1 resulted in the report “Outline of Distribution Network Rules.” This report dealt with development of an information model and business processes linked with the role of the DSO as independent market facilitator; the report is expected to serve as a guideline for development of the rulebooks1 and other legal documents that should cover business processes associated with DSO market roles. In continuation of the work, as a result of assistance to the efforts of Subgroup 2, this report – Guidelines for Electronic Data Interchange (hereinafter EDI) report has been prepared. EDI can be described as computer to computer exchange of business documents in standard electronic format within a business process between different parties. Using as an input international best practices from one side and a developed information model and business processes 2 from the other side, EDI infrastructure is reviewed and recommendations specified. This report considers the actual situation in BiH related to existing EDI between key electricity retail market players. As a result, the report suggests options and specifies recommendations that should facilitate development of technical standards for EDI. The review and analysis EDI for the traditional role of the DSO (for example: network operation, maintenance, development, etc.) is not a subject of this document, as these processes are already well covered with existing material.
1
These rulebooks will be developed by the working group and could constitute future sections of Distribution Network Rules that relate to DSO market roles. 2 In the “Outline of Distribution Network Rules” document.
5
General project approach for BiH The main objective of Subgroup 2 is development of standards for electronic data interchange (EDI) in the retail market, which implies a sequence of messages between market participants (in our case mainly DSO and Supplier, but also the BRP and ISO 3 ), either of whom may serve as the originator or recipient. Implementation of these standards should enable the transmission of messages (formatted data messages, representing the documents) from sender to recipient via telecommunications facilities with a high degree of automation.4 The automation enables efficient processing of a large amount of data with minimal human interaction. Human intervention in the processing of received messages is typically intended only for error conditions, for quality review, and in special situations. As a precondition for the development of the technical standards, the work of Subgroup 1 related to an information model and business processes development needed to be finished and information model and business processes defined. An information model, as a common language for all participants, shall be used for clear and unambiguous definition of EDI guidelines. These guidelines shall take into consideration business processes. The impact of the current situation in EDI at key retail market players shall be assessed and taken into consideration.
Information model
Business processes
International best practice
Current EDI situation EDI Guidelines
Figure 1 Influence on development of EDI Guidelines
3
Regardless the fact that ISO is not a part of retail electricity market, there is significant data exchange between the ISO and DSO related to imbalance settlement process. 4 Technology that provides automation includes computers and other communications devices that can gather, store, manipulate, prepare and distribute data.
6
Options for issues of particular interest shall be specified with elaborated recommendation based on a best practice and current situation. Guidelines for technical standards shall provide guiding principles and recommendations for messages/file formats and content, communication means/infrastructure, security measures, etc.
Objectives of this report As full retail market opening5 occurred in BiH on the 1st of January 2015, the issue of establishment of the DSOs as independent retail market facilitators with a suitable infrastructure for data exchange become urgent. The objective of this report is to provide support to the work of the MWG and Subgroup 2 in Development of technical standards for data exchange in the market by specifying these Guidelines for EDI. The support is necessary for the setup of the EDI in a competitive market. Based on the Guidelines for EDI, the MWG and Subgroup 2 will be able to develop a detailed and sound set of rules for EDI, which comply with today’s best practices and are accommodated to the situation in BiH. The mentioned set of rules will additionally improve transparency and contribute to the efficiency of electricity retail market in BiH. As Guidelines for EDI are based, inter alia, on the current situation in BiH, review and evaluation of current practices shall be included in document. This development will, as expected, have as an additional result capacity strengthening of the MWG and Subgroup 2 participants in the EDI domain.
5
Introduction of free choice of supplier for all customers.
7
EDI Framework 4.1 Introduction Quick, reliable and secure data exchange is a priority in modern electricity markets. Business processes that are implemented in retail electricity markets are usually composed of a number of steps, of which the majority falls into the category of data exchange. One party sends data, and another party (or parties) receives data. Therefore, the definition of Electronic Data Interchange (EDI) framework for data exchange of the relevant business services (existing and missing) is of utter importance. Such framework shall be based on a modern IT infrastructure for data exchange, as well as on the definition of messages (and their structures, contents and formats) that will be exchanged. Additionally, it is necessary to specify a simple and efficient pattern/system for the identification of roles (actors, market participants) and domains6 (metering points) in the electricity market. By assigning unique and unambiguous identifiers to the each object (market participant, metering point, and the like), the exchanged messages coming to the “proper” addresses, related to the “proper” objects is assured. The transfer of messages is usually realized using standardized Internet protocols. There are several typical solutions that will be presented in this document. The right solution for communication protocols depends on the parties involved in the business process data exchange. The structure and content of messages will be explained and illustrated in this chapter. Although not directly linked, an adequate database (metering, market participant) has significant influence on the usability of EDI. A database is the collection of data that is organized in a suitable way that enables quick data access and management. An example of a logical database table structure will be presented in this chapter as well. The general recommendations for development of the EDI framework are:
Delivery of proxies -To enable a highly automated process, the delivery of proxies (physical copy, etc.) should be avoided as a rule, and the existence of proxies should be guaranteed in a contract. Only in justified individual cases, transmission of proxies is required. Deadlines - In case of a requirement of an authorization or declaration for which the requesting party has deadlines, this cannot be in question in the process. The requesting party may only cancel the running process if the requested party does not respond immediately. Communication between Supplier and Customer - The communication between supplier and customer is not the primary subject of the processes, and no guidelines regarding EDI will be developed/described for it. This communication may be freely negotiated (according the legislation).
6
Roles and domains are already defined as a part of information model in Outline of Distribution Network Rules document.
8
4.2 Identification of roles and domains The unique identification of market participants (roles) and domains (e.g., metering point) in the electricity market is crucial for market functioning. Without the possibility to uniquely and unambiguously identify the DSO, Suppliers or metering points, data exchanged within business process between sender and receiver would be unrecognizable and automation not possible. Since there is no uniform European standard7 for identification, we suggest the assignment of identifiers as described below. Pease note that identification is related to a role. This implies, for example, if during a transitional period the DSO and the Supplier are in one company (not unbundled), nevertheless they need to have two IDs – one for the role as DSO and one for the role as Supplier. In the following, we will consider identifications that need to be unique during their life time and centrally assigned. Other identifications that may change several times (e. g., customer ID) are not part of these considerations. Once assigned, the DSO identification may not be changed during the validity period of the license. Options for identification schemes are considered in the continuation of this chapter. 4.2.1
DSO Identification directly issued by FERC, RSERC and SERC
Each DSO must have a unique Identification. Since the DSO must be licensed, and the regulators (FERC, RSERC and SERC) are issuing licenses, they can be responsible for issuing a unique DSO identification also. 4.2.1.1 Custom identification scheme Using a custom identification scheme, the DSO ID can be assigned as follows:
Digit 1 and 2 contain the country identification (fix “BA”) Digit 3 contains a “D” for DSO Digit 4 to 8 contain a consecutive number (filled with zero from the left) Example:
Digits
Country
Consecutive No.
B
A
D
0
0
1
2
3
1
2
3
4
5
6
7
8
7
Further recommendations for identification, see ebIX “ Recommended identification schemes for the European energy industry” http://www.ebix.org/content.aspx?ContentId=990&SelectedMenu=3.
9
4.2.1.2 GS1-GLN identification scheme The GS1 system 8 provides standard numbering structures for different applications, which guarantees worldwide uniqueness within the relevant area of application. The main idea is that the identification of parties and locations are unique and as stable as possible over time. This system encompasses a number of codes that are used for different purposes. Parties in the electricity market (and the DSO as well) are identified via Global Location Number (GLN) codes. These codes are composed of thirteen (13) digits:
GS1 company prefix – has variable size and is administered by GS1. The first three (3) digits is the GS1 prefix of national organization (which is for BiH 387) Location reference – has variable size and is assigned by company. Usually, it is a sequential number. Last digit – thirteenth digit is the check digit and is an integral part of the location number.
Example
3
8
7
1
2
3
4
5
6 7
8
9
1
Digits
1
2
3
4
5
6
7
8
9 0
1
2
3
In total, the GS1 company prefix and Location reference together have 12 digits. In the example above, the GS1 company prefix is presented in blue, the location reference in orange, and the check digit in grey. Companies may apply for different “capacities” (containing 10, 100, 1000 or more unique codes), depending on their needs and requirements. 4.2.1.3 EIC identification scheme An alternative assignment of the DSO ID can be done according the energy identification coding scheme (EIC) issued by ENTSO E, consisting of:
8
2 Digits: Local issuing office (LIO) number, for BiH the ISO is registered as LIO for BiH by ENTSO E (the LIO number 36 was assigned to NOS BiH). 1 Digit: Code type, in this case EIC object type X (Party). 12 digits: uppercase characters or minus signs allocated by the LIO in compliance with general and local rules to identify the object. 1 digit: check character based on the 15 previous characters used to ensure the validity of the EIC code. The check digit algorithm is described in the EIC implementation guide document.
Global Standards http://www.gs1.org/.
10
Example
3
6
X
D -
Digits
1
2
3
4
-
-
-
-
-
0 0 1 2 3 7
5 6 7 8 9 0 1 2 3 4 5 6
4.2.1.4 Recommendation As DSOs in BiH do not have assigned identifications, our recommendation is to assign EIC codes for them. 4.2.2
Supplier Identification
Each Supplier must have a unique Identification. Since Suppliers must be licensed, and the regulators (FERC, RSERC and SERC) are issuing these licenses, we suggest that they be responsible for issuing a unique Supplier identification also. Once assigned, the supplier identification may not be changed during validity of the license. Also, this identification should not change if the supplier company merges or splits. 4.2.2.1 Custom identification scheme Using a custom identification scheme, the Supplier ID could be assigned as follows:
Digit 1 and 2 contain the country identification (fix “BA”) Digit 3 contains a “S” for supplier Digit 4 to 8 contain a consecutive number (filled with zero from the left) Example:
Digits
Country
Consecutive No.
B
A
S
0
0
1
2
3
1
2
3
4
5
6
7
8
4.2.2.2 GS1-GLN identification scheme The same identifier scheme which is used for a DSO - Global Location Number (GLN) - could be applied for identification of the suppliers that are participating in the electricity market. As described above, the code is composed of 13 digit number with last digit in the structure check digit. 4.2.2.3 EIC identification scheme Again, an alternative assignment of the Supplier ID can be done according the energy identification coding scheme (EIC).
11
Example
3
6
X
S
-
-
-
-
-
-
0 0 1 2 3 6
Digits
1
2
3
4
5 6 7 8 9 0 1 2 3 4 5 6
The ISO in BiH has already issued a number of EIC codes,9 and it administers a registry of market participants and balance responsible parties. 4.2.2.4 Recommendation Numerous EIC-X10 codes have been already issued by ISO for participants in the wholesale electricity market. This assignment is also part of the Market Rules requirements related to registration of market participants at the ISO. Since this scheme is already successfully implemented, we suggest its extension to participants in the retail market, under responsibility of authorized regulators. Regulators could coordinate implementation of this activity with the ISO. 4.2.3
Metering point Identification
Each Metering point must have a unique Identification. Since the DSO is responsible for a building connection point and supply point and for metering at the metering point, also the DSO can be responsible11 for assigning a unique metering point ID to a metering point. Once assigned, the Metering point identification may not be changed during lifetime of the metering point (normally equal to the lifetime of the building). 4.2.3.1 Custom identification scheme The Metering Point ID could be assigned as follows:
Digit 1 to 8 contain the DSO identification Digit 9 to 28 contain a 20 digit consecutive number
Exampl e:
DSO identification
Consecutive number
B A D 0 0 1 2 3 0 0 0 0 0 0 0 0 0 0 0 1 2 3 4 5 6 7 8 9 Digits
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8
9
ISO registry is available at http://www.nosbih.ba/en/partneri/registers/46. EIC codes for market participants. 11 FERC Article 61 (1). 10
12
4.2.3.2 GS1-GSRN identification scheme For identification of metering (active, passive and virtual) and accounting points, a Global Service Relation Number (GSRN) may be used, as it provides a unique and unambiguous identification number. A GSRN code12 is defined with the code that has 18 digits:
GS1 company prefix – has a variable size and is administered by GS1. First three (3) digits is the GS1 prefix of a national organization (which is for BiH 387). Service reference – has a variable size. Last digit – eighteenth digit is the check digit and is integral part of location number.
Example
1
7
3
5
0
0
5
3
8
5
0
0
0
0
0
0
1
6
Digits
1
2
3
4
5
6
7
8
9
0
1
2
3
4
5
6
7
8
In total, the GS1 company prefix and Service reference together have 17 digits. The GSRN can be encoded in a barcode. 4.2.3.3 EIC identification scheme Again, an alternative assignment of the metering point ID can be done according to the energy identification coding scheme (EIC). In this case the code type “Z” will be used. Example
3
6
Z
1
2 3 -
1 2 3 4 5 6 7 8 2
Digits
1
2
3
4
5 6 7 8 9 0 1 2 3 4 5 6
4.2.3.4 Recommendation For metering points, each DSO already has an existing (one, or even more) identification scheme, which may be different from the identification schemes adopted by other DSOs. DSOs in BiH should have unique and unified identification scheme on the country level. It is recommended that DSOs solve this issue (not in the scope of this project), together as participants of a responsible workgroup.
12
For identification of meters as physical equipment (i.e., not as metering points) in the network, another identifier Global Individual Asset Identifier (GIAI) may be used. It may be up to 30 characters long, encompassing alphanumeric characters.
13
4.2.4
BRP Identifications
This process is already in place in BiH. The ISO, as the local issuing office, issues a unique identifier for BRP, using an EIC identification scheme. The same EIC-X code which is used for a particular market participant is also used for a corresponding BRP if it accepts balance responsibility for a particular balance group. 4.2.5
Check digit
A checksum is a small-sized datum from a block of digital data for the purpose of detecting errors which may have been introduced during its transmission or storage. Check digits are special cases of checksums. If the IDs are not assigned according to EIC or GS1, it must be decided whether the ID’s described above should have a check digit (created by means of a common checksum algorithm) as a last digit or not.
4.3 Communication means and file names There are many options for the exchange of messages between market participants. Internet-based standards, allowing automation of message transmission, receipt, validation and data storage, are used. 4.3.1
File Transfer Protocol (FTP)
A standard network protocol is used to transfer files between market participants via a computer network. FTP is widespread and reliable; but since data traffic is not encrypted, this protocol is today considered unsecure. FTP derivatives (SFTP and FTPS) which encrypt passwords and sensitive information from being transmitted openly over the network are popular for this purpose. 4.3.2
Web services
There is an open XML standard for data interchange from application to application over the Internet. This approach allows market participants to exchange data using web services. When a message is sent to the web service, its syntax is validated and the user is informed if the message is valid and correctly received (accepted by the IT system). Messages are received by calling the web service, which then returns a message indicating it is ready for transmission. Access to web services is commonly subject to strict security rules. 4.3.3
Simple Mail Transfer Protocol (SMTP)
For the transmission of messages SMTP (email) could be used efficiently. When this protocol is used, the message must be in the Multipurpose Internet Mail Extensions (MIME) format, which extends SMTP-enabling additional functionality. The actual message must be sent as an attached file, not as a part of the body text. There will only be one attachment in one SMTP-exchange, the attachment containing only one message type. The usage of SMTP is illustrated by the following Figure 2. The Sender (computer) prepares and sends an email message with the address, subject, body and attached file in “proper” format. The e-mail message is
14
received at the destination by Receiver (computer), which validates the e-mail message and attached files and data, and performs further processing.13
Data file
Sender Receiver E-mail message
Figure 2. E-mail message with attached data file, sent from one market participant to other
All described activities can be achieved in an automated manner. 4.3.4
Recommendation
SMTP could be used efficiently for the transmission of messages in the first set of rules, for the following reasons: 4.3.5
Simple to use Straightforward and easy implementation Widespread and available solution Simple monitoring of messages exchange File names
The name of the attached file (attached to an email using SMTP) shall be according the following
Creation date (CCYYMMDDHHMMSS) Sender (e.g., BAD00123) Receiver (e.g., BAS00123) Message type (e.g., MeterReading)
An example of a complete file name: 20150531181501BAD00123BAS00123MeterReading. Such file names may be automatically generated and parsed by IT systems involved in data exchange, and they are humanreadable.
4.4 Data exchange When data is interchanged between different parties via telecommunication - transmission methods, a “common language" will be used with an agreed mode of expressing it, i.e., common protocols, message identification, agreed abbreviations or codes, etc. Besides using compatible systems, interchange partners should follow uniform rules with respect to communication procedures that include the types of messages
13
For example, enters data into a database or sends acknowledgement or confirmation messages.
15
acceptable, identification of parties, reference to previously agreed protocols or agreements on character set, language, transliteration, and interchange structure.14 As mentioned before, our proposals reference electronic data exchange between the following retail market partners:
Supplier Distribution system operator Balance responsible party
Data exchange is described in the relevant business processes. The exchange of information between customer and supplier or DSO will be explained in the description of the business processes but is not included in our considerations regarding electronic data exchange. All existing and future business processes in the BiH retail market (defined and elaborated in the report “Outline of the Distribution Network Rules”) shall be supported by EDI framework. Detailed definition of nonstandard messages 15 that are transferred between market participants within these processes should be defined in separate document. General overview of EDI interchanges within the retail market business processes is presented in the below table: Business process group
Parties in retail market EDI
Exchanged data
Customer switch
Customers, Suppliers (including Supplier of Last Resort and Public Supplier), DSOs, BRP
Applications, requests, notifications, certificates, proposals, contracts, metering data reads…
Connection contract
Network users, DSOs
Imbalance settlement in retail market
DSOs, Suppliers, BRPs, ISO.
Applications, requests, power consent, (initial) power permits, denials, complaints, decisions, contracts, declarations Aggregated metering data
Table 1 Overview of EDI interchanges
The column “Exchanged data” in the table specifies the types of documents/messages that are exchanged between a retail market participant within one business process group.
14
The principles mentioned above led to the development of the United Nations Electronic Data Interchange for Administration, Commerce and Transport (UN/EDIFACT) syntax rules and standard messages. 15 These are messages different from standard messages, which are described in chapter 4.4.4 Message structure and content.
16
4.4.1
Acknowledgement and error messages
Messages which contain payload data are composed and sent by one market participant and received and accepted (or rejected, if invalid) by another market participant. The flow of these (business) messages may be accompanied by other ancillary messages that contribute to the successful data interchange and facilitate error handling:
Acknowledgement16 or confirmation messages are used for tracking of the main (business) messages in an end-to-end delivery process. They are sent by receiver in order to confirm that message is received and/or validated and accepted by the system. Error or rejection messages are used to assist a sender and receiver in case of error. They are sent by the receiver when error is registered. These messages contain information that should assist the sender to identify and remove error.
The implementation of acknowledgement and error messages is not mandatory for the functioning of EDI, but is recommended. 4.4.2
Message types
The following messages will be exchanged between the market partners: Type
Content
EDIFACT equivalent
Meter Reading
Contains the meter readings, i.e., single meter reading or load profiles.
MSCONS17
Master Data
Contains master data for the customer and contract metering point, e.g., begin supply, end supply.
UTILMD18
Invoice Data
Shall contain the data for billing grid fees.
INVOIC19
Table 2 Message Types
4.4.3
Message time, date and time period
Local time in BiH is depends on the season and may be either standard or daylight savings time.
16
These messages are used for identification if sent message is successfully delivered to the recipient and/or if message is successfully verified and accepted by the system. 17 MSCONS Metered Services Consumption Report. 18 UTILMD Utility master data message. 19 INVOIC invoice message.
17
Standard time in BiH is Central European Time (CET), which it is one hour ahead UTC20 (i.e., UTC+1, where +1 indicates time offset in hours from UTC), and it lasts from the last Sunday in October to the last Sunday in March of the next year. Daylight saving time in BiH is Central European Summer Time (CEST), which is two hours ahead UTC (i.e. UTC+2), and it lasts from the last Sunday in March till the last Sunday in October of the same year.
This time transition complicates the time presentation in messages, as the last Sunday in March has 23 and the last Sunday in October 25 hours. All other days in year have 24 hours. IT systems that are handling messages commonly take these circumstances into account by keeping UTC as their basic time-keeping standard, with the provision of services for automatic time conversion from UTC to local time. This approach provides the exact registration of the time in the exchanged messages, regardless of daylight saving time changes. Time
Content
Number of hours
Standard time (CET)
UTC from 23:00 to 23:00 on the following day
24
Last Sunday in March
UTC from 23:00 to 22:00 on the following day
23
Daylight savings time (CEST)
UTC from 22:00 to 22:00 on the following day
24
Last Sunday in October
UTC from 22:00 to 23:00 on the following day
25
Table 3 Calendar days in BiH in UTC Time
It is also important that IT systems that are generating and exchanging messages between themselves are synchronized as much as possible within a certain range (for example +10 seconds) around local time. Although IT systems may be available for non-stop processing of EDI messages, certain business processes cannot happen in a certain time period. Usually, business days (Monday to Friday), weekends and holidays are defined with strictly limited work time. Depending on the defined limitations, IT systems may (or may not) process the messages, provide IT support, perform error handling, etc. In case a system is not available due to down time, the involved player should notify affected parties promptly. System maintenance should be performed in a manner so as not to disturb market operations.
20
UTC is the same time as GMT (Greenwich Mean Time).
18
4.4.4
Message structure and content
In Europe there are some common message formats based on the EDIFACT (Electronic Data Interchange for Administration, Commerce and Transport) standard. In any case, this standard has been adapted to the needs of the specific region. Same procedure needs to be done for BiH requirements taking into account that in consideration of the actual small number of switching customers in BiH, it may be reasonable to start with a “light” version. This means that instead of using complex EDIFACT standard rules as a basis for the creation of message formats for EDI, it may be just as sufficient to use some simpler file format, such as CSV (CommaSeparated Values) or XML (Extensible Markup Language). A decision on the implemented message format for EDI will have influence on the time and resources for implementation of the necessary software modifications. The selection of message format may depend on several criteria:
Message format complexity. Not all message formats that are in use for EDI are of the same complexity. o The CSV file format is the simplest, where data (number and text) are delimited with a comma (“,”) and stored in plain text form. These files are common message types and are supported by various systems and applications. o XML file format is more complex than CSV. It is a language that defines a set of rules for encoding documents into a format readable by machines and humans. As it is cheap and designed for communication via the Internet, today it is quite popular in EDI. o EDIFACT is widely accepted as an international standard. An EDIFACT document is based on strict rules which manage the position of data within a file. The structure of a file is hierarchical with a lot of elements. Estimated number data exchanges. If it is predicted that a modest number of messages will be exchanged within EDI, it may not be recommended to develop and implement a complex message format such as EDIFACT, as this is not cost-efficient. In-house (in-country) experience and expertise. The experience with data exchange, in particular the EDI format, may have an impact on the selection.
For the sake of convenience, we will use the names of the EDIFACT equivalents in the subsequent texts. Messages in general consist of a Message Header and a Message Body. The message header contains information regarding control of the message flow, the processing of the message body and has a unique structure valid for all message types. The message body contains the real application information and is different from one message type to another.
19
4.4.4.1 Message Header Information
Description
Example / Values
Message type
Identifies the type of the message (body)
UTILMD
Version
Actual version of the format description itself
V1.1
Sender
Identification of the sender of the message
DSO ID
Receiver
Identification of the receiver of the message
Supplier ID
Date
Date when the message was created
CCYYMMDDHHMMSS
Number
Unique number created by the sender of the message
Serial number
Response type code
Code specifying the type of acknowledgment required or transmitted
Not Applicable (No acknowledgement needed)
Table 4 Message Header
Please note: it must be insured that all market participants use the same time base. We suggest using UTC as time base. For BiH this is UTC+1, respectively UTC+2. 4.4.4.2 Message Body: Meter Reading The meter reading message is a message sent between parties in the power industry, providing consumption and/or associated technical information at locations for products or services, where the supply is metered. Regarding meter readings, we must distinguish between the following occurrences of meter readings
A counter reading in kWh monthly drawn from a conventional meter, or A consumption in kWh calculated as difference of two counter readings, or A time series containing power values (kW) or consumption (kWh) measured in a period (e.g., every quarter of an hour) as counter readings.
Information
Description
Example / Values
Metering point ID
Unique ID of metering point
BAD0012300000000000123456789
Address
Address of the metering point
Additionally for clarification
Meter-ID
Unique ID of meter
Usually the serial number of the meter
20
Information
Description
Example / Values
Date of meter reading
The time the meter is read
20150531180000 (31st of May 2015)
Begin period of meter reading
Period for meter reading (CCYYMMDDHHMMSS)
20150501000000 (from 1st of May 2015)
End period of meter reading
Period for meter reading (CCYYMMDDHHMMSS)
20150531240000 (until 31st of May 2015)
Reason
Reason for reading the meter
COM (change of meter)
(CCYYMMDDHHMMSS)
IOM (installation of meter) ROM (removal of meter) COS (change of supplier) COB (change of balancing area) PMR (periodic meter reading) COT (Intermediate meter reading Status
SMV (start measure value) e.g., change of meter
Status of meter reading
EMV (end measure value) e.g., change of meter MRV (meter reading value) e.g., periodical reading Product Identification
Identifies the measured product
OBIS21-Value e. g., 1-b:1.8.0 (active energy import (+))
Quantity Qualifier
Indicates how the quantity has been measured
True Value (relevant for billing) Estimated Value (relevant for billing) Preliminary Value (not relevant for billing)
21
The Object Identification System (OBIS) defines the identification codes (ID-codes) for commonly used data items in electricity metering equipment (IEC 62056-61).
21
Information
Description
Example / Values Not usable value
Unit
Unit of measurement
KWT (kilowatt) KWH (kilowatt hours)
Quality
Quality of measurement
Single counter reading Consumption
Number of values
The number of values transmitted in this message
1 (for a single reading at residential customer) 96 (a load profile with ¼ hours values)22
Quantity
Measured quantity “123460” (with fixed decimal places)
Quantity read from the meter
Table 5 Message Body Meter Reading
4.4.4.3 Message body: Master Data The message can be used for submission of master data regarding metering points
request for supplier switch (DSO to old supplier) confirmation of supplier switch (old supplier to DSO) request for submitting master data (supplier requests of DSO as authorized by the customer)
Information
Description
Example / Values
Transaction
Coding of the reason of this request
Request for master data of the metering point Request for supplier switch Request for confirmation of the supplier switch
Status
22
The status of the answer to a request
Request for supplier switching rejected
In the Nordic countries UTILTS is used for transmitting time series.
22
Information
Description
Example / Values
Metering point ID
The ID of the metering point on which the request refers.
BAD0012300000000000123456789
Customer data
Data of the customer on which the request is related
Name, Address
Metering point data
Data related to the meter point
Address, consumption
See chapter 5.3
Table 6 Message Body: Master Data
4.4.4.4 Message body: Invoice Data This message is used for the transmission of invoices for services and products. In our case, it is used for invoicing grid fees. Since billing of grid fees is not yet clarified, the message structure and content shall not be presented within this document.
4.5 DSO Metering Point Database In our analysis we assume that the model for data exchange is a bilateral model. This means that the exchange of information while executing the business processes is direct and bilateral; the market parties directly send one another standardized messages.23 According to its role as market facilitator, the DSO needs to have a database24 containing all metering points, the customers consuming energy at these points and the supplier which is supplying the metering point. In Figure 3 DSO Metering Point Database, there is a schematic diagram of the data model concept. Keeping a database with this structure enables the DSO to support the business processes (switching, etc.) deliver accurate data (i.e., meter readings) to the authorized parties and to meet its obligations within the legal framework.
23
The bilateral model is not explicitly stated in the regulations of FERC and RSERC; but a reading of them indicates into this direction. 24 According F-GCS Art. 5 (1), F-GCS Art. 61 (2) and R-GCS Art. 9.
23
4.5.1
Brief functional description of the database
The DSO will register all metering points in its network in the database. Once registered, a metering point may not be modified or deleted during its lifetime. The only reason to change a metering point is when the physical condition changes (e.g., when the building is demolished). During the lifetime of the metering point, several customers may be linked to the metering point (e.g.m when a customer moves into the building). The link between customer and metering point has always a start date and an end date. The supplier is linked to the customer and the metering point when the customer concludes a contract of supply with the supplier. Again this link has a start date (commencement of contract) and an end date (termination of contract). The meter installed at the metering point is linked to the metering point. Whenever a meter is read, the meter reading (valid for a certain period) will be recorded. Following this concept, the DSO will be able to know the actual status of the metering point and its actual and past relations at any time.
24
Balance Reponsible Party PK
BPR ID
Is balanced by
Customer PK
Metering Point ID
Supplier Has a conttract on supply
PK
Adress
Supplier ID
Adress
Metering Point Consumption metered via PK
Metering Point ID
Adress
Is installed at
Meter Serial No.
Is part of
Register Register No.
Is part of Meter Reading Date from Date to Value
Figure 3 DSO Metering Point Database
25
supplies
Roadmap for development of EDI in BiH This section presents summary recommendations, which are structured in the form of a roadmap for development of EDI for the retail market in BiH. The roadmap is composed from a set of steps (or activities), which are sorted in chronological order below in list. Steps describe the work to be done by the retail market participants, responsible actors, the estimated duration and expected outcome. The DSOs shall have central role, with necessary involvement of other retail market participants (Suppliers and BRPs), which are the DSO’s counterparties in EDI. Suppliers and BRPs will have an opportunity to participate in activities of concern and to influence decisions and development processes. In addition, they will also have responsibility for implementation of the agreed requirements. The Roadmap is composed of the following steps: 1. Determine communication means – In the beginning, the implementation of SMTP for the exchange of messages is recommended, since this approach is most cost efficient for bilateral data exchange. However, it is necessary to review alternatives. A means for backup communication (phone call or fax) should be determined and should be used in case the main communication means becomes unavailable. o Responsible: Group of experts (IT and power engineering) composed of representatives of retail market participants and regulators. o Duration: 1-2 months. o Outcome: Decision on implementation of SMTP (or other) communication means, and details on implementation of this approach. 2. Unique and unambiguous market participant identification – This process is already implemented in BiH for wholesale market and suppliers, and BRPs are identified using EIC X codes by the ISO, which acts as the Local Issuing Office. This approach is legally binding and stated in the Market Rules. It is necessary to extend this process to cover the retail market as well. Still, it is necessary to review other possibilities and their applicability in this case. o Responsible: Group of experts composed of ISO representatives (as Local Issuing Office for BiH should have a pivotal role), DSOs, and regulators. o Duration: 1-2 month. o Outcome: Decision on identification method for retail market participants. 3. Unique and unambiguous identification of metering point scheme –DSOs in BiH have their own internal schemes (in-house solution on a company level), which are used for identifying metering points. Therefore, it is necessary to jointly (all DSOs in BiH) review the existing identification schemes and agree about a unified identification scheme for the whole country. In case an agreed identification scheme is different (most likely) than the internal coding scheme of a particular DSO, the DSO will be responsible to implement the agreed scheme on external data exchanges.25 25
And do the mapping of external and internal coding scheme where necessary.
26
o o o
Responsible: Group of experts composed of representatives of DSOs and regulators. Duration: 1-2 months. Outcome: Adopted/agreed coding scheme for metering points on the level of country.
4. Definition of messages that are exchanged – The messages are exchanged within the business processes where the DSO acts as “independent market facilitator” and are described in document “Outline of Distribution Network Rules.” This document shall be a starting point for the work on this step. Each and every message should be reviewed, having in mind the sender and receiver and context of business process in which the message is exchanged. o Responsible: Group of experts (power engineering) composed of representatives of market participants. For the purpose of reducing the process duration, the Group workload may be divided into a few subgroups, each dealing with set of co-related processes. o Duration: 2-3 month. o Outcome: Adopt/agree on list of messages for all data exchanges, within business processes described in the retail market. 5. Definition of content (data, list of information) or messages – This step continues the previous step, going into more detail about message structure. Each message that is exchanged within the retail market business processes (defined in the previous step) should have its contents defined26 (i.e. data set/list of info). The message content (data) must be sufficient to provide business process implementation. o Responsible: Group of experts (power engineering) composed of representatives of market participants. For the purpose of reducing the process duration, the Group workload may be divided into a few subgroups, each dealing with a set of co-related processes. o Duration: 3-5 months. o Outcome: Define structured data for each message from the list defined in the previous step. Each message shall be presented on a conceptual level with a list of info (data set). 6. Definition of file formats for messages – Once the content of all messages is defined (in the previous step), it is further necessary to define/develop data file formats in which the data set/list of info shall be “packed” or “embedded.” Initially, a decision on the file format that shall be used shall be adopted. Afterwards, it will be necessary to develop all messages with their contents into a workable file format. o Responsible: Group of experts (IT, power engineering) composed of representatives of market participants. For the purpose of reducing process duration, the Group workload may be divided into a few subgroups, each dealing with set of co-related processes. o Duration: 3-5 months. o Outcome: Decision on the data file type; development and creation of the file formats and templates for EDI exchange as well as a document (manual) for usage of the developed
26
For some messages/business processes, contents (a list of info that shall be exchanged) are already defined in detail in the document “Outline of Distribution Network Rules”; for others this task should be performed within this step.
27
format. Each message shall be presented on paper with the concrete data file, format (EDIFACT, CSV, EXCEL or XML) and values. 7. Definition of necessary software modifications and/or procurement for EDI implementation – It is necessary to identify possibilities of existing systems to implement requirements27 stemming from the above-mentioned activities. Hence, a “gap analysis” should be performed, the result of which will be a functional design of missing/required functionality that will be achieved either with an upgrade of existing or procurement/development of new software platforms as in the next step. o Responsible: Each market participant shall form a group of IT and power engineering experts dedicated for this task. o Duration: 2-3 month (estimate for DSO, depending on the situation). o Outcome: List of basic functional requirements that are needed for exchange messages defined (content, formats, templates) in previous steps. 8. Technical specification, procurement, development, commissioning – Based on the functional design from the previous point, a detailed technical specification for software upgrade/new solution must be developed. Procurement, development and the commissioning of this upgrade/solution needs to be done, selecting one of following three approaches: o In-house development, where a party’s own IT personnel develop the requested software upgrade/new solution. o Development by a local independent software company, where market participant contracts with a software company in BiH to do the job of development and commissioning. o Buying an “off the shelf” solution for the market (if applicable) and adapt it to the situation in BiH. Each market participant shall make a decision in accordance to its own situation and company circumstances, with some kind of cost-benefit analysis in place. o o o
Responsible: Each market participant shall form a group of experts (IT, power engineering, financial, decision makers) dedicated to this task. Duration: 6-9 months (estimate for DSO, depending on the situation). Outcome: Decision on scope and method for software modification/new solution; technical specifications of upgrade/new solution; development and commissioning of software solution based on specifications; working software solution.
9. Test operation – Once a software solution is developed and commissioned, a testing operation of proper work during a specific period is required. During this period, users shall be able to get to use the software, and developers will have opportunity to correct errors and unpredicted behaviour. o Responsible: Each market participant shall form a group of experts (IT and power engineering) dedicated for this task. 27
For example, implementation and translation of identification schemes (new and existing) for metering points, data exchanges in specific file formats, other requirements related to business processes, etc.
28
o o
Duration: 3-6 months (estimate for DSO, depending on the situation). Outcome: Software solution tested and emerging errors and dysfunctionalities corrected.
10. Regular-commercial operation – After a successful testing period, the software solution is ready for regular, day-to-day operation. o Responsible: Each market participant shall form a group of experts (IT, power engineering, and decision making) dedicated for this task. Software operation shall be monitored and the group will suggest potential improvements on a regular basis. o Duration: During software lifetime o Outcome: Stable and reliable operation of a software solution for EDI data exchanges in the retail market in BiH. Development of first three steps (1 – 3) could start at the same time and may work concurrently (in sync), as they relate to basic EDI fundamentals (communication means and identification schemes). Decisions that are adopted through these activities should be issued in a legal document. The next three steps (4 – 6) should be developed sequentially, as the start of next step depends on the completion of the previous step. Development of step 4 may start at the same time as the first three steps, as these processes are not intertwined. EIA will take a lead in coordination of activities described in steps 1 – 6, as these steps require a common understanding and agreement between stakeholders and will result in setting up mandatory standards. Last four steps (7 – 10) should be also developed sequentially, but by each market participant (DSOs, Suppliers and BRPs) individually. The precondition for development of step 7 (and later steps) is the successful implementation of all previous steps. The concerned market participants are responsible for the activities described in steps 7 – 10. The duration of steps is roughly estimated and is an example. It depends on a number of issues, such as the actual situation, organization, efficiency of participants, etc. Before realization of the roadmap activities, the estimated duration of each step should be reviewed in this sense. The estimated summary duration period of all processes in the roadmap ranges from 19 (minimum) to 31 (maximum) months. In Figure 4, the roadmap is presented as a Gantt chart for both the maximum and minimum periods. September 1st was arbitrarily selected as a starting date.
Figure 4 Gantt chart – minimum period
29
Figure 5 Gantt chart – maximum period
30
