Pseudo-regulation F: EDI communication in the electricity market
Appendix report 2: Principles and rules of acknowledgement
MarchJune 20110 Version 12.0
16-6-2010
1.0 2.0 REV.
DESCRIPTION
24-6-2010 27-6-2010
DATE
BOO
MBN
JHH
25-2-2011
1-3-2011
22-3-2011
DATE
FSD
CCO
JHH
NAME
PREPARED
CHECKED
REVIEWED
NAME
APPROVED
64819-10 DOC. NO.
© Energinet.dk
Pseudo-regulation F, Appendix report 2
Table of contents 1.
Objective........................................................................................ 3
2.
Principles and rules of acknowledgement ............................................ 4 2.1 Terms, acknowledgement and error message levels.................. 4 2.2 Generic acknowledgement flow .............................................. 6 2.3 EDIFACT acknowledgement (APERAK) ....................................10 2.4 XML acknowledgements .......................................................13
Doc. 64819/10, Case 10/3365
2/16
Pseudo-regulation F, Appendix report 2
1.
Objective
This appendix report to Pseudo-regulation F describes the current rules and principles for using acknowledgements in message communication in the electricity market. The purpose of using acknowledgements is to allow the sender and receiver to indicate whether a particular message has been received and/or is being processed and whether or not the process has been positive (successful). Correct use of acknowledgements is of major importance for the confidence in the DataHub as the acknowledgements build confidence in the electronic data interchange in the electricity market. The acknowledgements ensure quality as they reveal any errors that may have occurred – and thus allow the sender and the receiver to correct the errors before problems occur in the business layer. To ensure that electronic messages are sent and received correctly and that any errors are identified and remedied, players must comply with the rules for using acknowledgements and error messages as described in this document. The appendix report describes the principles and rules for using the formats EDIFACT and XML acknowledgements. At the end of the document, the difference between the two formats is described.
Doc. 64819/10, Case 10/3365
3/16
Pseudo-regulation F, Appendix report 2
2.
Principles and rules of acknowledgement
This section describes the principles and flow of acknowledgements. Basically, the terms and principles for using acknowledgements are the same for EDIFACT and XML. Both types of acknowledgements are implemented in the same way in the DataHub, and as set out is this appendix report, the difference between the two formats is mainly found in syntax.
2.1
Terms, acknowledgement and error message levels
Three overall acknowledgement levels are used. Figure 1 below describes the connection between the terms used in the principles of acknowledgement.
Begreber og niveauer for kvitteringer Niveauer for kvitteringer
Modtagelseskvittering
Meddelelse (header-niveau)
0..*
Indholdskvittering
Transaktion
Dokument Forretningsdokument
Figure 1: Terms and acknowledgement levels
[Translation to be inserted in the figure above: Begreber og niveauer for kvitteringer = Terms and levels of acknowledgement Niveauer for kvitteringer = Levels of acknowledgement Modtagelseskvittering = Acknowledgement of receipt Indholdskvittering = Processability error report Forretningsdokument = Business document Transaktion = Transaction Meddelelse (header-niveau) = Message (header level) Dokument = Document] The figure uses three terms to describe the various transaction abstraction layers: 1) The term 'transaction' covers an interchange protocol containing the message and any number of documents. 2) The 'message' describes general information (header information) applying to all underlying documents such as sender and receiver. 3) The document is the repeated message information such as one time series out of all the message time series.
Doc. 64819/10, Case 10/3365
4/16
Pseudo-regulation F, Appendix report 2
««The acknowledgement levels mentioned in Figure 1 are specified below. The principles are identical – irrespective of data format. However, the detailed rules differ for XML and EDIFACT. The following sections are therefore divided into specific rules for EDIFACT and XML. Acknowledgement of receipt Description The web service received always makes a syntax and structure validation upon receipt. This takes place by means of interchange in the formats EDIFACT and XML and is described in appendix report 4 to Market Regulation F.
Message type EDIFACT and XML – same response in the receipt situation
««The web service replies immediately after receipt of the message in the same web service session with a positive or negative reply (can be seen from the red, broken-line box in Figure 2). The acknowledgement of receipt is not a document but a reply. Alternatively, the web service call is interrupted with an exception, and a fault value is therefore not returned. XML schemas are used for XML validation which – depending on use – has a number of advantages compared to EDIFACT syntax validation. XML validation is more comprehensive than EDIFACT, which allows more accurate validation. Processability error report Description The processability error report is used to acknowledge the contents at document level, e.g. validation of a metering point ID. A processability error report is only sent if a message fails in the contents validation. Normally, only negative processability error reports are sent.
Message type EDIFACT: Negative APERAK XML: Acknowledgement
In some cases, a positive processability error report can be used. The use of this will always be described in 'Business processes for the Danish electricity market'. As a rule, a processability error report must be sent within one hour of receipt of any message. Business document Description
Doc. 64819/10, Case 10/3365
Message type
5/16
Pseudo-regulation F, Appendix report 2
A business document is a reply to a query message and has a business document layout. Message replies will not be addressed any further in this appendix report.
2.2
Specified in 'EDI transactions for the Danish electricity market'
Generic acknowledgement flow
This section describes the message flow and the corresponding acknowledgement flow to and from the DataHub. 2.2.1 Message sent to the DataHub Figure 2 below describes the flow for acknowledgement exchange in connection with the exchange of messages (EDIFACT and XML) between a player and the DataHub. The flow ends when the DataHub can process the message without generating errors or when the player processes the error message received. The message received may be one of several messages included in the overall business process ('Business processes for the Danish electricity market'). The acknowledgement process applies to each individual message. The steps/phases shown in Figure 2 are described in more detail in Table 1.
Figure 2: Generic acknowledgement flow – The player sends a message to the DataHub
[Translation to be inserted in the figure above:
Doc. 64819/10, Case 10/3365
6/16
Pseudo-regulation F, Appendix report 2
Afsendende aktør = Sending player DataHub’en = DataHub Sender meddelelse = Sends message Synkron webservicekald = Synchronous web service call Modtager meddelelse = Receives message Syntaksvalidering = Syntax validation Negativ/positiv = Negative/Positive Modtagelseskvittering = Acknowledgement of receipt Negativ kvittering = Negative acknowledgement Fejlbehandling = Error processing Pkt. = Step Behandling af meddelelse = Message processing Indholdsvalidering = Contents validation Negativ indholdskvittering = Negative processability error report Afslut = End] The table below describes the five steps shown in Figure 2. #
Name
Description
1
Initiating message
All acknowledgement processes begin with an initiating message sent by a player. The player opens a DataHub web service session which is not closed until the DataHub has sent an acknowledgement of receipt (positive or negative). The extent of the web service session is indicated in Figure 2 by a red, broken-line box.
2
Is the sender known?
The DataHub must be able to identify the sender (player) and validate the sender against approved senders registered in the DataHub. The sender validation process has the following two possible outcomes: -
-
3
Syntax validation?
Unknown sender: The sender (player) is invalid or unknown. In such cases, the web service ends with a negative acknowledgement of receipt (see step 3 – failed validation). Known sender: The sender (player) is valid. In such cases, message processing continues.
The DataHub validates the message received for syntax and structure errors. The syntax validation process has the following two possible outcomes: -
-
Doc. 64819/10, Case 10/3365
Validation failed. The DataHub sends a positive acknowledgement of receipt which makes a unique reference to the initiating message, ensuring that the player can easily identify the negatively validated message. Validation OK. The DataHub sends a positive acknowledgement of receipt which makes a unique reference to the initiating message, ensuring that the player can easily identify the positively validated
7/16
Pseudo-regulation F, Appendix report 2
#
Name
Description message. Regardless of the outcome of the validation, the DataHub always sends one acknowledgement of receipt in the same web service session as opened by the sender. When the syntax validation has been sent, the web service session is closed, and the remaining acknowledgement flow is asynchronous.
4
Message processing / processability error report
Once the initiating message has been validated OK, the DataHub processes the contents of the message and makes contentsw validation against the message received. The contents validation process has the following two possible outcomes: -
-
Validation failed. The DataHub sends a negative acknowledgement of receipt to the player containing a reference to the initiating message and the specific document generating errors. The acknowledgement of receipt specifies the error and its location in the message/documents. Validation OK. The DataHub registers the message received as positively validated, and the acknowledgement flow ends. If the message content is validated OK, the acknowledgement process ends, and the message contents are processed further, see 'Business processes for the Danish electricity market'.
The DataHub only sends negative processability error reports regardless of the message format (EDIFACT and XML), and only one processability error report is sent for each message. 5
Error processing
The players involved in a data interchange process with the DataHub are always under an obligation to respond to negative acknowledgements of receipt and processability error reports. After receipt of a negative acknowledgement, the players must be able to identify the message generating the error and initiate error correction - possibly in cooperation with Energinet.dk.
Table 1: Description of the acknowledgement flow from the DataHub
2.2.2 Message sent from the DataHub Below, the message and acknowledgement flow from the DataHub is shown in Figure 3 and described in Table 2.
Doc. 64819/10, Case 10/3365
8/16
Pseudo-regulation F, Appendix report 2
Modtagne aktør
DataHub’en Start
Pkt. 2. Meddelelse behandles
Initierende meddelelse
Send meddelelse
Pkt. 1.
Afslut
Pkt. 2. Indholdsvalidering?
Negativ indholdskvittering
Negativ
Modtager meddelelse Pkt. 3.
Positiv Afslut
Fejlbehandling
Afslut
Figure 3: Generic acknowledgement flow – the DataHub sends a message to the player
[Translation to be inserted in the figure above: Modtagne aktør = Receiving player DataHub’en = DataHub Pkt. = Step Meddelelse behandles = Message processed Indholdsvalidering = Contents validation Negativ indholdskvittering = Negative processability error report Positiv/negativ = Positive/negative Initierende meddelelse = Initiating message Send meddelelse = Send message Modtager meddelelse = Receives message Fejlbehandling = Error processing Afslut = End] The table below describes the four steps shown in Figure 3.
#
Navn
Beskrivelse
1
Initiating message
The DataHub sends a message to the player (in practice the DataHub sends the message to the player’s message queue on the DataHub which the player is responsible for emptying at regular intervals). In the event that the receipt of messages from the DataHub fails in terms of syntax, Energinet.dk must be contacted. It is not possible for the player to reply with an acknowledgement of receipt.
Doc. 64819/10, Case 10/3365
9/16
Pseudo-regulation F, Appendix report 2
#
Navn
Beskrivelse
2
Message processing?
Once the initiating message has been received, the player processes the contents of the message and makes contents validation against the received message. The contents validation process has the following two possible outcomes: -
-
Validation failed. The player sends a negative acknowledgement of receipt to the DataHub containing a reference to the initiating message and the specific document generating errors. The acknowledgement of receipt specifies the error and its location in the message/documents. Validation OK. The player registers the message received as positively validated, and the acknowledgement flow ends. If the message contents are validated OK, the acknowledgement process ends.
The player can only send negative processability error reports regardless of the message format (EDIFACT and XML), and only one processability error report is sent for each message. 3
Error Processing
The DataHub places (negative) processability error reports in an error queue which Energinet.dk is responsible for processing.
Table 2: Description of the acknowledgement flow to the DataHub
2.3
EDIFACT acknowledgement (APERAK)
This section describes the EDIFACT processability error report APERAK and how it is used. In addition to the acknowledgement of receipt, which is sent directly through the web service1, the DataHub may send a negative APERAK (processability error report). This is used if message validation fails according to the rules specified in the relevant business transaction ('EDI transactions for the Danish Electricity market'). The negative APERAK makes rejections at message and/or document level and is used as reply to a received failing EDIFACT message (see Figure 1 for a description of the levels in a transaction). The business contents of the message and its underlying documents will be validated on the basis of the rules specified in the relevant business transaction ('EDI transactions for the Danish electricity market').
1
Formerly EDIFACT CONTRL, which is no longer used
Doc. 64819/10, Case 10/3365
10/16
Pseudo-regulation F, Appendix report 2
-
-
In the event of errors in the general information at message level, which are decisive for the underlying documents, the entire message is rejected, including any underlying documents. If the general message level is validated OK, each document in the message must be validated and rejected in case of errors.
If the received EDIFACT message contains several documents (such as time series), the sender can choose whether DataHub will send one overall APERAK for all documents is to be sent or whether one APERAK is to be sent for each document. 2.3.1 Description of APERAK The negative APERAK contains different data depending on where in the EDIFACT message the error is – eg whether the error is in the header section or at document level. Errors at header level The following conditions will be validated at header level: The message ID has not previously been received. The required attributes as specified in the business transaction are present, including UNB attributes. The version of the implementation guide (UNH/S009) matches the business transaction. The message function is consistent with the business transaction. The DataHub is capable of processing documents for 'interchange recipient'. The encoded value is included in the list of accepted values. In the event of header-level errors, processing will stop, and the negative APERAK will be generated and sent according to the following rules: Segment group. segment.element BGM.4343
Value
Description
27
Rejected.
SG3.ERC.C901.9321
Error code (see above table) Text description
The error code must appear from the element in the ERC segment.
SG3.FTX.C108.4440
SG1.RFF.C506.1154
Doc. 64819/10, Case 10/3365
Message ID
Error description in Danish and English separated by a slash (“/”). The name of the element containing an error (eg a message date). Contains a reference to the message ID from the message validated.
11/16
Pseudo-regulation F, Appendix report 2
Example of an APERAK acknowledgement specifying a header error: UNA:+.? ' UNB+UNOCW:3+5790000701278:14+5790000432752:14+100718:1447+ 4471' UNH+1+APERAK:D:09B:UN:E5DK03' BGM+294+12435ID+27' DTM+137:201007181245:203' DTM+735:?0000:406' RFF+ACW:7179' NAD+MS+5790000701278::9' NAD+MR+5790000432752::9' NAD+DDQ+5790000432752::9' ERC+D02:DK' FTX+AAO+++Forkert meddelelsesnavn / Wrong Message Name' UNT+11+1' UNZ+1+4471' Errors at document level Messages containing repeated documents, such as time series in UTILTS, must be acknowledged individually. The sender can either send one APERAK for each document containing errors or one overall APERAK (no positive APERAKs are sent). It will be validated that: -
the the the the
required attributes as specified in the business transaction are present. encoded value is included in the list of accepted values. values are correctly formatted as specified in the implementation guide. attributes observe the number of repetitions.
Irrespective of the number of errors, all documents will be validated, and an APERAK will subsequently be sent, providing detailed information about such errors, according to the following rules: Segment group. Value Description segment.element BGM.4343 34 Approved with comments SG1.RFF.C506.1154
Message ID
Contains a reference to the message ID from the message validated.
SG3.ERC.C901.9321
Error code (see above table) Text description
The error code must appear from the element in the ERC segment. Error description in Danish and English separated by a slash (“/”). The name of the element containing an error (eg a start date). Based on the relevant business transaction, it can be established what to use in the APERAK.
SG3.FTX.C108.4440
SG4.RFF.C506.1154
Doc. 64819/10, Case 10/3365
Transaction ID, series ID or time series ID
12/16
Pseudo-regulation F, Appendix report 2
The player receiving a negative APERAK is always under an obligation to respond (see Table 1).
2.4
XML acknowledgements
This section describes the XML acknowledgements used and the principles for using them. 2.4.1 XML processability error report The XML processability error report (also called acknowledgement) is used if the receiver of a message cannot make a positive message validation. This means that the semantic interpretation of data contains errors. The receiver is then under an obligation to send a negative processability error report to the sender, specifying the cause of the error, within one hour. The procedure for when the processability error report must be sent is described in chapters 2.1.1 and 2.2.2. As the APERAK, the XML processability error report can indicate errors at message and document level (see Figure 1 for a description of the levels in a transaction). By means of the element 'OriginalBusinessDocumentReferenceIdentityDocumentIdentification', the XML processability error report makes a unique reference to the message which cannot be processed and the specific reason. If specific documents or time series contain errors, the acknowledgement is supplemented by a repeating group in the processability error report uniquely identifying the specific document/time series.
All error descriptions have two elements as shown in Figure 4.
Figure 4: XML elements for error indication
The element 'StatusTypeReasonCode' is required and indicates by code the cause of the error. The error code may be supplemented by 'ResponseReasonTypeReasonText', but this is not required. The complete XML processability error report is described in 'EDI transactions for the Danish electricity market'.
Doc. 64819/10, Case 10/3365
13/16
Pseudo-regulation F, Appendix report 2
2.4.2 Acknowledgement of submission of notifications and schedules This section contains a specific description of the rules for XML acknowledgements used in connection with the submission of notifications and schedules.
Doc. 64819/10, Case 10/3365
14/16
Pseudo-regulation F, Appendix report 2
The following two types of acknowledgement are used for XML messages: Level Message level (syntax error report/model error report) Document level (syntax error report/model error report)
Type of acknowledgement Processability error report (Acknowledgement)
Only one type of acknowledgement is used in an XML context, reflecting the use of ETSO’s Acknowledgement Document2. The message will be validated on receipt against an XML schema and with contents validation. An acknowledgement document will subsequently be returned. This document will be positive or negative depending on the outcome of the validation. All messages/documents must thus be acknowledged by one and only one acknowledgement document referring to the original message. When a player calls an Energinet.dk web service, the XML message received will be validated against the XML schema related to the current web service. The validation process will check whether the XML message is well formed, ie whether its contents complies with the W3C standard for XML message structure. A check will also be performed to see whether the elements making up the XML message are permitted according to the XML schema. Furthermore, the schema may also define rules for the contents of the individual elements. If the XML content is not consistent with the related XML schema, a negative acknowledgement will be returned, and the acknowledgement process will end. If schema validation generates no problems, the next step will be to validate the semantics (the outcome of application validation). The result is a negative acknowledgement if the application check or the content somehow does not comply with the rules specified for semantics (see the validation table for the current business process). At message level, the acknowledgement will contain general information about the message (message ID, sender and receiver) as well as a reason consisting of a code and a relevant description. When an acknowledgement document is used as a positive acknowledgement, the reason will be attached with reason code A01 (message fully accepted). The acknowledgement thus serves three purposes:
-
2
Inform the sender that the message contents have been rejected because of a syntax error (message level).
http://www.entsoe.eu/fileadmin/user_upload/edi/library/acknowledgementv5r0/Acknowledgement-v5r0.pdf
Doc. 64819/10, Case 10/3365
15/16
Pseudo-regulation F, Appendix report 2
-
Inform the sender that the message has been successfully received and that processing the entire message will continue (message and document level).
-
Inform the sender that the target application has received the individual message or parts of the message and provide details of content errors, if any.
Doc. 64819/10, Case 10/3365
16/16
