FFI-rapport 2014/01454

Enabling service discovery in a federation of systems – WS-Discovery case study

Frank T. Johnsen, Trude H. Bloebaum, and Ketil Lund

FFI

Forsvarets forskningsinstitutt

Norwegian Defence Research Establishment

FFI-rapport 2014/01454

Enabling service discovery in a federation of systems – WS-Discovery case study

Frank T. Johnsen, Trude H. Bloebaum, and Ketil Lund

Norwegian Defence Research Establishment (FFI) 22 September 2014

FFI-rapport 2014/01454 1277

P: ISBN 978-82-464-2434-7 E: ISBN 978-82-464-2435-4

Keywords Service discovery Tjenesteorientert arkitektur Nettverksbasert Forsvar

Approved by Rolf Rasmussen

Project Manager

Anders Eggen

Director

2

FFI-rapport 2014/01454

English summary The NATO Network Enabled Capabilities (NNEC) feasibility study has identified Web services as the key enabling technology for NNEC. This is due to the NNEC demands for interoperability, and the fact that Web services are founded on standards. These standards function well in infrastructure-based networks. Leveraging the technology in tactical networks requires additional attention and often domain-specific optimizations. Due to the dynamic nature of the networkcentric battlefield, mobility and abrupt disappearance of services must be supported. This means that service discovery must be decentralized and robust. Discovery of Web services can be performed either by utilizing a service registry, or a decentralized non-registry based solution. There are three standards, all by OASIS, addressing Web services discovery: The registries, UDDI and ebXML, suffer from liveness and availability problems in dynamic environments. The third standard for Web services discovery is WSDiscovery. It is better suited to dynamic networks than the registries in that it offers a decentralized discovery mechanism, thus removing the single point of failure that a centralized registry constitutes. This report covers work the FFI project “Information and integration services in INI” recently has performed related to service discovery. We have looked into the WS-Discovery standard, and developed an experimental solution which allows this multicast-based standard to function across networks where support for multicast is lacking. The work has been submitted to NATO IST/IST-118 “SOA recommendations for disadvantaged grids in the tactical domain”. Also, we had planned to test it in practice at Unified Vision ’14 (UV14). Sadly, our project’s participation in UV14 was canceled. Thus, this report covers only small-scale testing of the proof-of-concept implementation using FFI’s INI-lab.

FFI-rapport 2014/01454

3

Sammendrag I NATO Network Enabled Capabilities (NNEC) mulighetsstudien ble Web services identifisert som den grunnleggende teknologien for å realisere NNEC. Sentralt i NNEC er interoperabilitet, noe Web services kan understøtte på grunn av omfattende standardisering. Standardene fungerer fint i nettverk med infrastruktur, men å benytte Web services i taktiske nett krever ofte at man tar spesielle hensyn og gjør domenespesifikke optimaliseringer. Stridsteknisk nivå kjennetegnes av mye dynamikk, slik at det nødvendigvis trengs støtte for mobilitet og at tjenester plutselig kan bli utilgjengelige uten forvarsel. Dette betyr at funksjonaliteten for å oppdage tjenester (service discovery) må være desentralisert og robust. Det finnes tre ulike standarder, alle fra OASIS, som kan benyttes for å oppdage Web services i et nettverk. De to registrene, UDDI og ebXML, er sentraliserte løsninger som typisk får problemer med tilgjengelighet og utdaterte data i dynamiske omgivelser. Den tredje standarden er WSDiscovery. Den er bedre egnet for bruk i dynamiske nett enn registrene fordi den tilbyr en desentralisert mekanisme for tjenesteoppdagelse, og dermed ikke er avhengig av tilgjengeligheten av en sentral node som for en registerløsning. Denne rapporten beskriver arbeidet FFI-prosjektet “Informasjons- og integrasjonstjenester i INI” nylig har utført i forbindelse med forskning relatert til tjenesteoppdagelse. Vi har tatt for oss mekanismen WS-Discovery og utarbeidet en eksperimentell løsning som lar denne multicastbaserte mekanismen fungere på tvers av nettverk der multicaststøtte mangler. Arbeidet er spilt inn i NATO STO/IST-118 “SOA recommendations for disadvantaged grids in the tactical domain”, og var tenkt testet i praksis på Unified Vision ’14 (UV14). Prosjektets deltakelse på UV14 ble dessverre avlyst. Rapporten omhandler derfor kun testing av demonstratoren i mindre skala ved bruk av FFIs INI-lab.

4

FFI-rapport 2014/01454

Contents

1

Introduction

7

2

Background

8

2.1

Discovering Web services

8

2.2

Intended application at Unified Vision ’14

11

3

Design and implementation

12

3.1

GUI

14

3.2

P2P module

15

3.3

WS-Discovery module

15

3.4

Service cache

16

4

Results and recommendations

16

5

Conclusion and future work

20

Appendix A Adapting WS-Discovery for use in tactical networks

27

A.1

Introduction

27

A.2

Related work

28

A.2.1

Web services discovery standards

28

A.2.2

Other solutions

29

A.3

Compression

30

A.4

Implementation and evaluation

31

A.4.1

Evaluation framework

31

A.4.2

WS-Discovery network usage evaluation

32

A.5

Conclusion

35

Appendix B CoNSIS: Demonstration of SOA Interoperability in Heterogeneous Tactical Networks 36 B.1

Introduction

36

B.2

Background

36

B.3

Service discovery

39

B.4

Publish/subscribe

42

B.5

Topic handling

45

B.6

Conclusion

47

FFI-rapport 2014/01454

5

Appendix C Topic Discovery for Publish/Subscribe Web Services

48

C.1

Introduction

48

C.2

Related work

49

C.3

Design

50

C.3.1

Adapting WS-Discovery

51

C.3.2

Topic matching

53

C.4

Implementation

53

C.4.1

Experiment: Legacy compatibility

54

C.4.2

Experiment: Automatic topic discovery and subscription

56

C.5

Conclusions and future work

57

6

FFI-rapport 2014/01454

1

Introduction

NATO has identified Web services as the key enabling technology for NATO Network Enabled Capability (NNEC). The technology provides a means to build loosely coupled distributed systems following the principles of Service-Oriented Architecture (SOA). Interoperability is of paramount importance in a federated environment like a coalition network. As a consequence, systems built using the technology should follow the civil interoperability profiles from the Web Services Interoperability Organization (WS-I) as well as the NATO-specific initiatives like the SOA baseline and the Service Interoperability Profiles (SIPs) that are being developed in the TIDE community. The service invocation paradigms (i.e., request/response and publish/subscribe) are mature and well understood. Also, through initiatives like NATO STO/IST-090 “SOA challenges for real-time and disadvantaged grids”, it has been shown that the technology can also be used in tactical environments provided certain adaptations are made. However, there are still challenges related to the discovery of services in the tactical environment. In this report we discuss the importance of service discovery in a federated system. Further, we look at the Web services discovery standards and discuss their suitability for use in a federated tactical network. Finally, we present a proof-of-concept solution (i.e., a demonstrator implementation) for discovery in a federation leveraging the WS-Discovery standard. This work has been performed in the context of NATO STO/IST-1181 “SOA recommendations for disadvantaged grids in the tactical domain”, the follow-on group to IST-090. Realizing the NNEC vision, where information is available to those that require it, independent of where or how they are connected to the network infrastructure, requires advanced mechanisms for information sharing. The NNEC information infrastructure, called NII, builds upon the SOA principles. This paradigm implies that the functionality is broken down into small, reusable pieces of functionality, known as services. In order for users to get access to the functionality they require they need to be able to know which services are available to them at any given time, and how to use these services. This process is known as service discovery. In order to achieve the seamless information exchange required by the NNEC vision, information exchange, and thus service discovery, must be available across network boundaries. This means that the service discovery mechanism must be able to interact with each other without the need for manual configuration. In a static environment, one potential approach to service discovery across network boundaries is through the use of service registries, which can be configured to share information using a replication or federation mechanism built into the registries themselves. This method can be used to give access to full metadata descriptions of the services. However, in more dynamic environments services registry federation can be problematic, as the availability of the registry becomes a single point of failure in the network. Because of this problem, service discovery in dynamic networks is better done with a decentralized mechanism. Such decentralized 1

External publication of ideas, approaches, and results is very important for IST-118 as a means of disseminating knowledge during the course of the group. In this spirit, a shorter version of this document has been published and presented at the 19th ICCRTS [23]. For more on IST-118, see [20].

FFI-rapport 2014/01454

7

service discovery mechanisms are mostly intended for use within a Local Area Network (LAN), and do not always scale well to larger scenarios. In addition, the difference in capabilities between different network types means that one is likely to encounter scenarios where different distributed services discovery mechanisms are required to work together. This problem has been explored previously, and has been shown to be solvable through the use of gateways [25]. However, the issue of scalability with respect to achieving cross-boundary dynamic service discovery remains unsolved. In many cases, nodes operating closely together geographically can be connected to different networks. In order for these nodes to become aware of each other’s services, the service metadata may have to traverse multiple other networks. Extending the reach of a standardized LAN discovery mechanism across a Wide Area Network (WAN) can be done in multiple ways, but remains a major challenge when attempting to achieve pervasive service discovery across heterogeneous networks. In this report we present a federation mechanism for dynamic service discovery which addresses these issues by extending the reach of the locally scoped multicast-based ad hoc mode of WS-Discovery across heterogeneous networks by employing Peer-to-Peer2 (P2P) techniques. We discuss how this can be achieved on a conceptual level, before looking into one specific implementation – the functional demonstrator we have implemented in Java.

2

Background

2.1

Discovering Web services

The SOA reference model [36] states that A service is a mechanism to enable access to a set of one or more capabilities, where the access is provided using a prescribed interface and is exercised consistent with constraints and policies as specified by the service description. A service is provided by one entity – the service provider – for use by others, but the eventual consumers of the service may not be known to the service provider and may demonstrate uses of the service beyond the scope originally conceived by the provider. A service is accessed by means of a service interface, where the interface comprises the specifics of how to access the underlying capabilities. There are no constraints on what constitutes the underlying capability or how access is implemented by the service provider. Thus, the service could carry out its described functionality through one or more automated and/or manual processes that themselves could invoke other available services. A service is opaque in that its implementation is typically hidden 2

In the survey [3], the authors argue that Peer-to-Peer (P2P) network overlays provide a good substrate for creating large-scale data sharing, content distribution and application-level multicast applications. This is because P2P networks potentially offer an efficient routing architecture that is self-organizing, massively scalable, and robust in the wide-area, combining fault tolerance, load balancing and explicit notion of locality.

8

FFI-rapport 2014/01454

from the service consumer except for 1. the information and behavior models exposed through the service interface and 2. the information required by service consumers to determine whether a given service is appropriate for their needs. The consequence of invoking a service is a realization of one or more real world effects (i.e. the actual result of using a service). In summary, service discovery is the process performed by a service consumer to find (i.e., discover) service providers. As discussed in [21], a service consumer may need to perform service discovery in two phases of its lifetime; at design-time or at run-time. At design-time, the developer of a software component needs to discover services that provide functionality required by the program. This process could be automatic or manual, depending on the design process. For example, the programmer could search through a registry, or use a Web search engine to discover services with the given functionality. Similarly, helper programs provided by a development tool or middleware may attempt to automatically discover available services and present them as software components available to the developer. During run-time, the application is responsible for discovering the services it requires, possibly with help from the user. The programmer, or designer, is no longer involved. An example of run-time service discovery is when a printer is discovered on the local network. The application is not designed to use a specific printer, but may work with any service that follows the service definition it was created for during design time. Simply put, service discovery in the context of Web services is the process of discovering a service endpoint in the form of an URL that points to the address where a service implementing a given service interface is deployed. The complete service definition is represented as a WSDL [51]. One way to find the service definition is to use a registry service, such as Universal Description Discovery and Integration (UDDI) [39] or electronic business using XML (ebXML) [35], to search for services during design-time. At run-time, the discovery process can be simplified by not requiring a full service definition, provided that it can be determined that discovered services implement the same interfaces that were used during design-time. In tactical networks such as disadvantaged grids, service discovery can be difficult to perform due to the nature of such networks. Examples of this include frequent changes in the network topology when nodes move in and out of each other’s radio ranges. The network is also likely to become partitioned. Service discovery is especially important in these environments as the set of services that are available to each consumer may change often and repeatedly. If a new service becomes available, the service consumers are unable to use it until they are aware of its existence. Conversely, a service may be discovered, but no longer be reachable due to a network change. Using a central service registry is not a viable option as participants in temporarily isolated areas of the network would be unable to discover each other. An alternative is to use a decentralized

FFI-rapport 2014/01454

9

Figure 2.1

Cross-domain service discovery experiments conducted by CoNSIS

discovery protocol. For an approach to service discovery at different operational levels, see our in-depth discussion in [31]. A standardized protocol for decentralized discovery of Web services without a registry is WSDiscovery [37]. WS-Discovery uses multicast to distribute service information, and is further described in [27], where we have evaluated WS-Discovery combined with XML compression as a means to optimize it for use in the tactical domain (see Appendix A). A first attempt at employing WS-Discovery for cross-domain service discovery in tactical networks was made in the Coalition for Secure Information Sharing (CoNSIS) experiment [6] (see Appendix B). There, a multicast tunnel was used to bridge the two tactical domains as shown in Figure 2.1. In this experiment, two tactical units from two different nations were part of the same coalition network, and they needed to share information with each other. Both units used the same service discovery mechanism for exposing their services to users outside their own domain, but the mechanism used relied on multicast messages to be transmitted between the two coalition networks. Since these networks were not always directly connected to each other, the service information would have to traverse other networks in order to reach its intended recipients. In the CoNSIS experiments this was solved by setting up manually configured multicast tunnels on the routing layer, which allowed multicast messages from one network to reach the other network. This solution works in a small scale scenario, but it has a number of downsides: It relies on multicast support across domain boundaries. This is normally not supported. Also, the use of multicast together with forwarding to other domains through special-purpose multicast tunnels leads to a lot of network traffic and thus poor scalability. Further, the mechanism was very coarsegrained, as it was not possible to determine which services to share — all partners could see all published services, even local ones. This approach was viable for the CoNSIS experiment, but cannot be recommended for future deployments because of the drawbacks: The approach assumes both domains use the same metadata to describe services, it requires close coordination before deployment, and it might expose domain internal metadata. Thus, a new approach is needed for service discovery across a WAN which does not rely on multicast tunnels being present. Our approach to such a solution is presented in the remainder of this document.

10

FFI-rapport 2014/01454

2.2

Intended application at Unified Vision ’14

Unified Vision 2014 (UV14) was a trial that took place in Norway between 18-28 May 2014. Its aim was to test NATO’s ability to gather information and fuse intelligence from multiple sources, namely from space, in the air, on land and at sea. In the trial, analysts “fused” ISR products from these different sources, which were provided from 18 different countries, and the aim was to provide the commanders with one common picture, derived from all these different sources. Joint Intelligence, Surveillance and Reconnaissance (JISR) is an activity that synchronizes and integrates the planning and operation with exploitation and processing for all collection capabilities. Furthermore, it ensures the dissemination of the resulting information to the right person, at the right time, in the right format. The idea of JISR is to encourage dynamic, agile and coordinated use of platforms, sensors and systems, in order to support a wide range of staff functions. The JISR core activities focus on the JISR cycle, which consists of tasking, collection, processing, exploitation and dissemination, and they include both planned and dynamic aspects of JISR. Through the core activities, information and rudimentary intelligence are provided to commanders conducting a wide range of activities, such as intelligence, operations and targeting. From the point of view of FFI project 1277 “Information and integration services in INI” the main focus in UV14 was on applying the WAN discovery mechanism described in this report as the cornerstone for the two activities planned for UV14: First, each sensor regularly submits a System Status Update (SSU), also known as a “heartbeat”. These SSUs are dispatched using WS-Notification [38], the publish/subscribe standard chosen by NATO, and the plan was to subscribe to the SSUs, and then publish the corresponding sensors as services using WS-Discovery. A sensor that no longer submits SSUs will then be unpublished from WS-Discovery, after a timeout period. Thus, this use of the SSUs would enable us to present an up-to-date view of the available sensors at any given time, through WS-Discovery. Second, the WAN discovery mechanism was intended to be used in order to control the flow of information within the experiment network. Having extended service information, such as the topics offered by a notification producer, enables the user to set up subscriptions for relevant topics based on the information supplied by the service discovery mechanism. In a situation where the sensor information is provided over WS-Notification, service discovery is no longer sufficient for distinguishing between the services. It is therefore necessary to also be able to discover the information offered by the different WS-Notification endpoints in order to be able to find the correct service. We have therefore proposed to extend the WS-Discovery standard with a topic matching rule, which enables consumers to search for Web services providing information about a given topic (see Appendix C). This, combined with the fact that WS-Notification allows for creating subscriptions on behalf of others, means that subscriptions, and thus information flow, can be controlled from any point within the network. This in turn means that the administration of subscriptions can be handled centrally, where one information manager can take on the task of

FFI-rapport 2014/01454

11

creating, modifying and deleting subscriptions on behalf of the end users. While delegating this task to a central information manager frees the end user from having to think about subscription handling, it is important to note that introducing such a mechanism does not prevent the end user from taking control of their own subscription handling if they should require more direct control of their own information flow. Unfortunately, due to administrative difficulties, project 1277 was not able to carry out its planned activities for UV14. Instead, we tested our cornerstone WAN discovery mechanism in a lab environment at FFI. The design and implementation of the WAN discovery mechanism are discussed next, followed by a summary of the testing. Naturally we did not have access to the UV14 sensors in our lab, so the experiments presented here reflect dummy services. We also have further ideas for future work, which are outlined in Section 5.

3

Design and implementation

The general idea is to extend the reach of a service discovery protocol employed in a LAN across a WAN using a suitably efficient and scalable mechanism. This led us to consider P2P networks for the distribution mechanism, as such overlays can scale to a large number of participating nodes and can function across a WAN. P2P networks support basic overlay network functionality such as message routing, and managing nodes that join or leave the network. Being an overlay, a P2P network introduces its own addressing scheme, creating an abstraction layer above the underlying physical network(s). In addition to the basic functionality involving bootstrapping and maintaining the network, we need a means of efficiently employing one-to-many dissemination of messages. The reason for this is that we may have one (or more) LANs running different service discovery protocols that need to be interconnected across a WAN. If there is a case of heterogeneous discovery protocols being involved, then the discrepancies between the protocols’ expressive power need to be mitigated through a metadata repository or through some other means of obtaining the missing information when translating from one protocol to another. This issue has been shown to be solvable through the use of interoperability gateways [25], so we do not pursue that issue further here. It suffices to say that interoperability gateways are a concept than can be combined with the design we are about to describe, if there are different discovery protocols involved. Figure 3.1 illustrates this concept. For the sake of argument in this report, we focus only on WS-Discovery [37] which is an OASIS standard for Web services discovery. Thus, looking at a single node which has the task of extending the reach of WS-Discovery from the LAN across a WAN, we get the conceptual layer model shown in Table 3.1 (this is a simplified layered model collapsing some of the layers in the 7 layer OSI model). The WS-Discovery protocol has been implemented in different frameworks and operating systems, for example it is supported by Apache CXF [4], in the WSO2 ESB [53], and in recent versions of Microsoft Windows (Vista and newer). For our demonstrator we have chosen to leverage the Java open source implementation3 of WS-Discovery. 3

The WS-Discovery library can be obtained from http://code.google.com/p/java-ws-discovery/ – just make sure

12

FFI-rapport 2014/01454

Application layer

WS-Discovery service descriptions

P2P Overlay

Scalable one-to-many message dissemination (includes network maintenance and routing)

Network

“Everything over IP” mindset from NATO means that we focus on IP: IP network (IP routing, IP addresses)

Physical

Network hardware

Table 3.1

Figure 3.1

Simplified layered model showing our conceptual approach

Federation of two independent WS-Discovery (WS-D) domains

There are many different P2P overlay networks (see e.g., [3] for a recent survey). Our goal is to extend the reach of WS-Discovery, so we have not sought out to evaluate many different P2P overlays in order to identify the overall “best” approach. Rather, we have chosen a solution to employ in our demonstrator based on the following observations: • Many P2P approaches are only implemented for simulated environments, and have not been tested for real-world applications. Though interesting from an academic viewpoint, these solutions are of little value when attempting to build a functional demonstrator. Thus, we need a solution that exists as a software library. • We want a solution that can scale to a large number of nodes, and that solves the overlay maintenance issues in an efficient manner. • The solution must be able to provide efficient one-to-many communication, as that forms the basis for our demonstrator design. Given the above observations, we found that Pastry [41] coupled with SCRIBE [42] provides everything we require from the overlay network: Pastry nodes form a decentralized, selforganizing and fault-tolerant overlay network which provides efficient request routing, deterministic object location, and load balancing in an application-independent manner. Furthermore, Pastry provides mechanisms that support and facilitate application-specific object replication, caching, and fault recovery. Add SCRIBE to Pastry, and you get a generic, scalable you download it from the repository and build it yourself, as the binary release offered is outdated and contains known bugs.

FFI-rapport 2014/01454

13

Figure 3.2

The WAN service discovery demonstrator main components

and efficient group communication and event notification system providing application level multicast and anycast. Though not perfect (for example, bootstrapping the system requires prior knowledge of one node’s address) this combination, as implemented by the open source project Freepastry,4 solves all the important aspects outlined above adequately. Thus, in relation to Table 3.1, the P2P overlay constitutes SCRIBE over Pastry for our demonstrator. The implementation consists of four main components as illustrated in Figure 3.2: The Graphical User Interface (GUI), the P2P and WS-Discovery modules, and a service cache. Let us explore each of these in turn. 3.1

GUI

The GUI presents the user with all relevant configuration fields and control buttons. It is illustrated in Figure 3.3. Here, we see the parameters necessary to initialize the different modules. For the WS-Discovery module you need to enter the IP address of the network interface which should send and receive WS-Discovery related multicast IP packets. For the P2P mechanism you need to enter the local bind port, which is the port on which the P2P node instantiated by the P2P module should listen. Correspondingly, the local IP is the IP address of the network interface that the P2P module should use, i.e., the interface connected to the WAN. The boot IP and boot port should contain the IP and port of an already started instance of the demonstrator on another node, as it is being used by the P2P module to bootstrap the Pastry routing ring. The first instance being started needs to use its own port and IP address here since it has no other instances to connect to. Once all this information has been filled in the user can press “start” to start the other modules (i.e., P2P, WS-Discovery, and service cache). From this point on the software will run on its own, taking local WS-Discovery services and publishing them across the P2P network for other instances of the software to re-publish locally using WS-Discovery. The “probe” button can be used to force a probe (more on this under the WS-Discovery module section below), and the “exit” button quits the program. The debug panel is, as the name indicates, a text area where the application prints debug information as it is running (e.g., services seen locally and published across the P2P network, and services received from the P2P ring for local publication using WS-Discovery). 4

Freepastry is available for download at http://www.freepastry.org/FreePastry/

14

FFI-rapport 2014/01454

Figure 3.3

3.2

The GUI design

P2P module

The P2P module uses the freepastry library to create a local P2P node using the information entered in the GUI. After creation the node is connected to the already existing pastry network (if at least one other node is already started, as mentioned above). From this point on the module functions in the following manner: It receives local service instances from the WS-Discovery module. Each service is published across the P2P network for all P2P nodes to receive using SCRIBE. Every P2P node receives the published services, so services that originated locally are filtered out to ensure that they are not re-published in the local domain. Services that originated remotely are added to the service cache, which handles local publication and expiry of the remote services. 3.3

WS-Discovery module

The WS-Discovery module uses the discovery library to create a local instance of the WSDiscovery server using the information entered in the GUI. Once the server instance is created and active, it begins listening to the network. Any services published with WS-Discovery locally (i.e., any HELLO messages multicast in the network) end up in the library’s cache. This cache is queried periodically, and any services present are submitted to the P2P module for publication across the P2P network. We chose this simple approach for the demonstrator, but for future use it would be beneficial to implement some filter mechanism to limit which services get re-published across the WAN. One approach could be to filter on scope, for example. Here all services are disseminated, though.

FFI-rapport 2014/01454

15

Further, we chose to rely on the local WS-Discovery cache’s knowledge of services as the basis for what to disseminate. In a stable network like a wired LAN this is unproblematic. If, however, WS-Discovery is employed in a more disruptive environment then this may not be enough. For the sake of the demonstrator we just added a “probe” button to the GUI, allowing the user to probe the network at will to update the contents of the cache. For actual deployment another solution might be desirable, for example that the module actively probes the network itself at given intervals. That would introduce traffic into the network, though, so careful consideration should be made before one goes to that step. 3.4

Service cache

The service cache caches the remote services, i.e., the services received over the P2P network. The cache ensures that a remote service being added is also submitted to the WS-Discovery module for publishing locally. We expect services that remain available to be re-published across the P2P network periodically. A cleaner thread in the cache module ensures that services which have not been updated for a period of time expire from the cache and are unpublished locally by the WS-Discovery module.

4

Results and recommendations

Our demonstrator has been tested in our lab facilities. Figure 4.1 shows the test setup, where two distinct LANs (LAN1 and LAN2) were connected through a non-multicast-enabled router symbolizing a WAN. In LAN1, two nodes (PC1 and PC2) were both publishing services using WS-Discovery. In addition, PC1 was running our demonstrator software, designated “WAN Discovery” in the illustration. Thus, PC1 was both a regular WS-Discovery node and a P2P network node. Correspondingly, in LAN2, PC3 was also a WS-Discovery node as well as a P2P network participant. IP addresses were configured as shown in the figure. Since the FFI INI lab was utilized, all node instances were virtualized. All nodes had Windows 7 installed and were running Oracle Java 1.7. The router was created as a regular Windows 7 node, but with two network interfaces and routing between the interfaces enabled. After the router was configured, WS-Discovery was started on PC1-3. At this point, services were published using WS-Discovery on PC1 and PC3. Since the router does not tunnel multicast traffic, no services from LAN1 are visible in LAN2 and vice versa. The screenshots in Figures 4.2, 4.3, and 4.4 illustrate this. Note that PC2 can see the service published on PC1, since they are both in the same LAN and thus part of the same multicast domain. This far we have seen that WS-Discovery in conjunction with our router setup functions as expected. Following that we launched our demonstrator on PC1 and PC3, and configured the GUI IP addresses according to the network topology. We chose an arbitrary unused port for the test series, in this case 7654. Figures 4.5 and 4.6 show the demonstrator just being started on these two nodes.

16

FFI-rapport 2014/01454

Figure 4.1

Test setup using the INI lab

Figure 4.2

Screenshot PC1

FFI-rapport 2014/01454

17

Figure 4.3

Screenshot PC2

Figure 4.4

Screenshot PC3

18

FFI-rapport 2014/01454

Figure 4.5

Demonstrator being started on PC1

Figure 4.6

Demonstrator being started on PC3

FFI-rapport 2014/01454

19

Figure 4.7

PC1 — federated service discovery result

At this point the service from LAN1 became available in LAN2 and vice versa, showing that the demonstrator functioned as intended. We also published a service on PC2, so that all nodes were hosting a service. That service also became available in both LANs — in LAN1 due to WS-Discovery’s multicast dissemination, and in LAN2 due to our demonstrator’s P2P-based dissemination mechanism. This successful test is shown in Figures 4.7, 4.8, and 4.9. We also tested unpublishing a WS-Discovery service on one PC, and saw that it eventually disappeared from the known services on the other nodes. Thus, the demonstrator functioned to illustrate the proof of concept of our design. We think the approach is viable, and could be refined in an attempt to realize federated service discovery across coalition force networks.

5

Conclusion and future work

This work was performed in context of the NATO STO/IST-118 group which focuses on identifying what we call “tactical SOA foundation services” [20]. By this we mean which core enterprise services we need support for in the tactical domain. Examples include the messaging service, publish/subscribe service, and service discovery service. In other words, we aim to investigate how services from the SOA baseline can be extended for use in tactical networks. In this report we have addressed the service discovery service, as seen by IST-118 (i.e., leverage WSDiscovery (or perhaps other decentralized discovery protocols) rather than UDDI in disadvantaged grids) along with a solution for federation. Our demonstrator was realized by employing P2P techniques in an interoperability gateway using the existing standard WS-Discovery for service discovery. Tests in FFI’s INI lab have

20

FFI-rapport 2014/01454

Figure 4.8

PC2 — federated service discovery result

Figure 4.9

PC3 — federated service discovery result

FFI-rapport 2014/01454

21

shown that this proof-of-concept solution functioned as expected. We did, however, identify some shortcomings that need to be remedied if such a mechanism is to be used for actual deployments: • There is no redundancy, only one instance of the WAN discovery mechanism can run in a subnet at any time. • Currently, all services are shared across the networks. A selection mechanism needs to be present, for example configurable filtering using the services’ scope. • Freepastry was chosen for the P2P network because it was freely available. Other, better alternatives to extend the reach of WS-Discovery can possibly be found. • The demonstrator only supports one service discovery mechanism. Ideally, it should support other relevant protocols as well in a seamless manner. FFI project 1277 intends to offer and supervise a task related to investigating the issues mentioned above for one student at the Master’s degree level. Further, the JISR concept implies that information from a large number of potentially very different sensor types must be collected and combined. Thus, at the receiving end, many different data formats must be handled. This gave rise to the idea of combining service discovery, WSNotification, orchestration and mediation, which in turn could enable an operator to discover data sources (sensors), subscribe to them, and translate the data format using orchestration and mediation. Such an approach would provide a powerful means of dynamically combining information from different data sources. As a further step, we envision that this process could, to a large degree, be automated. This idea could be interesting to pursue.

22

FFI-rapport 2014/01454

References [1] D. S. Alberts and R. E. Hayes. Planning: Complex endeavors. http://www.dodccrp.org/files/ Alberts_Planning.pdf, 2007. [2] M. Amoretti, F. Zanichelli, and G. Conte. SP2A: a service-oriented framework for P2P-based grids. In proceedings of the 3rd International Workshop on Middleware for Grid Computing (MGC05), Grenoble, France, 2005. [3] A. Anitha, J. JayaKumari, and G. Mini. A survey of p2p overlays in various networks. in proceedings of the International Conference on Signal Processing, Communication, Computing and Networking Technologies 2011, pages 277-281. [4] Apache Software Foundation. Apache CXF: An Open-Source Services Framework. http://cxf.apache.org/. [5] P. Bartolomasi, T. Buckman, A. Campbell, J. Grainger, J. Mahaffey, R. Marchand, O. Kruidhof, C. Shawcross, and K. Veum. NATO network enabled capability feasibility study. Version 2.0, October 2005. [6] T. H. Bloebaum and K. Lund. CoNSIS: Demonstration of SOA Interoperability in Heterogeneous Tactical Networks. 2012 Military Communications and Information Systems Conference (MCC), Gdansk, Poland, 8-9 Oct 2012. [7] J. Cardoso. Semantic Web Services: Theory, Tools and Applications. IGI Global, 2007. [8] D. Chappell and L. Liu (eds.). Web Services Brokered Notification 1.3 (WSBrokeredNotification). http://docs.oasis-open.org/wsn/wsn-ws_brokered_notification-1. 3-spec-os.pdf, OASIS standard, 2006. [9] P. Costa, C. Mascolo, M. Musolesi, and G. P. Picco. Socially-Aware Routing for PublishSubscribe in Delay-Tolerant Mobile Ad Hoc Networks. IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 26, NO. 5, JUNE 2008. [10] D. Davis, A. Malhotra, K. Warr, and W. Chou (eds.). Web Services Eventing (WS-Eventing). http://www.w3.org/TR/ws-eventing/, W3C Proposed Recommendation, 27 September 2011. [11] C. Esposito, D. Cotroneo, and A. Gokhale. Reliable publish/subscribe middleware for timesensitive internet-scale applications. Proceedings of the Third ACM International Conference on Distributed Event-Based Systems, ser. DEBS ’09. New York, NY, USA, pp.16:1-16:12, 2009. [12] S. Graham, D. Hull, and B. Murray (eds.). Web Services Base Notification 1.3 (WSBaseNotification). http://docs.oasis-open.org/wsn/wsn-ws_base_notification-1. 3-spec-os.pdf, OASIS standard, 2006.

FFI-rapport 2014/01454

23

[13] E. Guttman. Vendor Extensions for Service Location Protocol, Version 2. RFC 3224 (Proposed Standard), Jan. 2002. [14] T. Hafsøe, F. T. Johnsen, N. A. Nordbotten, and E. Skjervold. Using Web Services and XML Security to Increase Agility in an Operational Experiment featuring Cooperative ESM Operations. 14th International Command and Control Research and Technology Symposium (ICCRTS), Washington DC, USA, June 2009. [15] A. Harrison and I. Taylor. WSPeer — An Interface to Web Service Hosting and Invocation. In HIPS Joint Workshop on High-Performance Grid Computing and High-Level Parallel Programming Models, IPDPS, Denver, Colorado, USA, 2005. [16] M. Hauge, J. Andersson, M. Brose, and J. Sander. Multi-topology routing for QoS support in the CoNSIS convoy MANET. IEEE MCC, Gdansk, Poland, 2012. [17] S. Helal, N. Desai, V. Verma, and C. Lee. Konark - a service discovery and delivery protocol for ad-hoc networks. Proceedings of the Third IEEE Conference on Wireless Communication Networks (WCNC), New Orleans, 2003. [18] R. Jeyaraman (ed.). SOAP-over-UDP Version 1.1 OASIS Standard. http://docs.oasis-open. org/ws-dd/soapoverudp/1.1/os/wsdd-soapoverudp-1.1-spec-os.pdf, 1 July 2009. [19] P. Jiang, J. Bigham, E. Bodanese, and E. Claudel. Publish/subscribe delay-tolerant messageoriented middleware for resilient communication. IEEE Communications Magazine, 49(9):124–130, Sept. 2011. [20] F. Johnsen, T. Bloebaum, P.-P. Meiler, I. Owens, C. Barz, and N. Jansen. IST-118 – SOA recommendations for Disadvantaged Grids in the Tactical Domain. 18th ICCRTS, Alexandria, VA, USA, June 19-21, 2013. [21] F. T. Johnsen. Pervasive web services discovery and invocation in military networks. FFI report 2011/00257, http://rapporter.ffi.no/rapporter/2011/00257.pdf. [22] F. T. Johnsen and T. H. Bloebaum. Topic discovery for publish/subscribe web services. IEEE IWCMC, Limassol, Cypus, August 2012. [23] F. T. Johnsen and T. H. Bloebaum. Travel report: 19th ICCRTS, Alexandria, VA, USA, June 2014. FFI-reiserapport 2014/01348. [24] F. T. Johnsen, T. H. Bloebaum, L. Schenkles, J. Sliwa, and P. Caban. Soa over disadvantaged grids experiment and demonstrator. IEEE MCC, Gdansk, Poland, 2012. [25] F. T. Johnsen, J. Flathagen, and T. Hafsøe. Pervasive service discovery across heterogeneous networks. IEEE MILCOM, Boston, MA, USA, 18-21 Oct. 2009. [26] F. T. Johnsen, J. Flathagen, T. Hafsøe, M. Skjegstad, and N. Kol. Interoperable Service Discovery: Experiments at Combined Endeavor 2009. FFI report 2009/01934, http://rapporter.ffi.no/rapporter/2009/01934.pdf, 2009.

24

FFI-rapport 2014/01454

[27] F. T. Johnsen and T. Hafsøe. Adapting ws-discovery for use in tactical networks. 16th ICCRTS, Québec City, Québec, Canada, June 21-23, 2011. [28] F. T. Johnsen and T. Hafsøe. Experiments with Web services at Combined Endeavor. 15th International Command and Control Research and Technology Symposium (ICCRTS), Los Angeles, CA, USA, June 2010. [29] F. T. Johnsen and T. Hafsøe. Using NFFI web services on the tactical level: An evaluation of compression techniques. 13th International Command and Control Research and Technology Symposium (ICCRTS), Seattle,WA, USA, June 2008. [30] F. T. Johnsen and T. Hafsøe. Service Advertisements in MANETs (SAM): A decentralized web services discovery protocol. IEEE Workshop on Ubiquitous Computing and Networks (UbiCoNet), Dec. 2010. [31] F. T. Johnsen, T. Hafsøe, and M. Skjegstad. Web services and service discovery in military networks. 14th ICCRTS, Washington, DC, USA, June 15-17, 2009. [32] K. Lund, E. Skjervold, F. T. Johnsen, T. Hafsøe, and A. Eggen. Robust web services in heterogeneous military networks. IEEE Communications Magazine, October 2010. [33] M. Gudgin et al. SOAP version 1.2 part 1: Messaging framework (second edition). http://www.w3.org/TR/2007/REC-soap12-part1-20070427/#bindfw, W3C Recommendation 27 April 2007. [34] A. N. Mian, R. Baldoni, and R. Beraldi. A survey of service discovery protocols in multihop mobile ad hoc networks. In IEEE Pervasive computing, pages 66–74, January-March 2009. [35] OASIS. ebXML. OASIS standard, http://www.ebxml.org/. [36] OASIS. Reference model for service oriented architecture. Committee draft 1.0, 2006, http://www.oasis-open.org/committees/download.php/16587/wd-soa-rm-cd1ED.pdf. [37] OASIS. Web services dynamic discovery (ws-discovery) version 1.1. OASIS Standard, 1 July 2009, http://docs.oasis-open.org/ws-dd/discovery/1.1/wsdd-discovery-1.1-spec.html. [38] OASIS. Web services notification tc. http://www.oasis-open.org/committees/tc_home. php?wg_abbrev=wsn. [39] OASIS UDDI Specification TC. Uddi v3.0.2 specification. OASIS Standard, http://uddi.org/pubs/uddi_v3.htm. [40] G. G. Richard III. Service and Device Discovery: Protocols and Programming. Mc Graw Hill, 2002. [41] A. Rowstron and P. Druschel. Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems. IFIP/ACM International Conference on Distributed Systems Platforms (Middleware), Heidelberg, Germany, pages 329-350, November, 2001.

FFI-rapport 2014/01454

25

[42] A. Rowstron, A.-M. Kermarrec, M. Castro, and P. Druschel. Scribe: The design of a largescale event notification infrastructure. NGC2001, UCL, London, November 2001. [43] F. Sailhan and V. Issarny. Scalable service discovery for MANETs. In Third IEEE International Conference on Pervasive Computing and Communications (PERCOM), pages 235–244. IEEE Computer Society, March 2005. ISBN 0-7695-2299-8. [44] S. Sakr. XML Compression Techniques: A Survey and Comparison. Journal of Computer and System Sciences (JCSS) 75(5), pp. 303-322, 2009. [45] J. Schneider. Efficient XML: Taking Net-Centric Operations to the Edge. 13th International Command and Control Research and Technology Symposium (ICCRTS), Seattle, WA, USA, June 2008. [46] H. Seifert and M. Franke. SOA in the CoNSIS coalition environment. IEEE MCC, Gdansk, Poland, 2012. [47] M. Skjegstad, F. T. Johnsen, T. Hafsøe, and K. Lund. Robust and efficient service discovery in highly mobile radio networks using the Mist protocol. IEEE Military Communications Conference (MILCOM 2010), Oct. 2010. [48] E. Skjervold, T. Hafsøe, F. T. Johnsen, and K. Lund. Enabling Publish/Subscribe with COTS Web Services across Heterogeneous Networks. IEEE International Conference on Web Services (ICWS) 2010, Miami, Florida, USA, July 5-10, 2010. [49] W. Vambenepe, S. Graham, and P. Niblett (eds.). Web Services Topics 1.3 (WS-Topics). http://docs.oasis-open.org/wsn/wsn-ws_topics-1.3-spec-os.pdf, OASIS standard, 2006. [50] S. Vinoski. Toward Integration – More Web Services Notifications. IEEE Internet Computing, vol. 8, no. 2, Mar./Apr. 2004, pp 86-90. [51] World Wide Web Consortium (W3C). Web services description language (wsdl) 1.1. W3C Note 15 March 2001, http://www.w3.org/TR/wsdl. [52] World Wide Web Consortium (W3C). XML Binary Characterization Working Group Public Page. http://www.w3.org/XML/Binary/. [53] WSO2 Inc. WSO2 Enterprise Service Bus. enterprise-service-bus/.

26

http://wso2.com/products/

FFI-rapport 2014/01454

Appendix A

Adapting WS-Discovery for use in tactical networks

This appendix summarizes the experiments performed with WS-Discovery and compression. It was published and presented at the 16th ICCRTS, Quebec City, Quebec, Canada, June 2011 (see [27]). A.1

Introduction

The NATO Network Enabled Capabilities (NNEC) feasibility study [5] has identified Web services as a key enabling technology for NNEC. This is due to the NNEC demands for interoperability, and the fact that Web services are founded on standards. This means that the technology, which is in widespread use in civil networks today, is emerging in military systems in NATO countries and will therefore also be an important part of civil-military interoperability in the future. Alberts and Hayes [1] describe a civil-military operation as a complex system which involves a large number of disparate entities such as military units, civil authorities, multinational and international organizations, non-governmental organizations, companies, and private volunteer organizations in dynamic environments operating with imperfect and incomplete information. Due to the dynamic nature of such an operation, mobility and abrupt disappearance of services must be supported. This means that service discovery must be distributed and robust. The participants may use heterogeneous devices, requiring standardized service oriented middleware to achieve interoperability. Discovery of Web services can be performed either by utilizing a service registry, or a decentralized non-registry based solution. There are three standards addressing Web services discovery, two registries and a non-registry solution. The registries, UDDI [39] and ebXML [35], suffer from liveness and availability problems in dynamic environments [26]. The third standard for Web services discovery is WS-Discovery [37]. It is better suited to dynamic networks than the registries in that it is a decentralized discovery mechanism, thus removing the single point of failure that a centralized registry constitutes. We attempted to use WS-Discovery in a disadvantaged grid, and found that it was unsuitable for use there since it generated too much traffic in the network and flooded the modem buffers (see our paper [14]). If we can reduce the overhead of WS-Discovery, however, then it may be better suited for use in military networks as well. Recent work by the W3C regarding efficient XML interchange (EXI) can potentially make WS-Discovery suitable for both civil and military networks, and thus interesting as a solution for use in multinational civil-military operations. The contribution of this paper is an evaluation of the performance gains (in terms of reduced bandwidth) that can be achieved by combining the WS-Discovery standard with the EXI specification. For evaluation purposes we combine an open source implementation of WS-Discovery with an open source implementation of EXI.

FFI-rapport 2014/01454

27

The remainder of this paper is organized as follows: Section A.2 discusses related work; standardized protocols and proprietary solutions. Section A.3 introduces the initiative regarding XML compression by W3C’s EXI Working Group. In section A.4 we describe our proof-ofconcept implementation: combining an open source WS-Discovery implementation with an open source EXI implementation, and the results of our evaluation. Section A.5 concludes the paper. A.2

Related work

There exist several standards and proprietary, experimental solutions that may be employed for Web services discovery. In this section we give an overview of the most prominent solutions that have emerged. A.2.1

Web services discovery standards

There are three standards related to service discovery, all by OASIS: Universal Description, Discovery and Integration (UDDI), electronic business using XML (ebXML), and WS-Dynamic Discovery (WS-Discovery). UDDI [39] allows service providers to register their services and service consumers to discover these services both at design-time and run-time. In principle, UDDI is centralized, but mechanisms for federating several registries have also been specified. Having multiple registries, or letting a registry consist of several nodes that replicate data, increases the robustness of the discovery solution. In UDDI, replication between registry nodes must be configured manually. It is also possible to let several separate UDDI registries exist independently of each other, but information will not be replicated unless a custom scheme is designed. Additionally, a hierarchical model may be used, using a root registry and affiliate registries. The UDDI registry supports reconfiguration as long as services do not go down unexpectedly. If so, advertisements will be in the registry forever because there is no liveness information in the current versions of UDDI. Another service discovery standard is ebXML [35]. It is a collection of specifications for conducting business-to-business integration over the Web. It allows registering services in a similar way as UDDI according to its own registry specification. The ebXML registry also defines inter-registry interaction, or cooperation between registries, a so-called federation of registries. Note that an ebXML federation is different from that of a UDDI federation, because ebXML supports a non-hierarchical multiregistry topology. Here, each registry has the same role, and registries may join or leave a federation at any time. This allows flexible deployment. Federated queries are supported, enabling query forwarding to other registries without the need to replicate data first. The ebXML registry is meant to support both discovery and business collaboration, as opposed to UDDI, which mainly targets discovery. Just like UDDI, ebXML has issues with liveness, in that it supports reconfiguration as long as services do not go down unexpectedly. WS-Discovery is the newest standardized Web services discovery mechanism. After being a draft since 2005, it became a standard in 2009 [37]. WS-Discovery is based on local-scoped multicast,

28

FFI-rapport 2014/01454

using SOAP-over-UDP [18] as the advertisement transport protocol. Query messages are called probe messages. Services in the network evaluate probes, and respond if they can match them. To ease the burden on the network, WS-Discovery specifies a discovery proxy (DP) that can be used instead of multicast. This means that WS-Discovery can run in two modes, depending on whether there is a DP available or not. However, this DP is not well-defined in the standard. The standard fully describes the decentralized operation of WS-Discovery, but the functionality of (and integration with) the DP is left to be implemented in a proprietary manner for now. We evaluate only the standardized parts of WS-Discovery in this paper, focusing on decentralized operation. A.2.2

Other solutions

There exist several service discovery protocols (see [34] for a survey), but few that are tailored for discovery of Web services. Traditional discovery protocols such as SLP, Bluetooth SDP, and UPnP are geared towards device discovery [40], and do not possess the expressive power that is needed to support Web services discovery. To address the need for a Web services discovery protocol for dynamic environments, OASIS has created the WS-Discovery standard. WS-Discovery is based on querying the network for services. Using multicast SOAP-over-UDP, so-called probe messages flood the network, and nodes that can match the query, respond with a unicast probe match message. In a broadcast environment, such behavior is not desirable, since it generates a lot of traffic. Even if your neighbor has just searched for services, you will have to repeat the search because you have not received the reply. This is a drawback that has been addressed by experimental service discovery solutions. Service Advertisements in MANETs (SAM) is a service discovery protocol that we have designed and implemented for use in MANETs [30]. It addresses the high resource use of WS-Discovery, and uses periodic service advertisements. The advertisements are compressed to reduce overhead, and caching with timeouts is used to address the liveness issue. The protocol offers a fairly up to date view of available services (in the local cache), and may significantly reduce discovery communication overhead because there is no need to query the network. Instead, service advertisements are sent at fixed intervals using IP multicast to reach all nodes. This spreads out the service discovery traffic over time, and eliminates traffic bursts due to frequent searches for services. As multicast is not always available in MANETs, we have designed a delay tolerant, publish/subscribe mechanism that we call Mist. Using this protocol, we have implemented a Web services discovery solution, Mist-SD [47], which is suitable for use in large MANETs. Like SAM it is based on periodic updates, but instead of completely distributing all information to all nodes, it uses a combined broadcast and subscription scheme which ensures that only relevant information is propagated through the network. Mist-SD is capable of distributing WSDLs, as well as other types of application defined metadata. Sailhan et al. [43] have created an experimental Web services discovery mechanism, Ariadne, that could potentially be usable in both large and small MANETs. It is based on having some nodes

FFI-rapport 2014/01454

29

that function as registries, and allowing these nodes to form an overlay for service discovery. Konark [17] is a fully decentralized service discovery mechanism for multi-hop MANETs. It is a complete middleware for service description, discovery and invocation. The framework is loosely based on Web services, in that XML is used for descriptions, and SOAP over HTTP is used for service invocation. However, the service descriptions are a proprietary twist on WSDL’s, meaning that COTS Web services cannot be used with this solution. Konark uses periodic advertisements with a TTL to handle liveness. Konark multicasts a subset of its service knowledge to reduce bandwidth. This means that you may not be able to access all services in your network partition, since service advertisements are made in a random fashion. The Service-Oriented P2P Architecture (SP2A) [2] supports service deployment with Web services. SP2A removes the loose coupling of Web services, demanding that the services are published, searched for, and invoked through the Peer-to-Peer (P2P) overlay. Another experimental P2P based Web services framework which also integrates discovery and service invocation is WSPeer [15]. WSPeer is a component based solution that provides an API for hosting and invoking Web services. However, being a complete framework it too, like SP2A, removes the loose coupling of Web services, tightly coupling service implementation, discovery, and invocation to their proprietary API. A.3

Compression

Efficient XML (EFX) was one of the formats the W3C XML Binary Characterization Working Group [52] investigated during their work with requirements for a binary XML format. It was later adopted by the W3C Efficient XML Interchange Working Group (EXI) as the basis for the specification of the efficient XML format. The objective of the EXI Working Group is to develop a specification for an encoding format that allows efficient interchange of the XML Information Set, and to illustrate effective processor implementations of that encoding format. EFX was originally developed by Agile Delta and provides a very compact representation of XML information [29, 45]. There also exists an open source Java implementation of the EXI specification called “EXIficient”. It is available from “http://exificient.sourceforge.net/”. In this paper we use the open source implementation of EXI, release 0.5. We can apply EXI to SOAP-over-UDP in WS-Discovery, and still remain compliant to the standard, because encoding the entire SOAP message, headers and all, is allowed according to the SOAP Messaging Framework standard [33], section 4.2: “5. SOAP Message Construct provides that all SOAP envelopes are serializable using an XML 1.0 serialization, so XML 1.0 or later versions of XML MAY be used by bindings as the “on the wire” representation of the XML Infoset. However, the binding framework does not require that every binding use an XML serialization for transmission; compressed, encrypted, fragmented representations and so on can be used if appropriate.”

30

FFI-rapport 2014/01454

Minimum size

Maximum size

Average

Standard deviation

Median

1643

149342

13830

19202

8514

Table A.1

A.4

Sizes (in bytes) for our 100 WSDL files.

Implementation and evaluation

An open source implementation of WS-Discovery written in Java is available from “http://code.google.com/p/java-ws-discovery/”. The current release 0.2.0 is only draft compliant, but the version in the repository adheres to the standard. We downloaded the most recent WSDiscovery revision from the open source repository (which was revision 116 at the time we performed our experiments). The evaluation was performed in two iterations: First, we evaluated the WS-Discovery standard on its own. Second, we evaluated WS-Discovery with added EXI compression. For the evaluation of the standard we compiled the sources and used the software unmodified. To evaluate the standard with EXI compression, however, we had to make some modifications: First, we modified parts of the SOAP-over-UDP library, where we added a new transport class that would apply EXI compression and de-compression to outgoing and incoming UDP packets, respectively. We enabled all the compatibility parameters for EXI (along with the parameter for maximum compression), thus ensuring that the lexical integrity of the XML documents was preserved. This was done to ensure that WS-Discovery functioned properly upon de-serializing the data. Then, we made two changes to the WS-Discovery library, where we added our new EXI capable transport under available transport types, and finally set this transport to be the default to be used. Finally, we compiled the libraries and repeated the tests made with the standard implementation. A.4.1

Evaluation framework

We evaluated WS-Discovery using WSDLs for services such as finance, news, weather services, etc. These WSDLs were fetched from “http://www.webservicex.net/” and “http://www.webservicelist.com/”, which provide lists of freely available Web services. Also, the WSDLs from Google and Amazon’s search services were included, yielding a set of 100 WSDLs in total. This provided us with a representative set of interfaces (see Table A.1 for a wide array of Web services which we could use in our evaluation. We used Wireshark version 1.2.1 for Windows from “http://www.wireshark.org/” to capture data traffic in a small network with two nodes. This enabled us to capture actual WS-Discovery traffic, and examine the packet payload sizes, thus giving a foundation for further analytical study.

FFI-rapport 2014/01454

31

Compression

Minimum size

Maximum size

Average

Standard deviation

Median

Uncompressed

807

887

834

17,04

830

EXI

373

420

390

9,22

388

Table A.2

A.4.2

HELLO message payload statistics (in bytes), calculated from HELLO messages corresponding to the Web services described by our 100 test WSDL files. For each Web service that is published, WS-Discovery sends two identical such messages.

WS-Discovery network usage evaluation

The standardized WS-Discovery behavior is a decentralized discovery protocol. Services are required to send UDP multicast HELLO messages that advertise when they become available. Also, services should send UDP multicast BYE messages when they go away. If services are able to do this, then each node will have an up-to-date view of the available services. In a dynamic network we cannot rely on receiving all such messages. It is also possible to actively query the network by sending PROBE messages. In order to accurately mirror the current network state of a dynamic network probing must be used, in which case each node replies with UDP unicast PROBE MATCH messages. This generates a lot of data traffic, but is required to ensure an upto-date view of the available services. The standard requires all multicast packets (i.e., HELLO, PROBE, and BYE messages) to be sent twice, and the unicast PROBE MATCH messages to be sent once. Since we are concerned with WS-Discovery in dynamic environments, we focus on the HELLO, PROBE, and PROBE MATCH messages in this paper. WS-Discovery is based on a query-response model, where a multicast query (probe) triggers unicast responses (probe match). The load incurred on the network by the number of querying nodes (q) in a network with a total number of n nodes can be calculated using the formula below. If all nodes should have an up-to-date view of the currently available services, then q = n, conversely, if only one node is querying, then q = 1.

LOAD = (sizeof(probe) + sizeof(probe match) * (n - 1)) * q

In our tests the HELLO messages yielded different sizes (see Table A.2 depending on the different WSDLs that were published (two HELLO messages generated per WSDL). An uncompressed PROBE message was always 581 bytes (using a generic probe querying for all available services with no scope limitations). An EXI compressed PROBE message varied between 272 and 274 bytes (compression varying with varying UUID and time stamp in message; for simplicity we assume a compressed size of 273 in our calculations below as this is the average over time). According to the standard the message had to be transmitted twice, meaning that sizeof(probe) = 2 * 581 bytes for uncompressed traffic (EXI compressed sizeof(probe) = 2 * 273 bytes). The PROBE MATCH varied in size with the number of services published, since it contained all the services published by a node. Table A.3 shows the different sizes of PROBE MATCH

32

FFI-rapport 2014/01454

Number of services

Uncompressed PROBE MATCH

EXI compressed PROBE MATCH

100

38200

5009

80

30292

4000

60

22902

3146

40

15531

2339

20

8122

1552

10

4476

1072

1

1092

511

Table A.3

The size (in bytes) of the unicast PROBE MATCH message sent by a node publishing a certain number of Web services with WS-Discovery.

messages sent by a node with 1 to 100 services published. We see that publishing just one service incurs a lot of overhead (1092 bytes to disseminate information about it), whereas for a larger number of services this overhead is reduced (more actual Web services porttype information in the response compared to SOAP headers, etc). Calculating the average when publishing multiple services (i.e., the average of the message sizes divided by the number of services) yields 497 bytes for uncompressed WS-Discovery, and 130 bytes for EXI compression. UDP can carry a payload of 65507 bytes, meaning that WS-Discovery has a theoretical upper limit of publishing approximately 65507/497 ≈ 131 Web services per node when considering results from our 100 WSDLs. Conversely, with EXI compression we may publish around 65507/130 ≈ 503 Web services per node. Naturally, the number is approximate, because in practice varying namespace lengths in different WSDLs can affect the PROBE MATCH size. We can also see that an increase in the number of services in a PROBE MATCH leads to an increased compression rate, because of recurring patterns in the XML encoding of the service information. For just one service, the compression rate is 511/1092 ≈ 0.47, whereas for a 100 services the compression rate has increased, yielding 5009/38200 ≈ 0.13. Using the LOAD equation, we fill in values for sizeof(probe match) using values from Table A.3, as well as the above mentioned sizeof(probe). The number of nodes in the network, n, is varied from 1 to 250. First, we set q = 1, meaning that only one node is querying. Figure A.1 illustrates WS-Discovery’s resource use (in megabytes) in this case when one node is querying in networks with up to 250 nodes. This means that in such a network, for every query issued, we get the resource use indicated by the graph. Next, we set q = n, so that in a network of a given size, all nodes query. In both cases, this means that the querying node(s) send PROBEs and receive(s) n PROBE MATCHES. Figure A.2 shows WS-Discovery’s resource use in the case where q = n. In both graphs, all nodes are equal and publish the same number of services. Please note that the graphs have a logarithmic Y-axis to ease comparison between uncompressed and compressed results. We see that with an increasing number of nodes and published services, the overall resource use increases substantially.

FFI-rapport 2014/01454

33

Figure A.1

WS-Discovery’s resource use when one node queries.

Figure A.2

WS-Discovery’s resource use when all nodes query.

34

FFI-rapport 2014/01454

A.5

Conclusion

Standards are preferable to proprietary solutions because they ease interoperability and reduce the chances of vendor lock-in. Though WS-Discovery in our previous research has proven itself to be less than optimal for use in tactical networks, its resource use is significantly reduced when coupled with EXI as our results in this paper show. Thus, WS-Discovery could well be of value in a civil-military operation, where it could provide a standardized Web services discovery capability for both the civil and the military dynamic networks.

FFI-rapport 2014/01454

35

Appendix B

CoNSIS: Demonstration of SOA Interoperability in Heterogeneous Tactical Networks

This appendix summarizes the experiments performed in context of CoNSIS with respect to SOA interoperability. It was published and presented at IEEE MCC 2012 in Gdansk, Poland (see [6]). B.1

Introduction

The Service-Oriented Architecture (SOA) concept, most commonly implemented as Web services, is seen as a key enabler for meeting the technical interoperability requirements needed to achieve the NATO Network Enabled Capabilities (NNEC) vision. Within NATO, Web services technology has been the focus of the Core Enterprise Services (CES) working group, which has defined a number of common infrastructure services as core enterprise services. Having a common understanding of how these services are to be implemented and used is critical when attempting to achieve interoperability across national and systems boundaries. An important part of the work towards achieving this common understanding is to utilize these services in experimentation, in which the candidate technologies are tested under conditions similar to those found in an operational network. The Coalition Network for Secure Information Sharing (CoNSIS) is a multinational group consisting of members from Germany, France, USA, and Norway, with participants from both research institutions and industry. The objectives of this group are to develop, implement, test, and demonstrate technologies and methods that will facilitate the partners’ abilities to share information and services securely in ad-hoc coalitions, and between military and civil communication systems, within the communications constraints of mobile tactical forces. The group has focused on practical application of information infrastructure technologies in a network-of-networks, consisting of a variety of low capability network technologies. The work done within the CoNSIS group has been divided into a number of tasks, each focusing on a different aspect of interoperability issues. This paper focuses on the work done by Task 2, which, together with [46], covers the domain of SOA and its application in limited capacity networks. During June 2012 CoNSIS conducted a large-scale experiment in Greding, Germany, in which all the different aspects of technical interoperability were tested; integrating the work of all the task groups of CoNSIS. B.2

Background

For Task 2, the goal of the CoNSIS experimentation was to show that by using the Web service standards specified by the NNEC CES as interoperability enablers, independent implementations are able to interoperate with only the service specifications as a common reference. The reasoning behind focusing on standards as a means of achieving interoperability was that it enables us to evaluate not only the implementations used, but also the standards themselves. In other words, we can assess how suitable the standards specified in the NNEC CES are for use in tactical networks.

36

FFI-rapport 2014/01454

A. SOA challenges Assessing Web services (and other SOA implementation approaches) through the use of standards based experimentation is not new; multiple other experiments covering several topics of Web service interoperability have been performed previously. One of the most recent of these was the “Making Services Interoperable”-experiment conducted by the NATO RTO IST-090 working group last year [24]. In this experiment, three different SOA-based information infrastructures were connected together and interoperability was achieved. This experiment does however differ from the CoNSIS experiment in multiple ways: • The CoNSIS experiment network was considerably more complex than the network used during the IST-090 experiments. This added complexity meant larger fluctuations in the network conditions experienced by the Web service frameworks being used. • The IST-090 experiment focused on service invocation, while the CoNSIS experiment covers both service discovery and service invocation. • IST-090 managed to achieve interoperability between both Web service and Data Distribution Service (DDS) implementations of SOA, but this required the use of specialized gateways which only supported some service types. The CoNSIS experiments are limited to Web service based implementations only, which allows for a more generic approach to service interoperability. • The CoNSIS experiment was performed using an IPv6 network, while the IST-090 experiment was done on IPv4. This difference meant that there were additional challenges imposed on the CoNSIS experimentation as the support for IPv6 in pre-existing software and software development tools is limited. There were primarily two NNEC core services that we wanted to test with respect to interoperability, namely service discovery and publish/subscribe. In addition, we also saw the need for focusing on topics, which is a method for classifying information, and as such constitutes the intersection between service discovery and publish/subscribe. These three areas are further described in the following sections. B. Experimentation Networks The CoNSIS land mobile part of the network consists of contributions from Germany and Norway. In addition to four German and four Norwegian mobile nodes located on military vehicles, there was also an NGO vehicle. However, this NGO vehicle was not used in the SOA experiments and will therefore not be further described in this paper. Figure B.1 shows the parts of the CoNSIS network that was being using during the SOA experiments. Two different radio types were used. This is partly because Germany and Norway do not have the same radio systems, but also because different wireless technologies, having different properties with regards to e.g., range, are needed. Together the two radio links form a heterogeneous network. Within the German part of the convoy the IABG HiMoNN radio was used, whereas the Kongsberg

FFI-rapport 2014/01454

37

Figure B.1

CoNSIS network, initial configuration.

WM600 radio was used within the Norwegian part of the convoy. To interconnect the two parts one German radio was placed on a Norwegian vehicle and one Norwegian radio was placed on a German vehicle. In addition to the vehicle nodes, one mobile network node was physically co-located with the deployable HQ network. This node connected the mobile network to the rest of the CoNSIS network, and was connected to mobile nodes terrestrial radio links as indicated. The eight vehicles formed two convoys, one German and one Norwegian. In the scenario, these two convoys were separated from the start, but at a later point in time, they merged into one large convoy. In addition, the network topology changed during the experiments, something that affected delay and available bandwidth. One such alternative topology is shown in Figure B.2. Germany and Norway each provided implementations of selected parts of the NNEC CES, using Web services technology as their foundation. The two implementations were technologically quite different, as the German implementation, Referenzumgebung Dienste (RuDi) [46], is based on an Enterprise Service Bus (ESB), while the Norwegian solution consisted of multiple stand-alone components implementing the set of core services. In the experiment, the core services were used to provide a infrastructure for functional services providing three types of information:

• Vehicle positions: Each vehicle reported its position (through a GPS-based positioning service) to the lead vehicle of its convoy. The lead vehicle then aggregated the positions of all convoy vehicles into a convoy Common Operational Picture (COP), which in turn was delivered to the HQ, where the two convoy COPs were aggregated into a full COP. This full

38

FFI-rapport 2014/01454

Figure B.2

CONSIS network, alternative network topology.

COP was then distributed back to the vehicles. • Operational messages: This is a service for sending messages to other users. The messages can be of the types Alert, Warning, Information, and Command. File attachments are also possible. • Chat: This service provides ordinary chat functionality, based on chat rooms.

All information types are distributed using WS-Notification, which is the Web service standard for Publish/Subscribe that is specified by the NNEC CES. Furthermore, all information types were distributed among all the vehicles as well as the HQ, as illustrated in Figure B.3. B.3

Service discovery

Service Discovery is the process of finding the services that are available in the network. NNEC CES has recommended the use of the registry based solution Universal Description, Discovery and Integration (UDDI) for this core service. However, when operating in a wireless network environment where node mobility and shifting network conditions can cause network partitions and loss of network connections, it is vital to use a service discovery mechanism that does not rely on the availability of any given node. In other words, we need a fully distributed service discovery mechanism [31]. The only standardized Web service discovery protocol that currently fulfills this requirement by operating in a distributed mode is WS-Discovery [37], which was utilized during the SOA experiments. WS-Discovery is designed for use in one of two modes: managed and ad hoc. In managed mode all nodes communicate through a discovery proxy, an entity which performs the service discovery

FFI-rapport 2014/01454

39

function of behalf of all the other nodes, and which communicates with the other nodes using unicast messages. This mechanism can be used to achieve interoperability between registry based service discovery mechanisms and WS-Discovery. In ad hoc mode, on the other hand, communication is fully distributed. Requests for service information are sent using multicast to a known address, and each node is responsible for answering requests from others about its own services. The ad hoc mode is intended to be used for local communication only, and the standard recommends limiting the scope of multicast messages by setting the time-to-live (TTL) field of the IPv4 header to 1, or by using a link-local multicast address for IPv6. The CoNSIS experiment consists of a number of ad hoc networks connected to each other using Multi-Topology Routers (MTRs) [16], forming a IPv6 based network-of-networks. The dynamic character of these networks implies that one cannot rely on a managed mode discovery proxy to remain available, meaning that the distributed ad hoc mode should be used. However, since this mode is limited to link local communication it will not provide the multi-network service discovery capability required in the CoNSIS experiments. In order to work around this issue, we decided to go against the recommendations in the standard, and allow the multicast discovery messages to travel across network boundaries by using a site-local IPv6 address, and increasing the Hop Limit in the IPv6 header. This solution works within a controlled network environment such as the one used during the CoNSIS experiments, but it is less than ideal for use in larger scale networks. That is because increasing the scope of the multicast messages might cause the messages to travel further than intended, and thus cause increased network load in networks where the messages are not needed. WS-Discovery is a hybrid discovery protocol, meaning that it has both a proactive and a reactive element (see [31] for further details on the different types of discovery protocols). The proactive element consists of the HELLO and BYE messages nodes send out when they first make a new service available, and when they remove a service, respectively. Other nodes then store the information gathered from these proactive messages, allowing them to perform service discovery without having to actively query for information. This proactive mode works well under stable network conditions, since the likelihood of these messages reaching all other nodes is high. The CoNSIS network is however not stable, which means that many of these messages will be lost, rendering the proactive element of WS-Discovery unable to provide service information to all nodes. This means that one has to rely on the reactive element of WS-Discovery, the PROBE and PROBE MATCH messages. In this reactive mode a node that requires the use of a service will ask for services matching its needs by sending a PROBE message. This message is sent using multicast, and with the extended scope of multicast messages described above, the probe will reach all other nodes that it currently has a network connection to. Nodes offering a matching service send a unicast PROBE MATCH message back to the probe sender. Note that this reactive mode should be used sparingly in low capacity networks as it generates some network traffic.

40

FFI-rapport 2014/01454

Figure B.3

Information types and flows.

The flow of WS-Discovery multicast messages is illustrated in Figure B.4. Since we allowed the packets to flow across routers, a request sent by any one node in the network is received by all other nodes. If the message sent was a probe for available services, then all nodes that did offer a service matching the request would reply with a unicast message to the sender. On the German side, WS-Discovery was integrated into the RuDi system, and connected to the internal service registry. This meant that any announcement made on WS-Discovery would be added to the service registry, which in turn meant that the announced service could be invoked from within RuDi. This was done by allowing RuDi to periodically probe for information, but at a low enough frequency to not cause overloading the network. On the Norwegian side, WSDiscovery was used as the only discovery mechanism. A self-contained WS-Discovery application was therefore used for announcing and searching for services, which made it possible to limit the amount of probes sent into the network by only probing when a new service was needed. As mentioned above, allowing the multicast packets to traverse routers is not an ideal solution. An alternative is to combine the managed and ad hoc modes in one deployment. When a WSDiscovery proxy announces its presence, all other nodes are asked to enter managed mode, relying on the proxy for service discovery. However, the WS-Discovery specification does not require the nodes to change to managed mode, and by allowing the majority of nodes to remain in ad hoc mode and at the same time keep a link local message scope, one can secure local service discovery without the risk of generating unneeded network traffic in other networks. Combined with discovery proxies that function as relays between the networks, cross-network discovery can be achieved as well. Note that, even though the WS-Discovery specification does allow nodes to choose not to enter managed mode when receiving a message telling it to do so, it does not clearly state what the expected behavior of nodes is once the network consists of nodes in both modes simultaneously. This combination of modes is desirable when working with multiple interconnected mobile networks, and therefore a profile of how to use the WS-Discovery standard in this context should be developed by NATO for interoperability between nations.

FFI-rapport 2014/01454

41

Figure B.4

B.4

WS-Discovery information flow.

Publish/subscribe

In the CoNSIS experiment the majority of the information exchanged was distributed according to the publish/subscribe paradigm. This means that instead of a node having to repeatedly check if there is new information, the node simply send a subscription request to the information provider, asking to be notified whenever new information is available (see Figure B.5). Using Publish/Subscribe instead of the Request/Response paradigm has several advantages: The network traffic is reduced, since the client doesn’t have to send periodic requests; the server load is reduced, since there are fewer requests to process; and the client will potentially receive new data sooner, although this is dependent on the request frequency in a Request/Response setting (which in turn will affect network and server load). WS-Notification [38] is an OASIS standard and consists of a group of specifications that enable Publish/Subscribe-based communication between Web services. It comprises WSBaseNotification, WS-BrokeredNotification and WS-Topics. While WS-BaseNotification defines which interfaces consumers (clients) and producers (servers) should expose, WSBrokeredNotification introduces the concept of a message broker, an intermediary node which decouples consumers and publishers, and relieves producers from several tasks associated with Publish/Subscribe. NNEC CES specifies WS-Notification as the standard to be used for Publish/Subscribe in NATO. The notifications are normally always of the same type, independent of the actual information that is delivered (i.e., the payload of the notification). When a client wants to subscribe to a specific type of data, it therefore expresses the type of information it is interested in by including a topic in the subscription request. For WS-Notification, the WS-Topics standard specifies how such topics should be expressed. It also defines three topic expression dialects, to allow for expression topics of different complexity. This use of topic dialects means that one can express a number of different topic structures within the same standard, including define one’s own dialect for handling topics. This topic handling

42

FFI-rapport 2014/01454

Figure B.5

Request/Response versus Publish/Subscribe.

scheme is flexible, but the added complexity using such topics means that one needs to agree not only on which topics to use, but also which dialect they want to use to express topics before communication is possible. One added complexity when using WS-Notification in a limited capacity network it that the standard is designed to use unicast message transmission only. That means that, even when multiple nodes in the same network want the same information, WS-Notification will send one unicast message to each recipient rather than send one multicast message that reaches all recipients. In radio based networks, where the transmission medium is shared, there is a potential for a significant reduction in network load by switching from unicast to multicast. Note that making such as switch will require further functionality to be implemented into WS-Notification, namely the ability to manage multicast group memberships. A. Norwegian Infrastructure The Norwegian infrastructure used during the CoNSIS experiments consists of a number of internally developed components. The core of the infrastructure is an implementation of the core features of WS-BaseNotification and WS-BrokeredNotification. This Java implementation has support for subscribing and unsubscribing, as well as for receiving and sending notifications. While not being a full implementation of the WS-Notification standard, this light-weight solution is well suited for use in test environments like the CoNSIS network. During the experiments all the Norwegian units were running a notification broker, and all clients and services on a node subscribed to, respectively delivered notifications to, its local broker. Between nodes, the brokers subscribe to each other. Web services, including WS-Notification, normally relies on HTTP over TCP for message delivery, meaning that it is necessary to establish an end-to-end TCP connection between client and

FFI-rapport 2014/01454

43

Figure B.6

The viewer component.

service. In the CoNSIS network this means that TCP connections in many cases would have to be established across several radio networks with unstable links and often very high delays. To enable standards-based Web services over such connections, we used our Delay and Disruption Tolerant SOAP Proxy (DSProxy) [32]. These proxies constitute a middleware that hides network delay and disruptions from the applications and also compresses all traffic, allowing XML to be sent over low bandwidth connections. B. German Infrastructure The German infrastructure consists of a complete SOA environment, RuDi, covering a range of functionality in addition to the aspects described here. For a more elaborate description of RuDi, including the German national security experiments conducted during CoNSIS, refer to [46]. In order to connect the Norwegian and German infrastructures together, ensuring reliable message delivery in an unstable environment, the DSProxy was used. RuDi supports the use of multiple transport protocols at the same time, and by including the DSProxy as one of these transport options, connectivity between the Norwegian and German infrastructures was achieved. C. Experiment Information Flow In addition to these infrastructure components, each vehicle had a GPS component that reads the vehicles position from a GPS, creates an NFFI message, and delivers this as a notification to the local broker. Furthermore, there was a component for creating Operational Messages, and delivering these as notifications to the local broker. There was also a chat component, which both subscribed to, and delivered notifications to, the local broker. Next, there was an aggregator function that subscribed to the position of each vehicle, and then combined all vehicle positions into one NFFI message which was then delivered to the local broker as a notification. Finally, there was a viewer application, which subscribed to NFFI tracks and Operational Messages from its local broker, and displaying them on a map (see Figure B.6).

44

FFI-rapport 2014/01454

Figure B.7

Initial configuration of flow position information.

As mentioned earlier, the COP was built in two steps, with the lead vehicle of each convoy building a convoy COP by subscribing to the positioning service of each vehicle, and then the HQ building the full COP by combining the two vehicle COPs. This is illustrated in Figure B.7, and represents the initial flow of position information. Later in the scenario, the two convoys merged into one. However, since the communication between the two convoy elements was done via the HQ, information flow between them was prone to disruptions and delays even when the two convoy elements were traveling together. In order to improve upon this situation, and take advantage of the higher capacity network connections now available within the convoy, we then changed the information flow: The two lead vehicles stopped subscribing to the full COP from the HQ, and instead started subscribing to each other’s vehicle COPs. The full COP could then be built locally at each lead vehicle, and then distributed to the other vehicles in the convoy (see Figure B.8). Due to the flexibility of WS-Notification this change in information flow was easily performed, with the added benefit of both an improved response time for positional updates within the convoy, and less traffic load on the narrow reach-back links. Thus, because the notification interface is the same, regardless of information type, any subscriber can subscribe to and receive notifications from any broker, as long as the business logic behind is able to parse the payload of the notification. B.5

Topic handling

All Publish/Subscribe systems require a mechanism for describing content of interest, and WSNotification uses topics for this purpose. A topic is a way of classifying content into logical channels, and topics are usually organized into hierarchies. Thus, the highest level topic, the root topic, represents the most general classification, and then an arbitrary number of subtopic levels refine this classification. This organization of information flows based on topics is fundamentally different from the Request/Response paradigm: When looking for a Request/Response service you are interested in a service with a particular interface. This is because that interface is the only aspect of the service

FFI-rapport 2014/01454

45

Figure B.8

Flow of position information after merging of convoys.

that is known to the consumer, and thus represents the only interface that consumer is about to invoke. The service description for a service, the WSDL file, does not contain information about the actual content provided by the service. For Publish/Subscribe, on the other hand, all services are equal with respect to the actual interface, and you need information about the content the service offers in order to distinguish between content providers. A consequence of this transition from Request/Response to Publish/Subscribe is that traditional service discovery becomes less useful. This is because all Publish/Subscribe endpoints will appear as the same service type, generating a need for additional meta-information about services, namely the topics. This shift in interest from service types to information types makes topic discovery an important issue when dealing with WS-Notification. In [22] we have described how WS-Discovery can be used to distribute information about which topics a service covers, while at the same time remaining backwards compatible with the WS-Discovery standard. As a preparation for the CoNSIS experiment, initial testing with topic discovery was performed at the Coalition Warrior Interoperability Experiment (CWIX), where WS-Discovery with topic support was tested by multiple partners. During this initial testing, as well as during the CoNSIS experiment execution, we discovered that while this approach provides enough information for nodes to be able to distinguish between the content offered by the different providers, certain extra functionality is desirable. In particular for notification brokers (as described in the previous section), which can serve many nodes and offer information on many different topics, it would be very useful to be able to query the broker itself about topics: For instance, which topics a broker currently provides notifications on, which topics it knows about (i.e., has seen at some point), and if and when it has last seen notifications on a given topic. One challenge when working with topic based information exchange is that it requires all the involved parties to have prior knowledge about how topics are organized. In order for an information consumer to get the information it desires, it needs to know in advance which topic to request from the broker. In the CoNSIS experiment we were working with two partners, making

46

FFI-rapport 2014/01454

a priori distribution of topic information possible. We decided that a client normally needs information following a given schema, and we therefore chose to have a 1:1 relationship between root topic and the XML Schema of the information in question. Thus, we had the root topics “nffi”, “OpMsg” and “Chat”. However, in other contexts, other classifications may be better suited. In general, for larger scale implementations of topics, it is necessary to utilize a common information model that describes how topics are organized. This means that NATO should be the driving force behind such a model, which would then be used by all member nations. B.6

Conclusion

Performing practical experiments with the technologies that will form the foundation of future operational networks is vital to ensure that these technologies will be capable of meeting the interoperability requirements of complex operations. During the CoNSIS experiment we had the opportunity to test Web services in a complex network, allowing us to verify that Web services can be used as an interoperability enabler also in limited capacity tactical network. Using the Web service standards as the common reference between nations made interoperability possible, but there is a need for further development and profiling of standards in order for them fully support the interoperability challenges faced by the nations. Due to the potential performance benefits of using the Publish/Subscribe paradigm in tactical networks, use of the WS-Notification standard is recommended. To be able take full advantage of the benefits of Publish/Subscribe however, multicast support for notification should be implemented. In addition, topic handling must be addressed, preferably by introducing a NATO recommendation addressing both the issue of incompatibilities between different topic expression dialects, and containing a common topic vocabulary and structure.

FFI-rapport 2014/01454

47

Appendix C

Topic Discovery for Publish/Subscribe Web Services

This appendix summarizes our work on topic discovery using WS-Discovery in conjunction with WS-Notification. It was published and presented at IEEE IWCMC, Limassol, Cyprus, August 2012 (see [22]). C.1

Introduction

In recent years, the Service Oriented Architecture (SOA) approach to designing and implementing large distributed systems has become mainstream, and Web services have grown to become the industry standard for implementing such systems. In addition, there has been an increase in the number of mobile devices that have the capability of connecting to these service oriented systems, leading to new challenges for system designers. Using Web services in mobile networks raises a number of challenges, such as limiting resource usage and ensuring that the services discovery mechanism provides accurate and up-to-date information about available services. Reducing the network resource demands of Web services can be done in a number of ways, for instance by using techniques such as compression [44] and notification based information dissemination [50]. When using the publish/subscribe message exchange pattern, nodes will automatically receive new information when it becomes available, thus removing the need for nodes to constantly have to check for new information. In [9], Costa et al. point out that this message exchange mechanism is particularly suited for mobile ad hoc networks (MANETs). This is because publish/subscribe decouples the provider from the consumer, thus removing the tight coupling that exists between these in the request/response pattern. In addition, fewer messages have to be transported between the consumer and the information source since network traffic is only generated when new information enters the system. Information dissemination in a publish/subscribe system is controlled through the use of a filtering mechanism, which can be either content or topic based. A content based system requires the subscriber to specify the attributes of the content it is interested in receiving. A topic based system divides messages into logical channels called topics. Here, the publisher is responsible for both defining the topics and for classifying its messages according to these topics. This is somewhat less flexible than a content based system, but has lower computational overhead and is easier to manage. In Web services, there are two competing specifications for publish/subscribe, namely WSEventing and WS-Notification. Both of these are topic based and provide basic notification support, but WS-Notification also specifies the usage of notification brokers, and comes with its own topic specification. One open issue when using these Web services specifications for publish/subscribe is the aspect of service discovery. In a topic based system, the information consumer specifies which topics it is interested in receiving information about rather than who it wants to receive information from. The Web service discovery standards, however, are designed to

48

FFI-rapport 2014/01454

allow for the discovery of information sources rather than information topics. This means that one can use these specifications to easily find endpoints that are notification producers, but they do not specify how the consumer can find out which topics each of these endpoints produce information about. The lack of topic discovery is addressed in this paper. Previously, we have investigated a solution for Web services publish/subscribe in dynamic networks [48]. Then we looked into enabling publish/subscribe, but did not focus on finding the topics provided by each publisher – this information was assumed to be known a priori, as is the case for the current use of Web services standards for publish/subscribe. However, in a dynamic environment where nodes can come and go you may require to switch to a new publisher if the old one goes away. Also, with several different publishers in the network there is the dilemma of which one to choose: They all implement the same service interface, provided to you by a service discovery mechanism. But which server provides updates on which topics? Currently, there is no support for topic discovery for Web services publish/subscribe. The contribution of this paper is a design of a topic discovery mechanism for Web services, with special emphasis on Web services standards and retaining backward compatibility with existing solutions. Furthermore, we have implemented our design as a proof-of-concept. The remainder of this paper is organized as follows: In Section C.2 we present related work. Our design and implementation are presented in Sections C.3 and C.4, respectively. Section C.5 concludes the paper. C.2

Related work

Discovery of Web services can be performed either by utilizing a service registry, or a decentralized non-registry based solution. There are three standards addressing Web services discovery, two registries and a non-registry solution. The registries, UDDI [39] and ebXML [35], suffer from liveness and availability problems in dynamic environments [28]. The third standard for Web services discovery is WS-Discovery [37]. It is better suited to dynamic networks than the registries in that it is a decentralized discovery mechanism, thus removing the single point of failure that a centralized registry constitutes. Since our main concern is publish/subscribe in dynamic environments, we focus on using the WS-Discovery standard for service discovery in this paper. Currently there are two competing specifications adding support for the publish/subscribe paradigm to Web services, namely WS-Notification [8, 12, 49] and WS-Eventing [10]. The WS-Notification standard consists of three parts: 1. WS-BaseNotification [12] 2. WS-BrokeredNotification [8] 3. WS-Topics [49] The first document covers the basics such as message format and rules concerned with creating

FFI-rapport 2014/01454

49

Figure C.1

Web services lifecycle (adapted from [7])

subscriptions and how to notify consumers. WS-Eventing covers the same aspects as WSBaseNotification, but the syntax is different and the two are incompatible. The second document extends WS-BaseNotification and introduces brokers, i.e., proxies or intermediaries allowing a publishing chain to be built. The third and final document describes different ways to express the topics used for categorizing and publishing content, along with sets of rules for topic matching. Note that WS-Notification is by far a more complete framework for publish/subscribe than WSEventing, and it is also the only of the two specifications that is currently standardized. As a consequence, we focus on WS-Notification in this paper. There exist many experimental publish/subscribe frameworks, e.g., [9, 11, 19, 48]. However, to our knowledge none of the current published proposed solutions address topic discovery – they focus on optimizing the notification dissemination in networks by attempting to reduce overhead and increase reliability. Thus, the work presented in this paper is orthogonal to existing solutions, in that we propose a method for topic discovery in conjunction with service discovery. Our design and implementation are geared towards Web services, but in principle the same ideas can be applied to other topic based publish/subscribe frameworks as well. For example, it should be possible to extend e.g., the Service Location Protocol [13] with similar functionality as we implement for WS-Discovery in this paper. However, since our focus is on Web services we limit our efforts to supporting the Web services standards. C.3

Design

When considering adding topic discovery to Web services publish/subscribe, there are several possible approaches. We considered two of these, namely 1) adding support for asking a notification producer about which topics it covers, or 2) including information about topics in the service discovery phase of the Web service lifecycle. The first option, integrating topic information with WS-Notification, would require us to add a new operation to the Notification Producer interface. Potential consumers would first discover the producer using a service discovery mechanism, and then directly query each producer about which topics it offers. This would mean adding a new step to the Web services lifecycle shown in Figure C.1, by placing a topic checking step after the Discovery step. Adding this step to the discovery process would be backwards

50

FFI-rapport 2014/01454

compatible, since consumers that do not know of this topic information option could still subscribe if they know the topics offered up front. This mechanism would however both require additional network resources, and the use of a proprietary operation not supported by the WS-Notification standard. The second option, integrating topic discovery with the already existing service discovery mechanism minimizes the issue of added network traffic. The mechanism would be required to distribute the extra metadata needed to express topics, but this will be significantly less than having an additional step to the discovery process. The downside to adding topics to the service discovery mechanism is that there is no one universally used service discovery mechanism, meaning that topic support will have to be added to multiple independent mechanisms. Amongst the available standardized Web service discovery mechanisms, the registries allow for the inclusion of additional metadata, although using a proprietary data format. Registries are, however, not particularly suited for use in mobile networks where the availability of services changes frequently [28]. Because of this, we decided to focus on WS-Discovery. C.3.1

Adapting WS-Discovery

The loose coupling offered by Web services means that strict adherence to standards is essential for remaining interoperable with others. Thus, we chose a design that means our enhanced WSDiscovery solution can function together with standard WS-Discovery without any adverse effects. First we need to consider the available options for adding topic information to the WS-Discovery data formats. The standard allows for three different approaches: 1. WS-Discovery uses WS-Addressing, and this standard has an optional Metadata field. 2. There is the option of adding a proprietary metadata field to the WS-Discovery message outside the WS-Addressing header. 3. The standard has two already existing metadata fields, which should be considered; the Types and Scopes fields. Option one relies on the use of a generic metadata field, in which any XML-statement can be added. This allows for a high level of flexibility, but has the downside that the formatting of the content in this field is proprietary. Furthermore, WS-Discovery does not give any guidelines as to how this type of metadata should be handled, beyond recommending that full metadata should be included in its messages. The second option has the same downside as the first, but unlike option one it does not utilize an already existing message field, but instead introduces its own. WS-Discovery does allow for this, but it is not specified how such a field should be handled during implementation. The last option is to look at the existing metadata fields within WS-Discovery, namely Types and Scopes. Types is used for relaying which PortTypes the service implements (in our case this will

FFI-rapport 2014/01454

51

be the NotificationProducer type from WS-Notification), and can thus not be used. Scopes, on the other hand, is intended to be used to divide services into logical groups. In order to determine if a service falls within a given logical group, one utilizes a set of matching rules defined in the standard, e.g., RFC2396, LDAP, UUID (see [37] for the complete list). Furthermore, the standard specifies that one can have a set of scopes for each service, but that a match is only achieved if the query matches all the individual scopes within that set. The Scope concept, in which a constraint is tested using a matching rule, is suitable for the topic matching needed, but the scopes and matching rules defined in the standard do not support our matching rules. The reason for this is twofold: We need to be able to achieve a match when a query matches just one of many topics a Notification Producer might support, and we want to be able to structure our topics according to a rule that WS-Notification supports. In order to achieve this, we chose to implement our own scope and matching rule, as allowed by the standard. At first glance this might seem no better than utilizing a proprietary metadata field, but the main benefit of using a self-defined scope is that the WS-Discovery standard explicitly states how unknown scopes are to be treated. The presence of an unknown scope will lead to a mismatch between a query and a service. This means that a standard WS-Discovery implementation can co-exist with an implementation that supports additional scopes without there being any risk of adverse effects on either mechanism. WS-Discovery supports two modes of operation, managed mode and ad hoc mode, and it defines a number of different message types to support these modes. The Hello and Bye messages are used to announce when a new service first becomes available, when a service changes its metadata and to signal that a service is no longer available. The topic information suggested in this paper will be included in the Hello messages, so that nodes in the network can then cache this information for later use. In a mobile network, however, it cannot be assumed that all nodes have received these proactive messages, and one has to rely on the reactive elements of WS-Discovery, namely the Probe and Probe match messages. In the following paragraph we explain how topic information is included in these message types, though the same approach is also used for Hello messages. In the WS-Discovery standard a Probe includes zero, one, or two constraints on matching Target Services: a set of Types and/or a set of Scopes [37]. Here, «Types» refer to PortTypes, i.e., a reference to the concrete service interface you are looking for. Since we are interested in services providing the WS-Notification subscription interface, we should look for services of the type «NotificationProducer». The «Scopes» field may contain any URI, and a specification of which matching rule the recipient should use. This field is intended for different uses, and the standard defines several matching rules that can be used. As previously mentioned, all existing rules support specific use cases and are not suitable for scope matching. However, the standard allows new matching rules to be defined, provided they are assigned a new identifier. Also, the standard requires nodes receiving a Probe using a matching rule that it does not understand to treat all its services as a mismatch, thus ensuring backward compatibility. This means that our modified WS-Discovery instances can co-exist in networks with instances just implementing the current standard without any adverse effects.

52

FFI-rapport 2014/01454

C.3.2

Topic matching

The WS-Notification standard includes WS-Topics, which defines several topic types. WSBaseNotification requires support for the SimpleTopic type, which allows for a direct matching between topic expressions consisting of a single text string. This would mean that one can only match root-level topics in a topic hierarchy, which is too restrictive for our usage. In order to properly express both general topics and more specific topics, we require support for sub-topics as well. This is supported by the ConcreteTopic type, which extends the type supported by SimpleTopic to include not only root topics, but also allows for sub-topics and partial prefix matching. The partial matching supported by ConcreteTopic is limited to prefix matches, which means that a search for a general topic type will match more specified sub-topics of that general type. We consider this topic type sufficient for our experiments, and we have thus chosen to implement this type. For information about further topics types, refer to the WS-Topics [49] standard. The WS-Topics standard states that: The Concrete Topic Expression is used to identify a single Topic within a Topic Namespace, using a path notation. As it uses a path notation, it can identify any Topic within a Topic Namespace – it is not limited to root Topics. A TopicExpression in this dialect is a token (as defined by XML Schema) with an additional constraint on its format. The constraint is the token must contain a TopicExpression. The pattern allows strings matching the following simple Extended Backus Naur Form: ConcreteTopicPath ::= RootTopic ChildTopic* RootTopic ::= QName ChildTopic ::= ’/’ (QName | NCName)

This is the rule that we support in our implementation. C.4

Implementation

Our concrete topic match implementation is a custom match rule for topic discovery. It allows scopes to match valid ConcreteTopicExpressions from the WS-Notification standard. It will signal a match if any topic (or prefix thereof) in a comma separated topic list in the scope section is a match according to the ConcreteTopicExpression dialect. We based our prototype on the open source implementation of the WS-Discovery standard from http://code.google.com/p/java-ws-discovery/. We used the newest source code from the repository, which at the time of writing this paper was revision 128. The release contains five folders: wsdiscovery-examples, wsdiscovery-example-ws, wsdiscovery-gui-2, wsdiscovery-lib, and ws-discovery-soap-lib. The example projects provide code showing how to use the WSDiscovery library. The GUI project provides a graphical interface to the WS-Discovery library allowing you to publish and discover services in your network. Finally, the two lib projects contain the implementations of the WS-Discovery [37] and the SOAP-over-UDP [18] standards,

FFI-rapport 2014/01454

53

respectively. Since we want to add a new matching rule to WS-Discovery, we have to modify the WS-Discovery library. Looking at the code, we found that the matchers are implemented in wsdiscovery-lib/src/main/java/com/ms/wsdiscovery/servicedirectory/matcher. There, MatchBy.java contains an enumeration of the supported standardized matchers along with the implementations of the corresponding classes and methods. We added our new concrete topic match rule to the enumeration, and placed the corresponding implementation in the same directory. The implementation was tested in two small experiments as a proof-of-concept to show the viability of our solution. C.4.1

Experiment: Legacy compatibility

Figure C.2

Custom probe, PortType=NotificationProducer

Figure C.3

Defining a custom probe

In this experiment, we tested our solution in a small MANET with laptops using 802.11b for communication. We had three nodes, two nodes were running WS-Notification and providing the standardized publish/subscribe interface (i.e., the PortType NotificationProducer). One of the nodes with WS-Notification was running the legacy WS-Discovery (WS-Discovery 1.0-beta1 GUI) from http://code.google.com/p/java-ws-discovery/, whereas the other two nodes were running our new extended WS-Discovery with topic discovery and matching capability. The node with the legacy WS-Discovery instance published its service endpoint with a blank scope, since it did not implement topic matching. The node with the new WS-Discovery instance published the endpoint not only with the appropriate PortType, but also with the scope of the subscriptions it

54

FFI-rapport 2014/01454

Figure C.4

Custom probe, PortType=NotificationProducer, scope=no/ffi

Figure C.5

Two WS-Notification capable nodes (1-2), where node one supports legacy WS-Discovery and node two supports our extended WS-Discovery.

Figure C.6

Experiment setup showing only the nodes supporting the extended WS-Discovery, the topic discovery, and the following publish/subscribe communication.

could offer, namely «no/ffi/tracks» and «no/ffi/incidents». Having set up our two WS-Notification capable nodes in this fashion, we started an extended WS-Discovery instance on the third node – the consumer – as well. First, we let the consumer issue a custom probe (using the «Send custom probe» button in the GUI) where we left the matching algorithm at the default value, let the Scope field be empty, and set the PortType field to NotificationProducer to limit the results to services implementing the corresponding WS-Notification PortType. Pressing «Send» resulted in both WS-Discovery instances sending probe matches with their services, as we can see in Figure C.2. Here the service with the empty scope is the service published by the legacy WS-Discovery library, whereas the other service comes from our modified WS-Discovery library. Second, we stopped and started the consumer’s WS-Discovery GUI to clear the local cache.

FFI-rapport 2014/01454

55

Then we issued a custom probe setting the PortType as before, but selecting our new matching algorithm http:// no.ffi/ wsn/ ConcreteTopicMatchRule and setting scope to «no/ffi» as shown in Figure C.3. Sending the probe gave us the result shown in Figure C.4. The network communication is illustrated in Figure C.5. Here we see that we got one reply – from the extended WS-Discovery library – since «no/ffi» matches at least one prefix of the topics it publishes, which is according to the concrete topic expression rule. Issuing a probe using the scope «no/test» yielded no response as expected, since it was a prefix mismatch of the topics published. This experiment showed that our modified WS-Discovery instance functioned as expected, and that it can co-exist in an environment together with standardized instances. C.4.2

Experiment: Automatic topic discovery and subscription

In this experiment we have a client (consumer) capable of subscribing to and displaying incidents. That is, a consumer of incidents published on the «no/ffi/incidents» topic. In a static network it would be possible to hardcode the publisher’s address in the consumer, so that it could start a subscription with a known provider. In a dynamic environment we cannot do this, and require the appropriate endpoint to be located at run-time. Thus, we implemented a discovery module in our client software utilizing our extended WS-Discovery library. This meant that our software could use WS-Discovery to find the endpoints of WS-Notification instances providing the NotificationProducer PortType and the «no/ffi/incidents» topic. If the result of the query was empty, then the client could do nothing. But, when one or more instances providing the desired topic was found, it should automatically start a subscription with one of the providers. Using the same setup as in the above small experiment, we now deployed the new client software on the consumer node instead of the WS-Discovery GUI. We started the client, and it found exactly one instance matching the topic it wanted. The endpoint address was then extracted from the xAddrs field in the received WS-Discovery service description, and a subscription for the appropriate topic was issued to this address. Subsequently, the client received the publications produced and sent by the NotificationProducer. Figure C.6 shows this part of the experiment. We repeated the experiment, this time with all WS-Discovery instances using our extended library. In this iteration, the client received a match from both of the other nodes since both of them provided the same PortType and scope. At this point there were two services in the WS-Discovery cache to choose from, both fitting the requirements of the client. Since this was a proof-of-concept test we did not implement any form of load balancing scheme, and merely let the client choose the first appropriate service from the cache. Yet again the client functioned as expected, thus showing the viability of our solution. As a step towards more intelligent client software, we plan to investigate the need for short subscription leases in MANETs in the future, allowing clients to re-subscribe to new NotificationProducers as old ones disappear.

56

FFI-rapport 2014/01454

C.5

Conclusions and future work

In this paper, we have pointed out the need for topic discovery in networks where Web services publish/subscribe is being used, including MANETs. Due to the dynamic nature of such networks, we need to be able to identify available topics at run-time. Furthermore, we have pointed out that current Web services standards support the publish/subscribe paradigm and the discovery of services, but that there is no way of discovering the available topics along with the service information. As a consequence, we have designed and implemented a solution for topic discovery for Web services. We leverage existing standards such as WS-Discovery and WS-Notification in our prototype, showing in a small proof-of-concept experiment the viability of our solution. Our extensions utilize the existing standards to the fullest extent. This ensures backward compatibility with existing implementations so that deployment of our solution will not be disruptive to the operation and functionality of a network. Our modified libraries can co-exist and cooperate with legacy implementations, so that clients supporting our extensions can leverage them, but legacy clients can still use all the standardized elements without any adverse effects. We have implemented support for topic discovery of topics adhering to the ConcreteTopicExpression as defined in the WS-Topics standard. In the future, we plan to extend WS-Discovery with matching rules for additional topic expression rules as well. Finally, we plan to submit our implementation to the developer of the open source WS-Discovery library we used, so that it can be made available to the research community as an extension in the future.

FFI-rapport 2014/01454

57