BACHELOR THESIS Electronic Engineering
Next Generation Remote User Interfaces for Machinekit
Author: Alexander Rössler Student ID: 1110254020 Supervisor: Beneder Roman, MSc Vienna, June 16, 2014
Kurzfassung Das Ziel dieser Bachelorarbeit ist es, eine neue Software zur Entwicklung von grafischen Benutzeroberflächen für das Maschinensteuerungsprojekt Machinekit zu entwickeln und die wichtigsten Teile des Designs und der Implementierung zu erklären. Der erste Teil der Arbeit beschäftigt sich mit einer kurzen Einführung in das Machinekit Projekt und einer Übersicht über die aktuell verfügbaren Werkzeuge zur Entwicklung von MachinekitBenutzeroberflächen. Im nächsten Abschnitt werden Konzepte und Vorraussetzungen für die Entwicklung beschrieben. Dies umfasst die Auswahl eines GUI Toolkits für plattformübergreifende Benutzeroberflächen-Entwicklung sowie Methoden zum aktualisieren von Werten, sowie Transport-Protokolle, Serialisierung von Daten und automatisches Auffinden von Diensten. Das dritte Kapitel beschäftigt sich mit den Implementierungsdetails der Software, einschließlich der entwickelten Klassen und Module. Im vierten Kapitel wird eine Validierung der erarbeitenden Ergebnisse durchgeführt. Weiters wird erklärt welche Änderungen und Erweiterungen notwendig und geplant sind, um die Funktionalität zu erweitern.
Schlagwörter: Embedded Systems, Maschinensteuerung, Graphische Benutzeroberfläche, Netzwerk
Abstract The goal of this bachelor’s thesis work is to develop the next generation remote user interface development kit for the realtime machine control software project Machinekit and to discuss the most important parts of the design and implementation. The first part gives an introduction to the Machinekit project and its main concepts. Furthermore, the current status of user interface development is described. The thesis then goes into the design details of the software. This includes the selection of a cross-platform user interface framework, a method for value updates, a transport protocol, serialization of data and the service discovery method. The third chapter addresses the implementation details of the software, including modules and classes of the resulting user interface development framework. The fourth chapter analyzes the performance and reliability of the implementation. The results are displayed graphically and analyzed. The thesis goal was reached: the user interface development framework was implemented and can be used to develop user interfaces for the Machinekit project. Furthermore, it is explained what future work needs to be done to extend the functionality.
Keywords: Embedded Systems, machine control, remote user interface, network, service discovery
Contents 1. Introduction 1.1. Machinekit . . . . . . . . . 1.2. Hardware Abstraction Layer 1.3. Current User Interfaces . . 1.4. Motivation . . . . . . . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
1 1 1 1 2
2. Design 2.1. Requirements . . . . . . . . . . 2.2. Cross-Platform User Interface . 2.3. Value Updates . . . . . . . . . 2.4. Transport . . . . . . . . . . . . 2.5. Serialization . . . . . . . . . . . 2.6. Service Discovery . . . . . . . . 2.7. Machinetalk . . . . . . . . . . . 2.8. QtQuick Virtual Control Panel
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
3 3 3 4 5 7 8 9 10
. . . . . . . .
11 11 13 14 14 15 16 17 19
4. Analysis 4.1. Test setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3. Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20 20 21 22
5. Conclusion
23
Bibliography
24
List of Abbreviations
25
List of Figures
26
List of Tables
27
3. Implementation 3.1. HalRemoteComponent 3.2. HalPin . . . . . . . . . 3.3. Service . . . . . . . . . 3.4. ServiceDiscovery . . . 3.5. ApplicationConfig . . 3.6. HalApplicationWindow 3.7. ConnectionWindow . . 3.8. HalRemote Controls .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
A. Test results
28
1. Introduction Traditionally user interfaces for Machinekit run on the same personal computer (PC) as the machine control part of the software. However, recent trends in the PC market and the appearance of new hardware platforms suggest to separate the execution platform of user interfaces and machine control software. Over the last decade desktop computers became faster, but - counter-intuitvely - less suitable for running of realtime applications, mostly due to agressive power-saving methods which impact realtime performance. This fact suggests porting Linux realtime kernel frameworks such as Xenomai or RTAI to other platforms such as the ARM platform widely used in Systems-on-a-Chip (SoC’s). However, SoC’s like the BeagleBone Black from Texas Instruments are not designed for complex graphic processing. On the other hand, mobile devices such as tablets and smartphones which are designed for complex graphics processing become more and more common. Therefore, the decision has been made that the next generation user interface for Machinekit should work remotely over the network in order to run on desktop computers as well as mobile devices.
1.1. Machinekit Machinekit [4] is a community-driven project with the goal to provide open-source software and protocols which can be used to create distributed applications to control and command moving things in realtime, and with high spatial accuracy. The roots of the Machinekit project go back to the open-source software project LinuxCNC which is mainly used to control CNC milling machines. Machinekit can also be used for the same use cases as LinuxCNC. However, it extends the application scope to arbitrary machines that need realtime interaction with hardware like robots, quad-copters and 3D-printers. Machinekit runs on a variety of different hardware platforms such as desktop x86 systems as well as ARM based SoC’s.
1.2. Hardware Abstraction Layer Machinekit has a very flexible hardware configuration layer. The scripting language called Hardware Abstraction Layer Command (HALCMD) is based on a circuit paradigm - similar to soldering electronic components together. Furthermore, Machinekit configurations usually consist of realtime and userland components as well as a Hardware Abstraction Layer (HAL) configuration to glue the different parts of the machine configuration together. While realtime components execute in a rather limited environment, userland components have the full Linux API at their disposal, enabling integration of a wide range of devices.
1.3. Current User Interfaces Machinekit comes with two major user interface creation toolkits (also known as virtual control panels (VCP’s)). All of these user interfaces currently communicate with the machine
1
control part of Machinekit using the HAL Python or C++ bindings and the Neutral Message Language (NML) C++ application programming interface (API). At the moment both interfaces use polling to update the values stored in a shared memory block.
PyVCP Python Virtual Control Panel (PyVCP) is based on the TkInter [11] widget set. A user interface is designed using a text editor to edit a Extensible Markup Language (XML) user interface description file and iteratively viewing results and improving them. PyVCP provides no support for user-defined event-handling and is purely based on HALs pin Input/Output (IO).
GladeVCP Glade Virtual Control Panel (GladeVCP) is based on the GTK+ [2] widget set. User interfaces are designed using the Glade What You See Is What You Get (WYSIWYG) user interface designer [1]. Furthermore, using GladeVCP it is possible to edit and configure the behavior of the user interface (UI) using the GTK event handling system by defining Python event handlers. Besides HAL pin IO GladeVCP supports communication with the Machinekit interpreter using the NML C++ bindings. Moreover, similar to PyVCP Glade UIs are simple XML files. Therefore, it is possible to clearly separate user interface appearance and functionality implemented in Python.
1.4. Motivation The main focus of this project is the development of an open-source cross-platform remote UI development framework for Machinekit. By creating a consistent API and a robust design, development and testing of the UI framework can be streamlined and likely made usable by a wider audience than just developers and engineers. Additionally, trough the introduction of a modern UI technologies new, end-user friendly UIs with customized and branded designs can be created which was previously impossible.
2
2. Design The goal of this project is to develop a remote user interface development framework for Machinekit. As currently Machinekit is designed as monolithic application with user interfaces executing on the same host as the realtime environment, it is necessary to survey the requirements to enable the development of remote user interfaces. However, the design details of the transition from Machinekit as a monolithic application to a component based application are out of the scope of this bachelors thesis. Therefore, only the relevant parts for the introduction of remote user interfaces will be described in this paper.
2.1. Requirements Considering the disadvantages and limitations of the current UI development framework implementations, we come to the following requirements for the new UI development framework. One of the most important requirements of the future framework is that it should be cross-platform and work on any major desktop platform (Linux, Microsoft Windows and Mac OSX) as well as the two most popular mobile platform operating systems (OS’s) Android and iOS. Furthermore, all required libraries and development tools of the UI framework should be open-source in order to comply with the General Public License (GPL) license used by Machinekit and to guarantee continued availability. Moreover, the remote interfaces should rely only on widely used and stable libraries and technologies to avoid the risk of obsolescence. Given the above requirements we can split the design of the project into five parts: CrossPlatform User Interface, Value Updates, Transport, Serialization and Service Discovery.
2.2. Cross-Platform User Interface Ideally, Machinekit user interfaces should run on different platforms with minimal or no porting overhead. Moreover, the framework used should be suitable for desktop, mobile and embedded UI development in order to satisfy all possible platform requirements. Furthermore, another criteria to select the user interface software development framework is that UIs should be loadable at run-time rather than compiled into the source code of the client application.
Qt Qt [10] is C++ class-library for platform independent development of graphical user interfaces (GUI). Besides the main usage as GUI framework Qt also provides cross-platform libraries for internationalization, databases, multi-threading, networking and more. The main contributor and company behind Qt is currently Digia [?]. However, the Qt project has also many contributors from the open-source community especially from the K Desktop Environment (KDE) project. Besides the official C++ binding from third parties also bindings for
3
Python, Ruby, Go and many other programming languages are available. Since version 4.8 Qt is triple licensed with a proprietary license, the GNU GPLv3 open-source license and the GNU Lesser General Public License (LGPL)v2.1 open-source license.
QtQuick and QML Qt comes with its own JavaScript based scripting language Qt Meta Language (QML) which is part of the software application framework QtQuick for rapid user interface and application development. QML is a declarative interface specification and programming language. QtQuick is designed with the requirements for mobile and embedded devices in mind providing a framework for highly dynamic and customized user interfaces. QML is a interpreted scripting language. Therefore, the QML interpreter is part of the QtQml module and can be included into any Qt application. This makes it possible to load QML applications at run-time either from the applications resources or from the file system. QML can be extended with new modules developed in C++ using the Qt framework. This make it possible to create custom QML modules based on any C or C++ library.
2.3. Value Updates Value Updates in the user interface should happen fast and efficient. However, there is a tradeoff between runtime overhead and UI responsiveness. To propagate value changes essentially two patterns are available: Polling scalars in a shared memory block, and eventdriven messaging.
Shared Memory Polling Polling describes the process of cyclically checking a signal source for value changes. In the case of the current user interfaces for Machinekit the process of polling involves checking a shared memory block (see figure 2.1). The advantage of polling is that it easy to implement and predictable. However, due to the fact that the data is checked in a specific interval value changes which happen within the checking interval may be lost. UI Polling Shared Memory
Polling UI Polling
HAL
UI
Figure 2.1.: Polling-based messaging Furthermore, polling user interfaces does not scale well in a networked setup, and the the overhead of polling the data increases with every additional signal source, and UI instance. This is especially important if UIs are built in a modular fashion and parts of the UI act as an individual instance.
4
Event-Driven Messaging The idea behind event-driven messaging is that a service consumer subscribes to the values it is interested in rather than checking all available data. The service provider informs the consumer in case the data changes. This messaging pattern is very efficient and reliable. Furthermore, it is suitable in all cases where the messaging overhead is acceptable compared to polling the data. For this project a compromise between polling and even-driven messaging has been made. Shared Memory
Polling Haltalk
Events
Events Events UI
HAL
UI UI
Figure 2.2.: Event-based messaging Depicted in figure 2.2 is that a single part of the software polls the shared memory block for data changes and informs all clients in case a value has changed using event-driven messaging. This results into a reasonable tradeoff between scalability as well as data overhead due to messages generated by events.
2.4. Transport The task of the transport layer is to provide a reliable end-to-end connection between network nodes. The transport layer can be thought of as a pipe between two end-points transporting binary data (see figure 2.5). For selecting a suitable transport layer several parameters have to be taken in consideration: Network performance, development effort, dissemination in other systems, setup requirements and management overhead.
Centralized Centralized messaging designs include a central broker that all node communicate through as can be seen in figure 2.3. The advantages of a centralized messaging design are certain reliability guarantees such as making sure that a node receives a message. The biggest disadvantages of a centralized messaging design are the high development costs and that the message broker itself is a single point of failure.
Decentralized Decentralized messaging designs include nodes communicating directly with each other as can be seen in figure 2.3. This peer to peer concept normally provides higher performance and lower latency than centralized messaging designs, because nodes can directly communicate
5
Node
Node
Node
Node
Message Broker
Node
Node
Node
Node
Figure 2.3.: Centralized (left) and decentralized (right) messaging with each other. Another advantage of this design is that nodes can join and leave the network at any time since there is no central server that needs to be available.
ZeroMQ ZeroMQ [13] is an asynchronous low-latency messaging library developed by iMatix. Altough ZeromMQ looks like a networking library its doubles as a concurrency framework. The library provides sockets with different transports like in-process, inter-process, Transmission Control Protocol (TCP) and multicast. ZeroMQ provides a way to connect sockets with N-to-N patterns like fan-out, pub-sub, task distribution and request-reply. The two patterns relevant for the use in Machinekit are router-dealer and publish-subscribe (see figure 2.4).
REQ
REQ
REQ
ROUTER PUB DEALER
REP
REP
REP
SUB
SUB
SUB
Figure 2.4.: Router-Dealer (left) and Publish-Subcribe (right) ZeroMQ messaging patterns ZeroMQ is widely used in different areas from various companies for both communication
6
inside applications and communication over networks. ZeroMQ is licensed under the GNU LGPLv3 open-source license.
nzmqt nzmqt [6] is a lightweight C++ Qt binding for ZeroMQ. Altough the ZeroMQ C++ bindings could be used when developing applications using Qt and C++, nzmqt provides an interface that integrates better with the development interfaces of Qt. Furthermore, nzmqt provides integration with Qt’s event loop and signals and slots. nzmqt is licensed under the Berkeley Software Distribution (BSD) open-source license.
2.5. Serialization Serialization is the process of converting data into an architecture-independent wire format.
Object
Bytes Serialization
0
0
1
1
0
1
0
0
1
1
Transport
Figure 2.5.: Serialization of objects The figure 2.5 depicts the serialization process. The output of the serialization process is a stream of binary data. The input depends on the serialization layer used. Serialization methods are usually highly optimized for certain purposes. Therefore, a choice needs to be made.
Protocol Buffers Protocol Buffers (protobuf) [7] is a method, set of tools and libraries to serialize structured data. Protobuf was developed by Google [?] for internal use in their products. Similar in goal to XML protobuf provides a means to exchange data structures between applications, however the implementation was designed to be faster and smaller compared to XML. protobuf involves a interface description language and a software package that generates source code for various programming languages. Per default the programming languages C++, Python and Java are supported by the protobuf compiler. protobuf is not only used for communication between applications, but also inside applications as alternative to C++ classes and structs to store and interchange data. protobuf is licensed under the BSD open-source license.
7
2.6. Service Discovery Service discovery subsumes methods for applications to find services offered by other applications on a computer network. Service discovery includes a common language for all participants to detect service providers without the need of user interaction. Service discovery methods can be divided into centralized and decentralized patterns.
Centralized A centralized service discovery is the easiest to implement. All clients connect to a single host to obtain information about available services (see figure 2.6). Service providers inform the service discovery about available services.
Decentralized Decentralized service discovery implies that every service provider announces its own services (see figure 2.6). Client
Service Service
Service Discovery
Service
Client Service Discovery
Service Service Service Discovery
Service
Client
Client Service Discovery
Figure 2.6.: Centralized (left) and decentralized (right) service discovery. Clients may search for all services required. The service discovery nodes are not coordinated. All information about the services is directly published by each service provider. This pattern has the advantage of no single point of failure relative to centralized service methods . Furthermore, it is easier to add new services since other parties are not involved.
DNS Service Discovery DNS Service Discovery (DNS-SD) is a service discovery method based on the standard Domain Name System (DNS) protocol to discover services on the network. In combination with Multicast DNS (mDNS) it builds the basis for Zero-configuration networking (zeroconf). However, DNS-SD is not dependent on mDNS and works also with standard unicast DNS. Two widely available implementations are Apples Bonjour and Avahi for Linux.
8
JDNS JDNS [3] is a simple DNS implementation capable of performing normal unicast DNS queries as well as mDNS queries. It also supports advertising of mDNS records. The origin of JDNS is the Delta Extensible Messaging and Presence Protocol (XMPP) project [12] (instant messaging). JDNS is a pure C implementation and has no particular external dependencies. Furthermore, it provides a Qt wrapper. The project is licensed under the Massachusetts Institute of Technology (MIT) open-source license.
2.7. Machinetalk The legacy middleware framework NML is unsuitable as a basis for a remote API into Machinekit for a variety of reasons. Therefore, the decision was made to swap NML for a combination of the above mentioned libraries and tools - ZeroMQ, protobuf, DNS-SD and lock-free queues. This middleware stack is called Machinetalk, and over time all NMLusing parts of Machinekit will be ported to use Machinetalk. The goal was to provide a end-to-end-capable communications stack which retains message encoding and structure all the way from UIs and even JavaScript Object Notation (JSON)/JavaScript to realtime components, without the need to transcode messages due to media breaks (for instance, NML - being defined in C++ - is not usable in a kernel RT threads setup, hence messages need to be transcoded to from C++ instance objects to C structs and vice versa at the realtime boundary). Machinetalk also adds the capability for message exchange between realtime (RT) components, RT and userland processes, and arbitrary userland-to-userland processes through a common API. Along these lines ZeroMQ and protobuf act as an remote-capable API to Machinekit. Key parts of Machinetalk are HAL remote components, HAL groups and HAL named ringbuffers. In this thesis only the HAL remote components are used, HAL groups and HAL named ringbuffers are currently not implemented on the UI side. [5]
HAL Remote Components Machinekit UIs like PyVCP or GladeVCP are implemented as HAL components. UI controls such as buttons and sliders are represented by HAL pins inside the HAL realtime environment. However, the current implementation of HAL components uses shared memory polling. As discussed in section 2.3 this design pattern is not suitable for networked setups. For this purpose the HAL remote components where introduced with Machinetalk. HAL remote components behave similar to HAL components inside the HAL realtime environment, but they have no thread function. Therefore, HAL remote components do not do anything by themselves and need a userland process to adopt and serve them. For remote UI this userland process is called Haltalk. The Haltalk server provides two ZeroMQ sockets for clients to communicate with the remote components. This sockets are announced using DNS-SD discussed in section 2.6. Haltalk cyclically refreshes the state of a components and publishes changes via the halrcomp service; but other than the previous scheme, HAL objects are scanned only once per Haltalk instance, not once per UI instance. The second service halrcmd can be used by remote clients to upate values of remote component.
9
2.8. QtQuick Virtual Control Panel The resulting user interface development framework based on QtQuick is called QtQuick Virtual Control Panel (QtQuickVcp). QtQuickVcp is designed with remote interfaces on mobile devices and embedded systems in mind. A UI is designed once for a specific form factor and can be deployed to any mobile, embedded or desktop device over the network. Communication with the remote Machinekit instance happens over the halcmd and halrcomp services provided by Haltalk. The UI is implemented as HAL remote component with UI elements acting as pins of the HAL remote components.
10
3. Implementation This chapter provides design and implementation details of QtQuickVcp software modules. All modules are implemented in C++ and designed to be used inside the declarative QML environment. Consequently, the C++ modules are built into several shared objects for use by the QML interpreter. Such shared objects designed specifically for use in QML are called QML plugins or QtQuick modules. At the moment QtQuickVcp consist of five QtQuick modules: • Machinekit.Controls - Generic UI items not associated with HAL pins which can be also used in non HAL specific applications • Machinekit.HalRemote - Non graphical user interface (GUI) HAL remote pins and components • Machinekit.HalRemote.Controls - UI items such as Slider and Button combined with HAL pins • Machinekit.PathView - GCode path and progress view • Machinekit.VideoView - Video view, for example Motion JPEG (MJPEG) webcam streams The remainder of this chapter will focus on the implementation Machinekit.HalRemote and Machinekit.HalRemote.Controls modules.
of
the
3.1. HalRemoteComponent The HalRemoteComponent is the UI counterpart of a remote component on the HAL side. It works as actor for representation and manipulation of the remote component running inside the HAL realtime environment. For each remote component in HAL zero or more actors may exist. Client
Realtime host Realtime environment
HalRemoteComponent HAL instance Haltalk halrcomp HalRemoteComponent
Remote component
halrcmd
Figure 3.1.: Relation between UI and HAL remote components
11
Figure 3.1 depicts how a HalRemoteComponent is related to a remote component in HAL. The HalRemoteComponent connects to a remote HAL instance using the halrcmd and halrcomp services provided by a Haltalk server running on the realtime host. The halrcomp service is implemented as ZeroMQ publish-subscribe pattern. If an update of a remote component happens all subscribers to a specific topic are informed. The halrcmd service is implemented in a router-dealer pattern (see figure 2.4). This ZeroMQ socket is used for updating data from the client side as well as heartbeat and other client-specific messages.
Disconnected
ready=false ready=true
ready=false
bind rejected socket error
Connecting bind accepted
Error ping acknowledge Connected pin change rejected ping timeout or other error
Figure 3.2.: HalRemoteComponent state-machine. Figure 3.2 depicts the state-machine of a component. The component starts in the Disconnected state. When the ready property is set to true both ZeroMQ sockets attempt to connect and the state transitions to the Connecting state. Figure 3.3 depicts the communication of both services with the Haltalk instance. Then the component recursively scans its containerItem for HalPins and sends a bind message to the halrcmd socket. In case the connection attempt fails or the bind request is rejected the state of the component transitions to the Error state. If the bind request is accepted the state transitions to the Connected state. Once the component is in the Connected state it receives a full update of the status of all pins. All further pin status updates happen incrementally relative to the current status of the remote component. Furthermore, the component cyclically sends a heartbeat message to the remote host to check the connection status. If one heartbeat message is not answered the state of the component transitions to the Error state. In this case it is possible that the component is able to recover from the Error state if the answer to the heartbeat messages is received after the timeout period. In all other cases it is necessary to change the ready property of the component to false in order to reset the error.
12
Client 1
Haltalk
Client 2
bind remote component Client 1 connects bind confirm full update
bind remote component Client 2 connects bind confirm full update
pin change Client 1 pin change incremental update (echo)
incremental update
ping Client 1 heartbeat ping acknowledge
Figure 3.3.: Haltalk communicating with two clients over halrcmd (blue) and halrcomp (red).
3.2. HalPin The HalPin is the counterpart of a pin of a HAL remote component. The HalPin component works in combination with the HalRemoteComponent. The value of a HalPin can be connected to a UI component such as a button using QML bindings. If the value changes in the HAL component, this is reflected in the local user interface, and vice versa.
not synced
update from remote component
update from UI
synced
Figure 3.4.: A state diagram for the synced property of the HalPin component. Figure 3.4 shows the state-diagram for the synced property of the component. The synced property is updated every time the value of the pin changes. In case the value was changed from the QML code the synced property is set to false. This indicates that a pin change syncing the local value with the remote components value is in process. When the echo from the remote component is received by the HalRemoteComponent it updates the value of the HalPin again. In this case the synced property of the component is set back to true.
13
3.3. Service The Service component represents a Machinekit service. In combination with the ServiceDiscovery component it can be used to automatically resolve services running on a remote host. The Service has to be in a ServiceList controlled by a ServiceDiscovery component in order to receive updates.
SD items >= 1
ready = false
ready = true
SD items >= 1
SD item = 0
Figure 3.5.: A state machine for the Service component. Figure 3.5 shows the stat-machine for the ready property of the component. The component is ready as long as the items property has one or more entries. When a Service component is used to represent a single service running on the remote host the items property usually should have only one entry. In case more than one service of a single type exists the services need to be filtered. For that purpose a ServiceDiscoveryFilter can be used.
3.4. ServiceDiscovery The ServiceDiscovery component can be used to discover and resolve available services on the network. The component uses mDNS and DNS-SD to discover published services. Depicted in figure 3.6 is how the discovery of services for different service providers works. Every client has a DNS-SD browser to discover published DNS-SD services and every service provider has its own DNS-SD publisher. Therefore, the service discovery is decentralized and has no single point of failure. mDNS names are generated using the serviceType and domain properties of the component as well as the type property of the Service. This results in the following scheme: _._sub._._tcp.. Therefore, a Service type acts as a mDNS subtype whereas the serviceType of the ServiceDiscovery component acts as mDNS service type. The domain for multicast queries is usually ’local’. As DNS-SD is not limited to mDNS the domain name could be easily changed when operating in unicast mode. However, at this moment unicast is not supported by the ServiceDiscovery component. The ServiceDiscovery component looks up services as requested by the user interface. For this purpose Service components need to be registered with ServiceDiscovery grouped into ServiceLists. When the updateServices function is called, the list of services is scanned by the ServiceDiscovery component and for each new Service a new DNS PTR lookup is created. Existing DNS PTR lookups for all removed services are closed. When the PTR lookup is answered by a node a new discovery entry is created by the component. The response is then followed by two additional DNS lookups for SRV and TXT types. When
14
config Service
DNS-SD Publisher
DNS-SD Browser
Client
Config-Server
halrcomp Service
DNS-SD Publisher
Client
DNS-SD Browser
Haltalk
halrcmd Service
DNS-SD Publisher
MJPGStreamer
video Service
Figure 3.6.: Service discovery of different services. both lookups complete the corresponding Services are updated with the data from the previous DNS lookups. The data relevant for a Service are the DNS name provided by the SRV lookup and the TXT records provided by the TXT lookup. The TXT records are then parsed and used to fill the properties of the Service component.
3.5. ApplicationConfig The ApplicationConfig component acts as a client for a config-server providing application configurations. This component has two main purposes, gathering and listing available configurations on a remote host and downloading QML files from a remote host. The ApplicationConfig component uses the config service by connecting to the remote configserver with the dealer-router socket pattern (see figure 2.4). The application configuration protocol supports two basic messages, one for listing all available configurations and one for downloading details of a specific configuration. When the ready property of the component transitions from false to true, the component starts to connect the ZeroMQ socket and sends a list applications query to the host. When the query is answered the ApplicationConfig component updates the configs property with the results. The list of available configurations provided by the configs property can be used to display a configuration selection. To select a specific configuration and to download
15
application configuration details as well as the QML files of the remote application the function selectInstance must be called with the name of the configuration. When the request is answered by the config-server the new data is accessible using the selectedConfig property of the ApplicationConfig component. With this information it is possible to execute the QML files downloaded from the remote host. Usually QML files downloaded from the config-server are placed into the temporary files directory of the OS. If more than one QML file is provided by the configuration the main file is detected using three rules. Firstly, it is checked whether a QML file with the name of the configuration exist. Then, the files are checked for a file with the name main.qml. If still no main file was found the first file of the list of downloaded files is selected.
3.6. HalApplicationWindow The HalApplicationWindow is a component provided as convenience for creating QtQuickVcp user interfaces. The component combines a HalRemoteComponent as well as the necessary halrcomp and halrcmd Services within a status and error information page. However, the HalApplicationWindow is not designed to work as standalone component. It is necessary to provide a source for the two services. This can be done by either combining it with a ServiceDiscovery component or by specifying the service Uniform Resource Identifiers (URI’s) manually. HalApplicationWindow ContainterItem
HalButton
HalButton
HalPin
HalPin
"button1" Bit Out
"button2" Bit Out
HalGauge
HalProgressBar
HalPin
HalPin
HalPin
"gauge" Float In
"bar" Float In
"bar.scale" Float In
HalRemoteComponent
Service
Service
"control"
halrcmd
halrcomp
Figure 3.7.: QtQuickVcp QML component structure Depicted in figure 3.7 is the structure of a typical user interface built upon HalApplicationWindow. The main principle behind the HalApplicationWindow is that
16
a user interface is presented inside the HAL realtime environment as a single remote component. All user interface components available within the user interface have HalPins assigned to their value. Therefore, a button in the user interface is represented by a pin of the remote component. For this to work the HalRemoteComponent provided within the HalApplicationWindow scans the containerItem, in this case the whole HalApplicationWindow, for HalPins. This makes it possible to place any component of the HalRemote.Controls module within the HalApplicationWindow and it will be represented by a pin in the remote component. Besides making it easier to build new user interfaces the HalApplicationWindow also features a error and status page for displaying information about the status of the halrcmd and halrcomp services as well as error information from the HalRemoteComponent. When the services are not ready a indicator is visible in order to inform the user about the application status. In case an error happens the window displays the type of the error as well as the error information provided by the HalRemoteComponent.
3.7. ConnectionWindow The ConnectionWindow component provides an all in one container item to load and display QtQuickVcp user interfaces. It combines service discovery as well as instance selection for local as well as remote deployed HalApplicationWindows. Furthermore, the ConnectionWindow component provides pages for selecting Machinekit instances, application configurations and displaying the network connectivity status.
network closed Network
network opened
Running
Instance back
back
application selected Application
instance selected
Figure 3.8.: State-diagram of the ConnectionWindow Figure 3.8 displays the state-diagram for the ConnetionWindow component. In case the network connection is closed the state of the ConnectionWindow transitions to the Network state displaying the network page. When the network connection is available the state transitions to the Instance state showing the instance selection page. Then the user needs to select a specific Machinekit instance and the state of the ConnectionWindow transitions to the Application state. In the Application state all available configurations will be
17
displayed. When the user selects a specific configuration the state transitions to the Running state displaying the actual application. If a user presses the back button the state transitions to previous state independent of the current state.
uuid filter
InstanceSelection
Service ready
HalApplicationWindow
config
Service halrcmd
ApplicationConfig Service ServiceDiscovery halrcomp select app load QML files connect Services
Service video
ApplicationSelection
Figure 3.9.: Remote application selection Depicted in figure 3.9 is a representation of how the components contained by a ConnectionWindow interact with each other in case the window works in the remote deployment mode. The selection of a specific Machinekit instance is implemented using the items property of the config Service. When an instance is selected a uuid filter is applied to the ServiceDiscovery component. The uuid filter results in a single entry in the items property of the config Service. Then the ApplicationConfig component is activated with the URI provided by the config Service. Now the application configuration is presented to the user. Once a configuration is selected first the QML files are downloaded from the remote host and then loaded inside the QML environment using a Loader component. Next, the services of the loaded HalApplicationWindow are now connected to the ServiceDiscovery component. When the halrcomp and halrcmd Services of the HalApplicationWindow become ready the user interface eventually becomes visible. Depicted in figure 3.10 is a representation of how the components contained by the a ConnectionWindow interact with each other in case the window works in local mode. The process works similar to the remote deployment process. However, the application configuration selection is skipped because the HalApplicationWindow is already available locally. It is also possible to configure the ConnectionWindow to automatically select a specific instance using a filter component. This use case is useful for development purposes in combination with local mode .
18
show
Service
HalApplicationWindow
InstanceSelection config
Service halrcmd
uuid filter
Service ServiceDiscovery halrcomp
Service video
Figure 3.10.: Local application selection
3.8. HalRemote Controls Within QtQuickVcp a few user interface components combined with HalPins are provided. This components are designed for rapid Machinekit user interface development and can be used as starting point to develop useful user interfaces. The UI components provided within the HalRemote Controls module are: • • • • • • • • • • • • • •
HalButton - A push button with a text label. HalCheckBox - A check box with a text label. HalColorPicker - Provides a beautiful color picker. HalComboBox - Provide a drop-down list functionality. HalContainer - A container item to enable and disable children. HalGauge - Provides a linear gauge control to display data. HalLabel - A text label. HalLed - Provides a LED-like control to display binary values. HalLogChart - A chart for logging data. HalProgressBar - A progress indicator. HalRadioButton - A radio-button with a text label. HalSlider - Provides a vertical or horizontal slider control. HalSpinBox - A spin box control. HalVirtualJoystick - A virtual joystick control.
However, machine control UIs often require custom UI controls. With QtQuickVcp this is possible by adding a HalPin to a component and binding the value of the custom control to the value of the HalPin. This makes it possible to extend the range of available controls without the need to modify C++ code.
19
4. Analysis This chapter analyzes the performance of the remote user interfaces in terms of network performance and response time as well as reliability. For this purpose two test setups where created. The first one tests the echo time of a remote component and the second one the total round-trip time between two connected remote components. Both test setups where tested under real-life conditions with a Ethernet network and a wireless network setup.
4.1. Test setup Depicted in figure 4.1 is the setup for the first experiment. A simple application with just one HalRemoteComponent and one HalPin is created. When the HalPin is connected to the HalRemoteComponent a pin change is performed and the synced property of the HalPin changes to false. Furthermore, the signal travels from the HalPin over the HalRemoteComponent to the Haltalk instance running on the remote realtime host. After the pin change has been processed the Haltalk instance sends a pin update to all connected HalRemoteComponents. In this case only one HalRemoteComponent is connected and updated. The message is then processed by the HalRemoteComponent and results in an update of the HalPin value. This update sets the synced property of the HalPin back to true. The time between the change of the synced property to false and the change of the synced property back to true is measured and used for the performance statistics.
pin change
Pin
pin update
measure time
Remote Component
Haltalk
Figure 4.1.: Test setup 1. Figure 4.2 shows the setup for the second experiment. For this setup an application with two individual HalRemoteComponents was created. Both components are connected to a single HalPin. The HalPin of the first component acts as input pin and the HalPin of the second component acts as an output pin. The pins of both remote components are connected inside the HAL realtime environment on the Machinekit host. When the value of the first HalPin is changed the pin change event is processed by the HalRemoteComponent
20
and sent to the Haltalk instance. The Haltalk instance sends the pin change event to the HAL realtime environment where the pin change event is processed. The pin change then results in an update of the connected pin of the second remote component. This update is then detected by the Haltalk instance and forwarded as pin update message to all connected HalRemoteComponents. The pin update event results in an update of the value of the second HalPin. The time between the change of the value of the first HalPin and the update of the second HalPin is measured and used for the performance statistics. measure time pin change
Pin
Pin
Remote Component
Remote Component
pin update
Haltalk
Figure 4.2.: Test setup 2. For both setups the value of the triggering HalPin was toggled 1000 times. Furthermore, the captured data was processed to derive statistics.
4.2. Results Both tests where performed with different Haltalk remote component timer intervals. Haltalk polls every remote component in a specific interval in order to detect changes and sends them to all connected remote instances. Therefore, this parameter is especially relevant for performance measurements. Table 4.1 shows the results of the first test setup and table 4.2 shows the results of the second test setup. A graphical representation of the test results can be found in Appendix A.
Type Ethernet Ethernet WLAN Ehernet WLAN Ethernet WLAN
Timer 5 10 10 20 20 100 100
Mean 9.048 11.734 20.123 18.085 17.571 53.984 63.625
Table 4.1.: Results of Test 1 StdDev StdErr Variance 3.405 0.108 11.591 4.265 0.135 18.187 40.747 1.289 1660.312 10.274 0.325 105.547 30.485 0.964 929.360 28.230 0.893 796.927 89.543 2.832 8017.962
21
Max 19 29 511 116 556 148 1927
Min 1 2 2 1 3 4 4
Median 9 11 14 18 14 52.5 57
Type Ethernet Ethernet WLAN Ethernet WLAN Ethernet WLAN
Timer 5 10 10 20 20 100 100
Mean 9.274 12.018 17.273 17.054 21.397 57.436 66.219
Table 4.2.: Results of Test 2 StdDev StdErr Variance 3.249 0.103 10.555 5.034 0.159 25.337 16.836 0.532 283.446 6.990 0.221 48.854 14.939 0.472 223.175 29.735 0.940 884.188 60.130 1.901 3615.569
Max 18 88 184 34 152 112 896
Min 1 2 3 2 3 3 5
Median 9 11 13 17 19 58 62
4.3. Interpretation The results of the performance tests show that the system performs very well on both Ethernet and Wireless Local Area Network (WLAN) networks. However, the standard deviation of the test results from the wireless measurement is higher than the standard deviation of the test results from the Ethernet measurement. If we analyze the scatter diagrams (see Appendix A) we can see that the wireless connection slows down from time to time. This can be tracked down to signal variation and interference during the measurement. Therefore, for controls where high reliability and small jitter is important a non-wireless network connection should be used. For most end-user devices such as 3D printers and self-built robots the wireless interface is reliable and fast enough.
22
5. Conclusion The new remote user interfaces provide a significant improvement in the capabilities of Machinekit as rapid development platform as well as production-ready machine control platform. Creating user interfaces independent of the machine control platform avoids the limitations of embedded platforms in terms of CPU speed and graphical performance. Furthermore, the software is designed in a way that it can be easily extended with new modules and functionality. This is especially relevant for incremental development of rapidly innovating technologies such as 3D printers and robots. Moreover, QtQuickVcp provides a set of predefined components and configurations for rapid machine control user interface development.
Future work Future work on QtQuickVcp will include the development of a GCode path and progress display. Moreover, the interface to communicate with the Machinekit interpreter needs to be defined and implemented in QtQuickVcp in order to enable remote user interface development for GCode controlled machines. Furthermore, the default client application should come with a set of useful tools such as a HAL scope and meter to further improve the rapid development capabilities of Machinekit.
23
Bibliography [1]
The Glade project (2014), Glade: A User Interface Designer https://glade.gnome.org/, 10 June 2014
[2]
The GTK+ Team (2007-2012), The GTK+ Project http://www.gtk.org/, 10 June 2014
[3]
Delta XMPP Project (2014), JDNS http://delta.affinix.com/jdns/, 10 June 2014
[4]
The Machinekit Project (2014), Machinekit in a nutshell http://www.machinekit.io/, 10 June 2014
[5]
Michal Haberler (2014), Machinetalk functional overview https://github.com/mhaberler/asciidoc-sandbox/wiki/ Machinetalk-functional-overview, 10 June 2014
[6]
Jonny Dee (2014), nzmqt - A lightweight C++ Qt binding for 0MQ https://github.com/jonnydee/nzmqt, 10 June 2014
[7]
Google Developers (2012), Developer Guide - Protocol Buffers https://developers.google.com/protocol-buffers/docs/overview, 10 June 2014
[8]
Digia Plc (2013), Qbs Manual Version 1.2.0 http://qt-project.org/doc/qbs-1.2/index.html, 10 June 2014
[9]
Digia Plc (2014), Qt Creator http://qt-project.org/wiki/category:tools::qtcreator, 10 June 2014
[10]
Digia Plc (2014), Qt Project http://qt-project.org/, 10 June 2014
[11]
John W. Shipman (2013), Tkinter 8.5 reference: A GUI for Python http://infohost.nmt.edu/tcc/help/pubs/tkinter/tkinter.pdf, 10 June 2014
[12]
Delta XMPP Project (2014), Delta XMPP Project http://delta.affinix.com/, 10 June 2014
[13]
iMatix Corporation (2014) ZeroMQ - The Guide http://zguide.zeromq.org/page:all, 10 June 2014
24
List of Abbreviations API Application Programming Interface
PyVCP Python Virtual Control Panel
BSD Berkeley Software Distribution
Qbs Qt Build Suite
DNS Domain Name System
QML Qt Meta Language
DNS-SD DNS Service Discovery
QtQuickVcp QtQuick Virtual Control Panel
GladeVCP Glade Virtual Control Panel GPL General Public License GUI Graphical User Interface HALCMD Hardware Abstraction Layer Command
RT Realtime SDK Software Development Kit SoC System-on-a-Chip SSH Secure Shell
HAL Hardware Abstraction Layer
TCP Transmission Control Protocol
IDE Integrated Development Environment
UI User Interface
IO Input/Output
URI Uniform Resource Identifier
JSON JavaScript Object Notation
USB Universal Serial Bus
KDE K Desktop Environment LGPL Lesser General Public License mDNS Multicast DNS MIT Massachusetts Institute of Technology
UVC USB Video Class VCP Virtual Control Panel WLAN Wireless Local Area Network
MJPEG Motion JPEG
WYSIWYG What You See Is What You Get
NML Neutral Message Language
XML Extensible Markup Language
OS Operating System PC Personal Computer
XMPP Extensible Messaging and Presence Protocol
protobuf Protocol Buffers
zeroconf Zero-configuration networking
25
List of Figures 2.1. 2.2. 2.3. 2.4. 2.5. 2.6.
Polling-based messaging . . . . . . . . . . . . . . . . . . . . . . . . . Event-based messaging . . . . . . . . . . . . . . . . . . . . . . . . . Centralized (left) and decentralized (right) messaging . . . . . . . . . Router-Dealer (left) and Publish-Subcribe (right) ZeroMQ messaging Serialization of objects . . . . . . . . . . . . . . . . . . . . . . . . . . Centralized (left) and decentralized (right) service discovery. . . . . .
. . . . . . . . . . . . . . . patterns . . . . . . . . . .
4 5 6 6 7 8
3.1. Relation between UI and HAL remote components . . . . . . . . . . . . . . . 3.2. HalRemoteComponent state-machine. . . . . . . . . . . . . . . . . . . . . . . . 3.3. Haltalk communicating with two clients over halrcmd (blue) and halrcomp (red). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4. A state diagram for the synced property of the HalPin component. . . . . . 3.5. A state machine for the Service component. . . . . . . . . . . . . . . . . . . 3.6. Service discovery of different services. . . . . . . . . . . . . . . . . . . . . . . 3.7. QtQuickVcp QML component structure . . . . . . . . . . . . . . . . . . . . . 3.8. State-diagram of the ConnectionWindow . . . . . . . . . . . . . . . . . . . . . 3.9. Remote application selection . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.10. Local application selection . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11 12 13 13 14 15 16 17 18 19
4.1. Test setup 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2. Test setup 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20 21
A.1. Test A.2. Test A.3. Test A.4. Test A.5. Test A.6. Test A.7. Test A.8. Test A.9. Test A.10.Test A.11.Test A.12.Test A.13.Test
28 28 28 29 29 29 29 30 30 30 30 31 31
1: 1: 1: 1: 1: 1: 1: 2: 2: 2: 2: 2: 2:
Ethernet 5ms timer interval . . Ethernet 10ms timer interval . WLAN 10ms timer interval . . Ethernet 20ms timer interval . WLAN 20ms timer interval . . Ethernet 100ms timer interval WLAN 100ms timer interval . Ethernet 10ms timer interval . WLAN 10ms timer interval . . Ethernet 20ms timer interval . WLAN 20ms timer interval . . Ethernet 100ms timer interval WLAN 100ms timer interval .
26
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
List of Tables 4.1. Results of Test 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2. Results of Test 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
21 22
A. Test results 1.2
50
1
probability []
time [ms]
40
30
20
0.8 0.6 0.4
10
0.2
0
0 0
200
400 600 measurement []
800
1,000
0
(a) Scatter diagram
10
20
30 time [ms]
40
50
40
50
(b) Gauss function
Figure A.1.: Test 1: Ethernet 5ms timer interval 1.2
50
1
probability []
time [ms]
40
30
20
0.8 0.6 0.4
10
0.2
0
0 0
200
400 600 measurement []
800
1,000
0
(a) Scatter diagram
10
20
30 time [ms]
(b) Gauss function
Figure A.2.: Test 1: Ethernet 10ms timer interval 1.2
150
1
time [ms]
probability []
100
0.8 0.6 0.4
50
0.2 0
0 0
200
400 600 measurement []
800
1,000
0
50
100 time [ms]
(a) Scatter diagram
(b) Gauss function
Figure A.3.: Test 1: WLAN 10ms timer interval
28
150
1.2
100
1
80
0.8
probability []
time [ms]
120
60
0.6
40
0.4
20
0.2
0
0 0
200
400 600 measurement []
800
1,000
0
(a) Scatter diagram
10
20
30 time [ms]
40
50
(b) Gauss function
Figure A.4.: Test 1: Ethernet 20ms timer interval 1.2
150
1
time [ms]
probability []
100
0.8 0.6 0.4
50
0.2 0
0 0
200
400 600 measurement []
800
1,000
0
50
100
150
time [ms]
(a) Scatter diagram
(b) Gauss function
Figure A.5.: Test 1: WLAN 20ms timer interval 1.2
200
1
probability []
time [ms]
150
100
0.8 0.6 0.4
50 0.2 0
0 0
200
400 600 measurement []
800
1,000
0
(a) Scatter diagram
50
100 time [ms]
150
200
(b) Gauss function
Figure A.6.: Test 1: Ethernet 100ms timer interval 1.2
200
1
probability []
time [ms]
150
100
0.8 0.6 0.4
50 0.2 0
0 0
200
400 600 measurement []
800
1,000
0
(a) Scatter diagram
50
100 time [ms]
150
(b) Gauss function
Figure A.7.: Test 1: WLAN 100ms timer interval
29
200
1.2
100
1
probability []
time [ms]
80
60
40
0.8 0.6 0.4
20
0.2
0
0 0
200
400 600 measurement []
800
1,000
0
(a) Scatter diagram
10
20
30 time [ms]
40
50
(b) Gauss function
Figure A.8.: Test 2: Ethernet 10ms timer interval 1.2
150
1
time [ms]
probability []
100
0.8 0.6 0.4
50
0.2 0
0 0
200
400 600 measurement []
800
1,000
0
50
100
150
time [ms]
(a) Scatter diagram
(b) Gauss function
Figure A.9.: Test 2: WLAN 10ms timer interval 1.2
50
1
probability []
time [ms]
40
30
20
0.8 0.6 0.4
10
0.2
0
0 0
200
400 600 measurement []
800
1,000
0
(a) Scatter diagram
10
20
30 time [ms]
40
50
(b) Gauss function
Figure A.10.: Test 2: Ethernet 20ms timer interval 1.2
150
1
time [ms]
probability []
100
0.8 0.6 0.4
50
0.2 0
0 0
200
400 600 measurement []
800
1,000
0
50
100 time [ms]
(a) Scatter diagram
(b) Gauss function
Figure A.11.: Test 2: WLAN 20ms timer interval
30
150
1.2
200
1
probability []
time [ms]
150
100
0.8 0.6 0.4
50 0.2 0
0 0
200
400 600 measurement []
800
1,000
0
(a) Scatter diagram
50
100 time [ms]
150
200
(b) Gauss function
Figure A.12.: Test 2: Ethernet 100ms timer interval
1.2
200
1
probability []
time [ms]
150
100
0.8 0.6 0.4
50 0.2 0
0 0
200
400 600 measurement []
800
1,000
0
(a) Scatter diagram
50
100
150 time [ms]
200
(b) Gauss function
Figure A.13.: Test 2: WLAN 100ms timer interval
31
250
300
