AIAA 2014-1753 SpaceOps Conferences 5-9 May 2014, Pasadena, CA SpaceOps 2014 Conference

Applying Virtualization Technology to Earth Station Systems Hiroshi Uegaki1, Kyohei Murakami2, and Satoshi Harauchi3

Downloaded by 37.44.207.190 on January 14, 2017 | http://arc.aiaa.org | DOI: 10.2514/6.2014-1753

Mitsubishi Electric Corporation, 8-1-1, Amagasaki City, Hyogo, 661-8661, Japan Melco's earth station systems are operating on virtualization environment. Virtualization is technology to emulate the hardware platform and provides great advantages such as server consolidation, software isolation, and so forth. Furthermore, virtualization technology offers flexibility of installation, especially when the earth station system is reconfigured into the next stage of the satellite operation. Since this kind of reconfiguration happens repeatedly, this flexibility saves time and costs. This paper illustrates the advantages of applying virtualization technology to earth station systems based on our performance results in satellite house keeping and in-orbit testing terms.

Acronym Melco = COTS = OS = VM = OR = IOT = HK = HDD = RAID = = SSD = NIC SMAC = GMAC = ORAMS = SOPS = SADA = TCT = TGW = SDP = CCSDS =

C

Mitsubishi Electric Corporation Commercial Off-The-Shelf Operating System Virtual Machine Orbit Raising In-Orbit Testing House Keeping Hard Disk Drive Redundant Array of Inexpensive Disks Solid State Drive Network Interface Card Satellite Monitor and Control Ground Monitor and Control Orbital Mission Analysis Satellite Operation Planning and Scheduling Satellite Data Analysis Total Control Terminal Telesat Gateway Space Data Processing Consultative Committee for Space Data Systems

I. Introduction

ost reduction of satellite projects is important. While aerospace markets scale has not changed a lot recently, the number of hardware and software requirement is increasing [1]. Using commercial off-the-shelf (COTS) products is effective way to save the cost of hardware. Another effective solution is to extend service life of satellites. It reduces the total cost of satellite systems such as earth station manufacturing, satellite launching, and so forth. However, lifespan of COTS products is shorter than satellite one. Thus, we need to replace the earth station system one or two times before deorbit. In this case, using virtualization technology [2] is effective to reduce this replacement cost. Virtualization is technology that enables us to run several different operating systems (OSs) on a single computer. When certain parameters are optimized, the number of required computers can be reduced without performance

1 2 3

Communication Systems Center, [email protected]. Advanced Technology R&D Center, [email protected]. Advanced Technology R&D Center, [email protected]. 1 American Institute of Aeronautics and Astronautics

Copyright © 2014 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved.

degradation. This reduction has many advantages: easy maintenance, low failure rate, simple hardware system, and low cost. Thus, virtualization is being used in a variety of industrial fields. In this paper, we discuss following points of Melco’s earth station systems with virtualization technology. i. Parameter optimization for good performance ii. Cost reduction

Downloaded by 37.44.207.190 on January 14, 2017 | http://arc.aiaa.org | DOI: 10.2514/6.2014-1753

II. Development of virtualized Melco’s earth station systems We have applied virtualization technology to our earth station systems since 2007. This chapter shows the application process of virtualization. Figure 1 shows Melco’s recent projects and when we applied virtualization to each project. We employed virtualization to the development environment in project #1 and #2 in 2007. This was the first time of application to a satellite project. To confirm software operation in virtualization environment, we had many tests and found some problems. For example, there was a case in which software did not work in an early version of virtual machines (VMs). We obtained technical knowledge through a trial and error process and then we applied virtualization to the orbit raising (OR), in-orbit testing (IOT), maintenance, house keeping (HK), and mission operation environment in project #1 in 2010. Nowadays, virtualizing these environments becomes our standard.

Figure 1 Melco’s recent projects

2 American Institute of Aeronautics and Astronautics

III. Parameter optimization of virtualized systems This chapter shows some points of attention when the earth station systems are constructed with virtualization technology. In general, two types of virtualization technology exist; one is hypervisor architecture, and the other is host architecture. The host architecture runs VMs on a host OS. On the other hand, the hypervisor architecture does not need a host OS and run VMs on the software which is directly installed on the hardware. In this chapter, we mainly describe the type of hypervisor architecture for servers.

Downloaded by 37.44.207.190 on January 14, 2017 | http://arc.aiaa.org | DOI: 10.2514/6.2014-1753

A. Parameter optimization Virtualization technology allows host hardware to run multiple guest OSs. However, too many guest OSs cause poor performance of VMs because the host hardware resource is limited. Thus, we develop some easy methods to optimize following parameters. i. Optimize the number of guest OSs from a memory perspective We calculate the number of guest OSs based on the inequality defined as follow:

∑

,

,

(1)

where is memory size of the host machine, is memory size allocated to the hypervisor or the host OS, is the number of guest OSs. , is memory size allocated to the n-th guest OS, and Physical memory access by memory swap causes performance degradation. This inequality prevents this type of access. Moreover, it is safe to set 2% margin to experimentally. In the model of Eq. (1), memory sizes of all guest OSs are same because such model is effective for easy reconfiguration. ii. Optimize the number of guest OSs from a CPU clock perspective Similarly, we decide the parameter condition of CPU clock rate.

∑

, ,

,

(2)

where is the host CPU clock rate, N is the number of cores in a host CPU, R , is the n-th guest CPU clock rate, is the number of cores on the n-th guest CPU, is the number of guest OSs. , iii. Possible number of guest OSs on a physical hard disk drive (HDD) on a single physical HDD based on the inequalities defined as follow: We decide the number of guest OSs

(3)

Let be the number of guest OS cores on a single physical HDD. We set 3 or less experimentally. In the case of host architecture, we found the influence of HDDs seek time bigger than the case of hypervisor architecture. Thus, we should set one guest OS to one physical HDD in the host architecture case, even if processing load is low. iv. Estimate the size of HDD We estimate the size of HDDs by

∑

,

where is the size of HDDs in a host machine, is the size of HDDs occupied by the n-th guest OS.

,

(4) is the size of HDDs occupied by a host OS, and

3 American Institute of Aeronautics and Astronautics

v. Storage technology When we apply storage technology such as redundant arrays of inexpensive disks (RAID), the performance of VM slightly decreases. To avoid this decrease, we should select a RAID level depending on the purpose of HDDs. We select RAID 5 for a data storage and RAID 0 for OSs. Using solid state drive (SSD) instead of HDD is a practical solution as well. vi. Throughput of guest OSs against network interface card (NIC) capacity Sum of VM throughput should be less than 30% of the host machines NIC capacity. Therefore, we define the inequality as shown below to avoid network collision:

∑

Downloaded by 37.44.207.190 on January 14, 2017 | http://arc.aiaa.org | DOI: 10.2514/6.2014-1753

where

,

, ,

(5)

is the throughput of the n-th VM. It is safe to set 70% margin to ∑

,

experimentally.

B. Solutions to miscellaneous problems i. CPU power saving mode CPU power saving mode often causes delay of scheduled events in VMs. For example, we experienced an event which was scheduled in one second interval in a VM did not work accurately. Thus, we should disable this mode in a BIOS setting. ii. Replacement of legacy machines Some legacy OSs requires legacy devices. Several problems as shown below could occur when legacy machines are replaced by new ones. (1) If applications depend on legacy devices, VMs do not emulate the platform of old hardware. In this case, we must do as shown below: Emulate legacy devices… a) by employing legacy device drivers for host machine interfaces. b) by employing host OS interfaces to legacy device drivers. c) by rewriting applications to remove legacy device dependency. (2) OSs freeze when the number of network sessions exceeds certain number. This is because legacy OSs do not expect such high performance of modern hardware. In this case, we must modify parameters of OSs. We deal with these problems in cooperation with R&D center in Melco. Furthermore, we have developed customized VM for special purposes. Its code name is MENTOS (Melco’s CentOS). It is Xen-based VM technology. iii. Replication of VM images through networks VM makes reconfiguration very easy because machines can be transferred just copying files. However, we experienced hang up of a network router when several VM images were replicated at the same time through a network for reconfiguration. Thus, some appropriate router tests are necessary.

IV. System example We have an in-house suite of software, called Birdstar, for satellite operations. Table 1 shows components and their functions of Birdstar. Figure 2 shows the example of an earth station system with Birdstar. This example is from Project #4 in Figure 1. We only use SMAC, ORAMS, and SADA from Birdstar. In this project, other system venders provide functions of 4 American Institute of Aeronautics and Astronautics

Downloaded by 37.44.207.190 on January 14, 2017 | http://arc.aiaa.org | DOI: 10.2514/6.2014-1753

ground monitor and control, and operation planning, so we do not use GMAC and SOPS. TGW and SDP are software for operating baseband facilities, whose functions are routing networks and processing CCSDS data. This system consists of two parts: The left part is for OR and the right one is for IOT and HK. Primary and secondary of each part are redundant systems. Primary system is mainly used and secondary is used only when primary one is out of operation. Moreover, there are several sets of servers each of which is dedicated to one satellite. Virtualization software is installed into these servers as shown in Figure 3. There are four guest OSs, each of which has one Birdstar component, on a bare metal hypervisor. Networks of dotted lines in Figure 2 are connected only in reconfiguration. The system for IOT and HK is constructed by moving VM images from the system of OR through dotted lines and routers. Each server in the system for OR has two NICs. One is for operation such as telemetry and command, and the other is for system reconfiguration. Thus, the traffic load of reconfiguration is separated from that of operation. Component SMAC GMAC ORAMS SOPS SADA TCT

Table 1 Components and functions in Birdstar Function Monitor and control satellites Monitor and control ground facilities Analyze satellite orbit and develop procedures of satellite operations Plan and integrate command operations for satellite and ground system Store telemetry Total control terminal of Birdstar

5 American Institute of Aeronautics and Astronautics

System for Operation

System for Orbit Raising Primary

Primary

Layer 3 Switch

Router #1

Layer 3 Switch

Satellite A SMAC

SMAC SADA ORAMS TGW & SDP

Router #2

SADA Satellite B ORAMS

SMAC SADA ORAMS TGW & SDP

Downloaded by 37.44.207.190 on January 14, 2017 | http://arc.aiaa.org | DOI: 10.2514/6.2014-1753

TGW & SDP

Layer 2 Switch #1 - #6

TCT #1 - #30 Satellite N

Secondary

Layer 3 Switch

TCT #1 - #10 SMAC

Layer 2 Switch

Secondary Layer 3 Switch

SADA

Satellite A ORAMS SMAC SADA ORAMS TGW & SDP

TGW & SDP

Satellite B SMAC SADA ORAMS TGW & SDP

Satellite N

Figure 2 System example

6 American Institute of Aeronautics and Astronautics

SMAC

SADA

ORAMS

SDP & TGW

Guest OS

Guest OS

Guest OS

Guest OS

Drivers

Drivers

Drivers

Drivers

Hypervisor Hardware

Downloaded by 37.44.207.190 on January 14, 2017 | http://arc.aiaa.org | DOI: 10.2514/6.2014-1753

Figure 3 Server architecture

V. Performance measurement This chapter shows the performance and reconfiguration time in the system of Figure 2. C. Performance We monitored VM performance when the system is in HK operation. The performance of CPU, network, and memory is shown in Figure 4, Figure 5, and Figure 6 respectively. Figure 4 and Figure 5 show CPU usage ratio and data transmit rate. The sharp fluctuations in these trend graphs are in 10 minutes intervals because SMAC transmits telemetry to SADA periodically. The machine had Intel Xeon CPU with 3.2 GHz. From Figure 4 and Figure 5, we consider the performance of CPU and network has an enough margin. Figure 6 shows one VM consumes approximately 2 GB memory and active memory is approximately up to 500 MB. Thus, we consider the VM has an enough memory margin. VM memory is allocated from physical memory. When enough physical memory is not available, network performance is degraded and this degradation causes packet loss problems. Thus, we should have enough amount of physical memory.

7 American Institute of Aeronautics and Astronautics

35

1200

30

1000

25

800

20

600

15

400

10

200

5

0 10:05:20

0 10:17:20

10:29:20

10:41:20

10:53:20

Time CPU usage in MHz

CPU usage

Figure 4 CPU performance Network Performance 120 100 80 KBps

Downloaded by 37.44.207.190 on January 14, 2017 | http://arc.aiaa.org | DOI: 10.2514/6.2014-1753

MHz

1400

60 40 20 0 9:39:20

9:51:20

10:03:20

10:15:20

10:27:20

Time Network data receive rate

Network data transmit rate

Figure 5 Network performance

8 American Institute of Aeronautics and Astronautics

Percent

CPU Performance

Memory Performance 2500000

2000000

Downloaded by 37.44.207.190 on January 14, 2017 | http://arc.aiaa.org | DOI: 10.2514/6.2014-1753

KB

1500000

1000000

500000

0 11:28:40

11:40:40

11:52:40

12:04:40

12:16:40

Time Memory consumed

Memory active

Figure 6 Memory performance D. Reconfiguration This section shows the reconfiguration time of the system in Figure 2. We reconfigured the system by replicating VM images from the OR system to the IOT and HK operation system in Figure 2. The size of VM images and reconfiguration time are shown in Table 2. We replicated 28 VM images. The average replication time for one image was 79 minutes and the average replication speed was 83.7 Mbps. Software Size Time

Table 2 Size of VM images and reconfiguration time SMAC SADA ORAMS 40 GB 20 GB 120 GB 79 min.

SDP & TGW 16 GB

VI. Evaluation & Conclusion E. Evaluation In this section, we compare the result of reconfiguration time between the virtualization applied system and nonvirtualization applied system. We overcame several difficulties related to virtualization technology as we shown in section III. As a result, we succeeded in reducing reconfiguration time as shown in Table 3.

9 American Institute of Aeronautics and Astronautics

Table 3 Reconfiguration time HW relocation

Construction time of a master VM image + Replication time

Application SW installation (Application SW installation is included in OS installation)

2 weeks

Construction time of a master OS image × the number of hardware

Installation time × the number of hardware

2 weeks

3 weeks (150Hr)

1 week

Using VM

Several hours

Not using VM

Time reduction effect in Figure 2 (8 servers & 30 TCT clients) Total reduction effect

Reconfiguration time

Downloaded by 37.44.207.190 on January 14, 2017 | http://arc.aiaa.org | DOI: 10.2514/6.2014-1753

Work item OS installation

1.5 month

i. Reduce hardware relocation time Hardware relocation is time-consuming process. For example, if we relocate hardware in Figure 2, relocation time would be about two weeks. This relocation includes server rack relocation, laying cables, and local network construction. Furthermore, we often need some additional servers in order to protect information between different customers. In this case, we expect at least one month as a lead time of additional servers considering their arrival time. Virtualization technology offers a solution to this problem. VMs are able to run several OSs in a single server, so the number of servers is decreased. Small number of servers reduces several workloads such as laying cables. ii. Reduce OS installation time OS installation takes time as well, especially when a new standard device interface is employed to the hardware. In this case, we must modify the drivers of OSs. Since it is difficult to obtain a new hardware with the specification equivalent to the original one, this situation often happens. Furthermore, we take several hours to construct a master image of an OS. For instance, if we use non-virtualization applied system in Figure 2, we must install an OS to each server in eight times. Each installation would take four hours. Similarly, we would also take four hours in TCT setup. However, if we use virtualization in Figure 2, we only need four hours for master VM image construction and replication time of eight servers and 30 TCT clients. iii. Reduce application software installation time Application software installation is less time-consuming than hardware relocation and OS installation. However, we must set up each software environment in non-virtualization applied systems. If we use virtualization, we need not to install application software because OS and application software is set up at the same time by just copying VM images. Virtualization based system provides solutions toward the problems mentioned above. As the number of servers decreases, the workload of hardware relocation is reduced. Installation of OS and software becomes faster and more accurate by using VM images. This is because the exact copy of computer can be constructed by just replicating a VM image. VM images demand only replication time and we need not to set each computer environment. Consequently, we reconfigured eight severs and 30 TCT clients in Figure 2 efficiently. We were able to shorten reconfiguration time with approximately 95% compared to the case where virtualization is not applied. F. Conclusion In this paper, the advantages of applying virtualization technology to Melco’s earth station systems with Birdstar were discussed. We showed some constraints in virtualization application and evaluated effectiveness using a virtualized system. According to our results, virtualization technology is effective to reduce reconfiguration time without performance degradation. Since the reconfiguration time was reduced with approximately 95%, the cost of 10 American Institute of Aeronautics and Astronautics

reconfiguration was also reduced dramatically. Thus, our standard earth station systems are built on virtualization technology.

VII. Future Work Automatic reconfiguration systems, which detect system failure and start reconfiguration, will improve the availability of our earth station systems. We will design and test such systems based on virtualization technology. We also need to develop a private cloud computing solution to our systems of servers and TCT clients in order to reduce computer hardware cost.

References 1

Downloaded by 37.44.207.190 on January 14, 2017 | http://arc.aiaa.org | DOI: 10.2514/6.2014-1753

http://www.aia-aerospace.org/assets/Stat_Series_4_Aerospace_Deflator_-_2014.pdf 2 Michael Schmidhuber, Ursula Kretschel, and Thomas Singer, “Virtualizing Monitoring and Control Systems: First Operational Experience and Future Applications,” AIAA 2010-2340, SpaceOps, Huntsville, Alabama, April 2010.

11 American Institute of Aeronautics and Astronautics