IBM Linux
Configuring IBM PowerKVM on Power systems
IBM Linux
Configuring IBM PowerKVM on Power systems
Note Before using this information and the product it supports, read the information in “Notices” on page 63.
Second Edition (October 2014) © Copyright IBM Corporation 2014. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
Contents Configuring IBM PowerKVM on Power Systems . . . . . . . . . . . . . . 1 Overview of installation steps for PowerKVM . . . 1 Download PowerKVM . . . . . . . . . . . 1 Burning a DVD from an ISO image using Linux . 2 Setting up a network boot server . . . . . . 2 Connecting to the service processor. . . . . . . 3 Connecting using a PC or notebook . . . . . 3 Connecting using DHCP network . . . . . . 5 Petitboot bootloader . . . . . . . . . . . . 6 Configuring Petitboot . . . . . . . . . . 6 Editing Petitboot configuration options . . . . 7 Automated network installation . . . . . . . . 8 The ibm-configure-system utility . . . . . . . 9 Kimchi management tool . . . . . . . . . . 9 Intelligent Platform Management Interface (IPMI) on PowerKVM . . . . . . . . . . . . . . 10 Installing IPMI . . . . . . . . . . . . 10 Connecting to your system with IPMI . . . . 10 Changing your IPMI password . . . . . . . 10 Common IPMI commands . . . . . . . . 11 Creating guests . . . . . . . . . . . . . 11 Creating a guest with Kimchi . . . . . . . 11 Setting up a template using Kimchi . . . . . 12 Creating a guest with the virsh command . . . 12 Creating guests with the virt-install command 13 Common virsh command options . . . . . . . 13 PowerKVM storage . . . . . . . . . . . . 14 Common virsh command options - Storage . . 14 Find storage pool sources with virsh . . . . . 15 Valid storage pools types . . . . . . . . . 16 Storage pool . . . . . . . . . . . . . 17 Setting up a storage pool with Kimchi . . . 17 Creating storage pool with virsh . . . . . 18 Networking on PowerKVM . . . . . . . . . 19 Common virsh command options - Networking 20 Setting up a network connection with Kimchi . . 21 Creating a network bridge with the ibm-configure-system utility . . . . . . . 21 Verifying the default virtual network . . . . . 22 Configuring KVM guests to use bridge . . . . 22 Adding guest para-virtualized network devices with libvirt . . . . . . . . . . . . . 23 Using vhost-net for high-bandwidth applications 23 Hot-plugging a network connection . . . . . 24 VLAN segmentation . . . . . . . . . . 24 Configuring VLAN segmentation . . . . . 25 Configuring 802.1q VLANs in Kimchi . . . 27 PowerKVM and Quality of Service . . . . . 27 Remote management of PowerKVM with libvirt . . 28 Overview of PowerKVM remote management . . 28 Remote management with SSH tunnels . . . . 30 Managing PowerKVM guests remotely with the virsh command . . . . . . . . . 30
© Copyright IBM Corp. 2014
Displaying the remote PowerKVM VNC console with any VNC client . . . . . . Remote management with SASL authentication and encryption . . . . . . . . . . . . Remote management with TLS . . . . . . . Step 1. Creating a CA key and certificate in your PowerKVM host . . . . . . . . . Step 2. Creating the client and server keys and certificates in your PowerKVM host . . . . Step 3. Distributing keys and certificates to the PowerKVM host server . . . . . . . . Step 4. Distributing keys and certificates to clients or management stations . . . . . . Step 5. Editing the libvirtd daemon configuration . . . . . . . . . . . . Step 6. Changing the firewall configuration . . Step 7. Verifying that remote management is working . . . . . . . . . . . . . User management . . . . . . . . . . . . KVM guest migration . . . . . . . . . . . Common virsh command options: Migrate. . . Migrating live with virsh command . . . . . Migrating a KVM guest offline with migrate option . . . . . . . . . . . . . . . Migrating a KVM guest offline with dumpxml . . The sVirt service. . . . . . . . . . . . . Overview of the sVirt service . . . . . . . Creating static sVirt labels . . . . . . . . Verifying sVirt labeling by examining the labels Verifying sVirt labeling by viewing the domain.xml file . . . . . . . . . . . . Manage PowerKVM resources . . . . . . . . Common virsh command options - Resources. . Manage processors and memory . . . . . . Simultaneous Multi-Threading (SMT). . . . Enabling micro-threading . . . . . . . . Processor pinning . . . . . . . . . . Over-committing processor and memory resources . . . . . . . . . . . . . Configuring huge pages . . . . . . . . Memory ballooning. . . . . . . . . . Enabling Kernel Same-page Merge (KSM) . . Enabling PCI passthrough . . . . . . . . Update PowerKVM. . . . . . . . . . . . Updating PowerKVM with Kimchi . . . . . Updating PowerKVM with yum . . . . . . . Updating PowerKVM with the ibm-update-system utility. . . . . . . . . Upgrade PowerKVM . . . . . . . . . . . Updating the firmware . . . . . . . . . Upgrading PowerKVM to a new release . . . . Monitor and debug information . . . . . . . Install and update packages . . . . . . . . . Migrating to IBM PowerKVM from IBM PowerVM Migrating to IBM PowerVM from IBM PowerKVM
31 32 33 34 35 37 38 38 39 39 40 41 41 42 42 43 43 43 44 44 46 46 47 47 47 48 49 50 50 51 52 54 56 56 56 57 57 57 58 59 59 60 60
iii
Notices . . . . . . . . . . . . . . 63 Privacy policy considerations
iv
.
.
.
.
.
.
.
. 64
Linux: Configuring IBM PowerKVM on Power systems
Trademarks .
.
.
.
.
.
.
.
.
.
.
.
.
. 64
Configuring IBM PowerKVM on Power Systems Use these instructions for installing and configuring PowerKVM™ on Power Systems™.
Overview of installation steps for PowerKVM IBM PowerKVM is available on specific Linux-only Power system servers that are built on POWER8 technology. For specific prerequisites and limitations, see PowerKVM concepts. Specific instructions for installing and configuring PowerKVM are available in quick start guides. To download a PDF, see Quick start guides for IBM PowerKVM. Table 1. Overview of installation steps Step
Description
Step 1: Download IBM PowerKVM
Download the IBM PowerKVM installer file. See “Download PowerKVM.”
Step 2: Connect to your server
Connect to the service processor of your server using either DHCP or by setting up a PC or notebook as a console.
Step 3: Setting up the service processor
Set up your service processor to use OPAL for a hypervisor and create an IPMI password.
Step 4: Powering on your server
After setting up the service processor, use IPMI as a console interface to power on your server.
After you download the installer file, make it available to the system by either “Burning a DVD from an ISO image using Linux” on page 2 or “Setting up a network boot server” on page 2.
Step 5: Choose your boot options The Petitboot bootloader scans your system and finds boot options for you. Select your boot option and press Enter. If you option is not available, configure Petitboot. See “Petitboot bootloader” on page 6. Step 6: Installing PowerKVM
PowerKVM provides an installation wizard to walk you through the installation process.
Step 7: Booting PowerKVM after installation
At the end of the installation process, restart your server. Select PowerKVM as your boot option.
Step 8: Use ibm-configuresystem utility to set your root password, network information, and other settings.
If you have not yet added networking information to your server, use the ibm-configure-system utility to set that up. See “The ibm-configure-system utility” on page 9.
Step 9: Connecting to Kimchi
Use Kimchi to manage your host server and guests. See “Kimchi management tool” on page 9.
Download PowerKVM If your system is not preconfigured with PowerKVM, you need to download the installer file. PowerKVM installation files are available from the Entitled System Support site at http://www304.ibm.com/servers/eserver/ess/index.wss. Fixpack files are available from Fix Central at https://www.ibm.com/support/fixcentral/.
© Copyright IBM Corp. 2014
1
After you download the file, make it available to your server. To update your current PowerKVM release, see “Update PowerKVM” on page 56. To upgrade PowerKVM to a new release, see “Upgrade PowerKVM” on page 57.
Burning a DVD from an ISO image using Linux Follow these instructions to burn a DVD from the ISO image using Linux
Procedure 1. If it is not already installed, install the Linux package dvd+rw-tools. To determine whether the package is installed, enter the following command: rpm -qi dvd+rw-tools 2. Download the installer image for the IBM PowerKVM to your Linux system. You must have a DVD-Recorder drive installed on your system 3. Determine the DVD recorder device ID by running the following command: hwinfo -cdrom | grep "Device File" In most cases, the output of this command is similar to this example: Device File: /dev/sr0 (/dev/hdc) 4. Insert a high-quality, blank DVD-R disk into the DVD recorder. Do not use DVD+R, DVD-RW, or DVD-RAM. 5. Run the following command: /usr/bin/growisofs -dvd-compat -Z /dev/hdc=file where file is the name of the file.
What to do next You can use a different method to create a DVD from an ISO image. For example, if you have a PC with software that allows the creation of DVD media from an ISO image, then you can use that software to burn a DVD. Regardless of the method you use to create a DVD, use high-quality DVD-R media to ensure the best data transfer. Do not use DVD+R, DVD-RM, or DVD-RAM media
Setting up a network boot server Optionally, you can set up a network boot server.
About this task In this example setup, a network boot server is loaded with the PowerKVM ISO file and then made available through an HTTP server over a DHCP network for the target Power® system to find. Your specific setup might vary from this example. You need a DHCP server on the same subnetwork as your target system and an HTTP server. v DHCP server on the same subnetwork as the target system v HTTP server network available to the target system v PowerKVM ISO image file
Procedure 1. Upload the PowerKVM ISO file to the HTTP server. For example, to wwwroot/powerkvm. 2. Extract the ISO image file content. For example, to wwwroot/powerkvm/os. The following directories are created in wwwroot/powerkvm/os: LiveOS/ TRANS.TBL etc/ packages/
2
Linux: Configuring IBM PowerKVM on Power systems
ppc/ 3. Create a boot configuration file for the target Power system. For example, to wwwroot/powerkvm/ pxe.conf. Edit it to include the following contents: label IBM PowerKVM Install kernel http://YOUR-SERVER-IP/wwwroot/powerkvm/os/ppc/ppc64/vmlinuz initrd http://YOUR-SERVER-IP/wwwroot/powerkvm/os/ppc/ppc64/initrd.img append selinux=0 rd.dm=0 rd.md=0 root=live:http://YOUR-SERVERIP/wwwroot/powerkvm/os/LiveOS/squashfs.img repo=http://YOUR-SERVERIP/wwwroot/powerkvm/os/packages console=hvc0 console=tty0
4. Edit your DHCP server configuration file. This file is typically called /etc/dhcpd.conf. Add the following options that are bold: (...) option conf-file code 209 = text; subnet 192.168.122.0 netmask 255.255.255.0 { option routers 192.168.122.1; (...) host your-system{ hardware ethernet system macaddr fixed-address system ip; option host-name "system hostname"; option conf-file "http://YOUR-HTTPSERVER-IP/powerkvm/pxe.conf"; } }
5. Restart your DHCP server.
Connecting to the service processor Plan your connection to your server.
Connecting using a PC or notebook You can connect to the service processor using a PC or notebook.
Before you begin Use this type of connection to configure a local network with a static IP address. 1. If your system belongs in a rack, install your system into that rack. For instructions, see IBM® Power Systems Hardware information. Look for instructions specific to your system. 2. Remove the shipping brackets from the power supplies. Ensure that the power supplies are fully seated in the system.
Procedure 1. Connect the power cables for the server. After the power cables are connected, the firmware starts. Wait for a few minutes for this process to complete. Look for the green power LED on the control panel to start flashing, indicating that it is ready to use, and for the prompt 01 N V=N to appear on the display. Note: You cannot connect to the service processor until this process is complete. 2. Connect an Ethernet cable from the PC or notebook to the Ethernet port labeled HMC1 on the back of the managed system. If HMC1 is occupied, connect an Ethernet cable from the PC or notebook to the Ethernet port labeled HMC2 on the back of the managed system. 3. Set your IP address on your PC or notebook to match the default values on your Power system. Use the table to determine your default values:
Configuring IBM PowerKVM on Power Systems
3
Table 2. Network configuration information Server connector
Subnet mask
IP address of service processor
IP address on PC or notebook
HMC1
255.255.255.0
169.254.2.147
169.254.2.140
HMC2
255.255.255.0
169.254.3.147
169.254.3.140
To set your IP address in Linux, follow these steps: a. Log in as root. b. Start a terminal session c. Run the follow command: ifconfig -a. Record these values. d. Type ifconfig ethx xxx.xxx.x.xxx netmask nnn.nnn.nnn.n, where the xxx.xxx.xxx.xxx is the IP address and nnn.nnn.nnn.n is the Subnet mask. Replace ethx with either eth0 or eth1, depending on what your computer is using. To set your IP address in Windows 7, follow these steps: a. Click Start > Control Panel. b. Select Network and Sharing Center. c. Click the network that is displayed in Connections. d. Click Properties. e. If the security dialog box is displayed, click Continue. f. Select Internet Protocol Version 4. g. Click Properties. h. Select Use the following IP address. i. Complete the IP address, Subnet mask, and Default gateway fields by using from the Network configuration information table. j. Click OK > Close > Close 4. Open a web browser and go to https://ip_address where ip_address is the IP address that you set. 5. The first time that you connect to the service processor, enter the default user and password: On a preinstalled system, use this user name and password: v User ID: admin v Password: admin After you log in, you must change the password. Be sure to remember this password! After you change the password, the service processor main menu appears. 6. From the main menu, select System Configuration > Hypervisor Configuration. 7. Select KVM as your Hypervisor Mode and set up a password for IPMI session. 8. To use IPMI, you must configure Network access. From the main menu, select Network Services > Network Configuration. To configure network access, follow these steps: a. From the Network Configuration display, select IPv4 and Continue. b. Select Configure this interface? c. Verify that IPv4 is selected. d. Select Static for the type of IP address. e. Enter a name for the host system. f. Enter an IP address for the system. g. Enter a subnet mask. h. Enter a domain name. i. Enter the IP address for the DNS. 9. After you set the values for the Network configuration, click Continue.
4
Linux: Configuring IBM PowerKVM on Power systems
10. Click Save Settings. The network settings are configured for the system. You can remove the Ethernet cable from your PC or notebook and connect it to the network switch.
What to do next You are now ready to power on your server! Go to “Connecting to your system with IPMI” on page 10 for instructions.
Connecting using DHCP network If you connect your server to a DHCP network, you can access the service processor through the web interface.
Before you begin 1. If your system belongs in a rack, install your system into that rack. For instructions, see IBM Power Systems Hardware information. Look for instructions specific to your system. 2. Remove the shipping brackets from the power supplies. Ensure that the power supplies are fully seated in the system.
Procedure 1. Before you power on the service processor, connect an Ethernet cable from the eth0 port to a network that has a DHCP server. 2. Connect your power cords to the server and plug the power cords into the power source. Wait a few minutes for the process to complete. Look for the green power LED on the control panel to start flashing, indicating that it is ready to use, and for the prompt "01 N V=N" to appear on the display. 3. If you do not know what IP address your DHCP server that is assigned to the service processor, follow these steps to find it: a. Access the control panel for your server. b. Select 02 to enter the Mode menu. c. Change N to M to start Manual mode. d. Press Enter to exit mode menu. e. Scroll to function 30 using Increment or Decrement buttons. f. Press Enter to enter subfunction. g. Use the Increment or Decrement buttons to select a network device. 00 = SP A: ETH0 01 = SP A: ETH1 h. Press Enter to display the selected IP address. After you record the IP address, put the server back into Normal mode: i. Use the Increment or Decrement buttons to select subfunction exit. j. Press Enter to exit subfunction mode. k. Scroll to 02 using Increment or Decrement buttons. l. Select 02 to enter the Mode menu. m. Change M to N to return to Normal mode. n. Press Enter to exit mode menu. You can now access the service processor through the web interface. 4. Open a web browser and enter https://ipaddress where ipaddress is the address you recorded. 5. The first time that you connect to the service processor, enter the default user and password: v User ID: admin v Password: admin Configuring IBM PowerKVM on Power Systems
5
After you log in, you must change the password. Be sure to remember this password! After you change the password, the service processor main menu appears. 6. From the main menu, select System Configuration > Hypervisor Configuration. 7. Select PowerKVM (or OPAL) as your Hypervisor Mode and set up a password for IPMI session.
What to do next You are now ready to power on your server. Go to “Connecting to your system with IPMI” on page 10 for instructions.
Petitboot bootloader After the system powers on, the Petitboot bootloader scans local boot devices and network interfaces to find boot options that are available to the system. Petitboot returns a list of boot options that are available to the system. If you set up a DHCP server and provided the boot arguments as outlined in “Setting up a network boot server” on page 2 or if you are using a media installation, your boot option is available. Select it and press Enter. If you are using a static IP or if you did not provide boot arguments in your network boot server, you must provide the details to Petitboot. Note: After approximately 10 seconds, Petitboot boots the first option.
Configuring Petitboot You can configure Petitboot to find your boot options. Use these instructions if you did not set up your DHCP network boot server or if you are using a static IP address.
About this task Follow these steps to configure Petitboot to find the IBM PowerKVM installer.
Procedure 1. To provide your network information to Petitboot, follow these steps: a. On the Petitboot main menu, select System Configuration and press Enter to open the Petitboot System Configuration window. b. On the Petitboot System Configuration window, specify your network information. DHCP on all active interfaces Automatically assigns IP addresses to each network interface. DHCP on a specific interface Automatically assigns IP addresses to a selected network interface. Static IP configuration Use this option to manually specify an IPv4 network configuration. If you select this option, you must provide an IP address, netmask, gateway address, and optionally a DNS server address. Note: The network interface names, such as eth0, that Petitboot uses are not guaranteed to match the names that are used by IBM PowerKVM. To ensure that you are using the expected network interface, compare MAC addresses. When you are finished, select OK and press Enter. 2. Select e to open the Petitboot Option Editor for edit or n to create new options. 3. Use the Petitboot Option Editor window to edit or create boot options.
6
Linux: Configuring IBM PowerKVM on Power systems
v If your boot resources are on a local disk device, select the device from the list, enter your boot resources as paths to that device, and click Enter. v If you want to use full URLs to your boot resources, select Specify paths/URLs manually and then enter your boot options: 4. In the Kernel field, enter the path to the kernel. This field is mandatory. For example, enter a path similar to this URL address: http://networkservername/path/ppc/ppc64/vmlinuz
In this example, replace networkservername/path with your network server IP address and the path on that server to the kernel. 5. In the Initrd field, enter the path to the init ramdisk. For example, enter a path similar to this URL address: http://networkservername/path/ppc/ppc64/initrd.img
In this example, replace networkservername/path with your network server IP address and the path on that server to the initrd.img file. 6. In the Device tree field, enter the path to the device tree blob. This field is optional. 7. In the Boot arguments field, enter the kernel command-line arguments. For example, enter a path similar to this URL address: rd.dm=0 rd.md=0 root=live:http://networkservername/path/LiveOS/squashfs.img repo=http://networkservername/path/packages console=hvc0 console=tty0
In this example, replace networkservername/path with your network server IP address and the path on that server. If you are using a static IP address, include an argument for it. The argument looks like this example: ifname=net0:MAC_address ip=static_ip_address::gateway:netmask:hostname:net0:none
Replace the variables as follows: net0: The name that you are calling this connection. MAC_address: The MAC address of the network device. static_ip_address: The static IP address of the device. gateway: The gateway of the device. netmask: The netmask of the device. hostname: The host name of the server. hostname is not required for this connection. After you set your options, select OK and press Enter. 8. On the Petitboot main window, select your boot option and press Enter.
Editing Petitboot configuration options You can change the default options of Petitboot to automatically boot the default option or to change the amount of time before Petitboot automatically boots the default option. v To change an existing option, type E (edit) v To add a boot option, type N (new) v To display information about the system, type I (information) or select System Information in the window. v To change to the system configuration, type C (configuration) or select System Configuration in the window. On the configuration window, you can edit the following options: Autoboot
Configuring IBM PowerKVM on Power Systems
7
Don't autoboot Select to stop at the Petitboot bootloader window without continuing the boot. Autoboot from any disk/network device Select to allow Petitboot to automatically boot the default option. Petitboot quickly boots your system without changing any options. Only autoboot from a specific disk/network device Select to allow Petitboot to automatically boot from a specific disk or network device. Petitboot quickly boots your system without changing any options. Timeout Sets the length of time, in seconds, before Petitboot automatically boots the default option. Network options DHCP on all active interfaces Automatically assigns IP addresses to each network interface. DHCP on a specific interface Automatically assigns IP addresses to a selected network interface. Use Static IP configuration Use this option to manually specify an IPv4 network configuration. If you select this option, you must provide an IP address, netmask, gateway address, and optionally a DNS server address.
Automated network installation You can install PowerKVM using an automated network installation file similar to a kickstart file and the boot option kvmp.inst.auto. The boot option kvmp.inst.auto supports HTTP, TFTP, and NFS network installations. Supported items for the installation file include the following options: v Pre-install scripts v Post-install scripts v Partition (required) v Network v Root password v Timezone A typical installation file looks like the following example: partition / --ondisk=/dev/sda network --device enP3p6s0f0 --bootproto dhcp rootpw passw0rd timezone US/Chicago
Note: You can also use /dev/disk/by-id or /dev/disk/by-path for the value of --ondisk. For example, partition / --ondisk=/dev/disk/by-id/1IBM_IPR-0_5EXXXXXXXXXXXX. Running the command, ls -l on /dev/disk/by-id provides the corresponding values for sdx disks and if multipath is enabled. You can find paths by running the command ls -l /dev/disk/by-path/ To run the installation file, configure Petitboot to use it, similar to the following example: label IBM PowerKVM Automated Install (kickstart) kernel http://192.0.2.1/kvmonp_netboot/vmlinuz initrd http://192.0.2.1/kvmonp_netboot/initrd.img append ksdevice=bootif lang= kssendmac text root=nfs:192.0.2.1:/var/www/kvmonp_netboot/iso/ibm-powerkvm.iso rd.dm=0
8
Linux: Configuring IBM PowerKVM on Power systems
rd.md=0 ipv6.disable=1 kvmp.inst.auto=tftp://192.0.2.1/kickstart/powerkvm.ks ifname=net0:e4:1f:13:fd:cf:7d ip=192.0.2.10::192.0.2.254:255.255.255.0:powerkvm-host:net0:none
The ibm-configure-system utility The ibm-configure-system utility is a command-line interface that you can use to change your root password, configure time zones, set the date and time, configure network interfaces, and set your DNS configuration. To use the ibm-configure-system utility, run ibm-configure-system from an IPMI console window that is connected to the server. Navigate the menu with the up and down arrow keys or TAB. To change the root password, select Root Password and press Enter. Type a new password and confirm it. Select OK and press Enter. To set your time zone, select Timezone Selection and press Enter. Select your time zone from the list. To set the system to use Coordinated Universal Time (UTC), navigate to it and then press Space bar. When you are finished, select OK and press Enter. To change your system date and time, select Date and Time and press Enter. Type your date and time. When you are finished, select OK and press Enter. To configure your network, select Configure Network. Select a device and then select OK and press Enter. On the Network Device Configuration window, define your network connection. Select Save and press Enter. To set your DNS server information, select DNS configuration. Enter your DNS configuration, select OK, and press Enter. When you are finished configuring your system options, press Tab to select Exit and press Enter. Confirm your Exit and press Enter again.
Kimchi management tool Kimchi is a management tool, which is designed to get you started with IBM PowerKVM. Kimchi is an HTML5-based management tool that runs as a daemon on the hypervisor host and interfaces with underlying libvirt, QEMU, and KVM components. Use Kimchi to create and manage guests, monitor your host system, create networking interfaces, add storage, and update packages. To use Kimchi, open a browser and point it to https://ip_address:8001 where ip_address is the IP address of your KVM system. Log in using the admin user name and password. Note: When you connect to Kimchi, make sure that you enable SSL connections in your browser. For Firefox browsers, you might also be required to connect to https://ip_address:64667/ where ip_address is the IP address of your host KVM system and accept the self-signed certificate. Connect using the HTTP secure (HTTPS). Use Kimchi to allocate storage, configure a network bridge, apply firmware updates, and install guests.
Configuring IBM PowerKVM on Power Systems
9
Intelligent Platform Management Interface (IPMI) on PowerKVM The Intelligent Platform Management Interface (IPMI) tool is the default console to use with PowerKVM.
Installing IPMI The IPMItool utility is available with most distributions of Linux.
About this task If you do not have a version of IPMItool on your system, follow these steps:
Procedure 1. Download a version from SourceForge at http://ipmitool.sourceforge.net/. 2. Install the .tar file.
What to do next v You must be logged in as root to use the IPMItool command works only for root user. v You can find your version of IPMI by running ipmitool -V v IPMI can be impacted by network traffic and will disconnect after seven attempts. The default retry-interval is 100 milliseconds. You can reset the default retry-interval to higher number by running the follow command: ipmitool -I lanplus -H myserver.example.com -P mypass sol set retry-interval 50
This command sets the retry-interval to 500 ms.
Connecting to your system with IPMI To connect to your system with IPMI, you need to know the IP address of the server and have a valid password. To power on your Power server, run the following command: ipmitool -I lanplus -H fsp_ip_address -P ipmi_password power on
where ipaddress is the IP address of the Power server and ipmi_password is the password set up for IPMI. To activate your IPMI console, run the following command: ipmitool -I lanplus -H ip_address -P ipmi_password sol activate
For additional commands, see “Common IPMI commands” on page 11.
Changing your IPMI password Learn how to change your IPMI password
About this task Your IPMI password is set on the IBM PowerKVM host through the ASMI menus. To change your IPMI password, follow these steps:
Procedure 1. Access the ASMI menu. 2. From the main menu, select Login Profile -> Change Passwords. 3. Select IPMI from the list of user IDs. 4. Enter the current password for the administrator and then enter and confirm a password for IPMI.
10
Linux: Configuring IBM PowerKVM on Power systems
5. Click Continue.
Common IPMI commands You can use the IPMI command to perform several managing tasks for your Power system. Table 3. Common IPMI commands Command option
Description
ipmitool -I lanplus -H myserver.example.com -P mypass chassis power on
Powers on the server
ipmitool -I lanplus -H myserver.example.com -P mypass chassis power off
Powers off the server
ipmitool -I lanplus -H myserver.example.com -P mypass chassis status
Checks the server status
ipmitool -I lanplus -H myserver.example.com -P mypass chassis power cycle
Power cycle the server
ipmitool -I lanplus -H myserver.example.com -P mypass sol activate
Activates SOL system console
ipmitool -I lanplus -H myserver.example.com -P mypass sol deactivate
Deactivates SOL system console
ipmitool -I lanplus -H myserver.example.com -P mypass sel list
Returns an error log
ipmitool -I lanplus -H myserver.example.com -P mypass sdr list
Lists status of all sensors
ipmitool -I lanplus -H myserver.example.com -P mypass sol set retry-interval value
Sets the default retry-interval value in milliseconds.
ipmitool -I lanplus -H myserver.example.com -P mypass fru print
Prints the FRU information
ipmitool -I lanplus -H myserver.example.com -P mypass user list
Lists the IPMI users
Creating guests You can create guests by using Kimchi or by using the command line interfaces.
Creating a guest with Kimchi Use Kimchi to create virtual machines, or guests.
Procedure 1. Open a browser and go to https://ip_address:8001 where ip_address is the IP address of your KVM system. 2. Select the Guests page. 3. Click the green plus sign (+) to create a guest. 4. On the Guests page, enter a name for the virtual machine guest. Select a template from the list. If no templates exist, click Create a template. If you want a different template from the ones that are displayed, select the Templates page and create or edit one. 5. Select the source media for the template. 6. Select an ISO image. 7. Click Create. The guest is created but not started. 8. Start the guest by clicking the red power button or by selecting Start from the Actions menu. The button changes to green as the guest starts. 9. To connect to the remote console (Livetile), select Connect from the Action menu
Configuring IBM PowerKVM on Power Systems
11
Results Note: v If you create a non-persistent guest outside of Kimchi and then use Kimchi to stop it, the guest will be deleted. In Kimchi, the Stop option calls the virsh destroy command, which deletes non-persistent guests. To avoid this issue, either use the virsh shutdown guest command where guest is the name of your virtual machine or the Kimchi console to shut down the guest within the guest operating system. v If you are using a logical storage pool for storage, creating a guest disk with the same capacity as the remaining pool capacity may fail. Libvirt does not report the exact number of extents required for a disk. Consider creating a guest that uses 1G less than what Kimchi and libvirt report as available. If you have trouble starting a guest that has storage allocated in a logical storage pool, check your system or Kimchi log files for the error libvirtError: internal error: Child process (/usr/sbin/lvcreate --name some_image.img -L [X]K some_pool_name) unexpected exit status 5: Volume group "some_pool_name" has insufficient free space ([X] extents): [X] required.
Setting up a template using Kimchi You can use Kimchi to define templates for your guests.
About this task To view the current templates available, start Kimchi and select Templates. From this page, you can perform the following actions: v Select Edit to edit the template. v Select Delete to delete the template. You can also add a template by following these steps:
Procedure 1. On the Templates tab, click the plus (+) icon. 2. Select the location of the source media from the following options: Local ISO image Select to scan storage pools for installation ISO images available on the system. Remote ISO image Select to specify a remote location for an installation ISO image. If you selected Remote ISO image, the image is retrieved and loaded onto the system. If you selected Local ISO image, all ISO images available on the system are displayed. 3. To create a template from an ISO image, choose from the following options: v Select an ISO image from which to create a template, then click Create Templates from Selected ISO. v Select All to create a template from each listed ISO image, then click Create Templates from Selected ISO. If the ISO image that you want to use does not appear in the scan results, you can select from the following options: v Select I want to use a specific ISO file to specify a path to the ISO image. v Click Search more ISOs to search for more ISO images. 4. Click Create.
Creating a guest with the virsh command The virsh command is a common command-line utility that can be used to create KVM guests.
12
Linux: Configuring IBM PowerKVM on Power systems
Run the virsh command as a single command or as a shell program. Most virsh commands require root privileges, although unprivileged users can use the virsh command for read-only operations. The virsh command uses the libvirt API. PowerKVM includes the libvirt package, but you can ensure that the libvirt daemon is running before you run virsh commands. To check the libvirt daemon status, run the following command: # systemctl status libvirtd
If the service is running, output similar to this example is shown: libvirtd.service - Virtualization daemon Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled) Active: active (running) since Fri 2014-03-14 22:32:43 GMT; 4 days ago Main PID: 3517 (libvirtd)
If the output displays inactive, then start the libvirt daemon by running this command: # systemctl start libvirtd
You can use the virsh command to create a virtual machine by creating a libvirt XML file that describes the virtual machine and then running the virsh create command. To create a libvirt XML file, you can create the xml file or else export an existing one and edit it to the parameters that you want. You can find more information about libvirt at http://libvirt.org/formatdomain.html.
Creating guests with the virt-install command The virt-install command is used to create new KVM guests. The virt-install command is useful for when you do not have access to a graphical desktop and, when given the necessary options, can run unattended. If some of the required options are omitted, the virt-install command runs interactively, prompting you for input when required. You must run the virt-install command as root. For more information about the options that you can use with the virt-install command, see http://linux.die.net/man/1/virt-install In this example, create a guest with name test, using two virtual processors and 4 GB of RAM, connecting to default network, and using an ISO file as installation source. virt-install --machine=pseries --name=test --virt-type=kvm --boot cdrom,hd --network=default,model=virtio --disk path=/var/lib/libvirt/images/test.qcow2,size=10,format=qcow2,bus=virtio,cache=none --memory=4096 --vcpu=2 --cdrom=/var/lib/libvirt/images/RHEL6.5-20131111.0-Server-ppc64-DVD1.iso
Common virsh command options You can use the virsh command to manage several tasks for your KVM system.
Configuring IBM PowerKVM on Power Systems
13
Note: As a rule, do not run any virsh commands as a background process as timeouts and errors can occur at unpredictable times. Table 4. Common virsh command options. Command option
Description
virsh connect
Connect to the KVM hypervisor
virsh create xmlfile.xml
Creates and starts a guest from an XML configuration file.
virsh list --all
Lists all the guests on a host.
virsh dumpxml guest_name
Creates an XML configuration file of the guest as an output file.
virsh start guest_name
Starts an inactive guest.
virsh destroy guest_name
Immediately stops the guest.
virsh define xmlfile.xml
Creates a guest from an XML configuration file. The guest is not started.
virsh reboot guest_name
Restarts the guest
virsh restore fileName
Restores a guest from a saved file
virsh resume guest_name
Resumes a guest that was paused
virsh save guest_name fileName
Save the state of the guest to a file
virsh suspend guest_name
Pauses the guest
virsh undefine guest_name
Deletes the guest, but not the image file
virsh undefine guest_name --remove-all-storage
Deletes the guest and all the associated storage
virsh nodeinfo
Displays information about the host
virsh dominfo guest_name
Displays information about a guest
PowerKVM storage Storage for PowerKVM consists of storage volumes and storage pools. A storage pool is file, directory, or storage device that is available to the guests. Storage pools are divided into volumes that are then assigned to guests.
Common virsh command options - Storage You can use the virsh command to manage your storage in PowerKVM. Note: As a rule, do not run any virsh commands as a background process as timeouts and errors can occur at unpredictable times. Table 5. Common virsh command options: storage Command option
Description
virsh find-storage-pool-sources
Creates an XML definition file for all storage pools of a specific type.
virsh pool-define-as pool_name path mountpoint
Create a storage pool. Provide the name, the path to the storage, and a mount point on the local system.
virsh pool-list
Lists all storage pools. To include inactive storage pools, add --all.
virsh pool-build pool_name
Creates a mount point for the storage pool.
virsh pool-start pool_name
Starts the storage pool
14
Linux: Configuring IBM PowerKVM on Power systems
Table 5. Common virsh command options: storage (continued) Command option
Description
virsh pool-autostart pool_name
Causes the pool to be started every time that libvirt is started. To disable this option, run virsh pool-autostart pool_name --disable.
virsh pool-info pool_name
Displays information about the pool.
virsh vol-create-as pool_name vol_name size --format format_type virsh vol-list pool_name virsh vol-clone existing_vol_name new_vol_name --pool pool_name
Creates a volume. Specify the pool where the volume is located, the name of the volume, size of the image (in K, M, G, T), and the format of the volume. Lists the volumes in a pool. To include inactive volumes, add --details. Copies and creates a volume in a storage pool.
virsh vol-delete --pool pool_name vol_name
Deletes a volume from a storage pool.
virsh pool-destroy pool_name
Stops a pool.
virsh pool-delete pool_name
Deletes a pool directory from the host.
virsh pool-undefine pool_name
Removes the pool definition.
Find storage pool sources with virsh Use the virsh command to discover storage pool sources on your system. Discovery of storage pool sources is supported for the following pool types: v logical v netfs (requires NFS server IP address or hostname) v iscsi (requires iSCSI server details)
find-storage-pool-sources type [srcSpec] Returns XML describing all storage pools of a given type that could be found. For netfs and iscsi types, provide an XML containing the required details (srcSpec). In this example, create a query to find NFS storage pool sources on server 192.168.122.3. Create an XML similar to the following:
Provide the XML as an argument in the command: virsh find-storage-pool-sources netfs nfs.xml
This command returns output similar to the following example: Configuring IBM PowerKVM on Power Systems
15
find-storage-pool-sources-as type [host] [port] [initiator] Returns XML describing all storage pools of a given type that could be found. For netfs and iscsi type, provide the host or other details as appropriate In this example, create a query to find NFS storage pool sources on server 192.168.122.3. virsh find-storage-pool-sources-as netfs 192.168.122.3
This command returns output similar to the following example:
Valid storage pools types You can create and use several types of storage pools with PowerKVM. Directory pool (DIR) Specifies a directory pool. When you use a directory pool, the file must exist and you must provide the path to the volume within that pool. Specify directory in the XML format looks similar to this format: kvmstorageimage /var/lib/virt/images
Network file system pool (NFS) Specifies an NFS file system pool as a type of exported storage. When you specify NFS, the file system is mounted and files are managed from its mount point. To use NFS file system pool, you need the host name and path of the exported directory. PowerKVM attempts to mount the network file system. Specify NFS in the XML format looks similar to this format: kvmstorageimage /var/lib/virt/images ;
16
Linux: Configuring IBM PowerKVM on Power systems
iSCSI server (iSCSI) Specifies a pool that is based on a target that is allocated on an iSCSI server. A iSCSI volume pool must exist.. Consider configuring the pool to use /dev/disk/by-path or /dev/disk/by-id for the target path. These provide persistent stable naming for LUNs. You can also add iSCSI authentication. The iSCSI authentication uses CHAP protocol. Specify the user name and password when you define the pool. Specify iscsi in the XML format looks similar to this format: kvmstorageimage /var/lib/virt/images
Logical volume storage pool (Logical) Specifies a logical volume (LVM) storage pool. For an existing LVM group, use the group name to identify it. If you are creating a new LVM group, specify the source devices to use. You must also provide a path to the LVM pool. Specifying logical in the XML format looks similar to this format: VolGroup /dev/VolGroup
SCSI Fibre Channel Specifies a pool that is based on a SCSI Fibre Channel. The SCSI Fibre Channel volume pool must exist. Consider configure the pool to use /dev/disk/by-path or /dev/disk/by-id for the target path. These provide persistent stable naming for LUNs. Specifying SCSI in the XML format looks similar to this format: kvmstorageimage /dev/disk/by-path
Storage pool Create a storage pool with the Kimchi interface or the virsh command.
Setting up a storage pool with Kimchi You can use Kimchi to define storage pools for your guests.
Configuring IBM PowerKVM on Power Systems
17
About this task To view the current storage pools available, start Kimchi and select Storage. From this page, you can perform the following actions: v v v v
Select Select Select Select
Activate to activate the storage pool so that it can be used. Deactivate to deactivate an active storage pool. Undefine to remove an inactive storage pool. the arrow at the end of a row to view details about a storage pool
You can also define a storage pool by following these steps:
Procedure 1. On the Storage tab, click the plus (+) icon. 2. In the Storage pool name field, type the name to be used to identify the storage pool. 3. In the Storage pool type list, select the type. You can choose: DIR
Specifies a directory pool. When you select DIR, type the Storage path (file path to the storage pool).
NFS
Specifies a network file system pool. When you select NFS, type the NFS server IP address and NFS path (path of the exported directory).
iSCSI Specifies a pool that is based on a target that is allocated on an iSCSI server. When you select iSCSI, type the iSCSI server IP address and Target on the iSCSI server. You can optionally select to add iSCSI authentication. Logical Specifies a logical volume storage pool. Select the location to the device in Device path. Note: You must create the storage pool using a partition. At this time, you cannot create a logical pool from an existing volume using Kimchi. 4. Click Create.
Creating storage pool with virsh Create a storage pool with the virsh command and a temporary XML configuration file
About this task To create a storage pool by defining a temporary XML configuration file and then using the virsh command to add it, follow these steps:
Procedure 1. Create an XML file for the storage device. In this example, define an iSCSI device with authentication. storage_image /dev/disk/by-path
2. Add the XML file to the storage definition:
18
Linux: Configuring IBM PowerKVM on Power systems
virsh pool-define ~/storage_image.xml
3. Verify that the pool was created: virsh pool-list -all Name State Autostart ----------------------------------------default active yes storage_images inactive no
4. Start the storage pool: virsh pool-start storage_images Pool storage-images started
5. Verify that the pool started: virsh pool-list --all Pool storage-images started Name State Autostart ----------------------------------------default active yes storage_images active no
6. Optional: Turn on autostart for the storage pool: virsh pool-autostart storage_images Pool storage-images marked as autostarted.
7. Verify that the storage pool was created correctly and is running: virsh pool-info storage_images Name: UUID: State: Persistent: Autostart: Capacity: Allocation: Available:
storage_images afcc5367-6770-e151-bcb3-847bc76c5e28 running unknown yes 115.53 GB 0.00 115.53 GB
Networking on PowerKVM Networking in a virtualized environment can be seen as a number of independent physical and virtual network interfaces. By default, a KVM installation provides two network communication bridges. The first, virbr0, is a NAT virtual network. This bridge allows communication to an external system with a host IP address, to other guests, and to the host. The second, virbr1, is a bridge for local host network connectivity only. Note: If you manually create a network interface using the name virbr0, attempts by Kimchi to create a default network will result in an error and libvirt will crash. To fix this error, remove virbr0 and Kimchi can then create the default network. If you want to manually create a different default network, use a bridge other than virtbr0. You can also create a network bridge. Using a bridge, guests can communicate with external systems and be contacted by external systems as if they were physical systems on the network. This type of access requires a physical network connection. To increase the security of the host, plan to configure one network interface for the host and a separate network interface for the guest operating systems.
Configuring IBM PowerKVM on Power Systems
19
In this configuration, the network traffic for the host travels on a different subnet than the network traffic for the guest operating systems. This configuration increases security in the following ways: v Helps isolate the host from the guest operating systems. v Helps prevent malicious users with a lower security clearance from breaching a guest operating system and attacking the host or other guest operating systems.
Common virsh command options - Networking You can use the virsh command to perform several tasks that are related to networking. Note: As a rule, do not run any virsh commands as a background process as timeouts and errors can occur at unpredictable times. Table 6. Common virsh command options: Network Command option
Description
virsh net-info network_name
Returns information about a network
virsh net-list
Returns a list of active networks. Can also add --all to return all defined network, including inactive ones.
virsh net-start network_name
Starts a defined but inactive network.
virsh net-create xml_file
Creates a virtual network from an XML file
virsh net-autostart network_name
Configures a network to automatically start.
virsh net-define xml_file
Defines a network from an XML file. The network is not started.
virsh net-dumpxml network_name
Creates an xml file of the virtual network information of a guest.
20
Linux: Configuring IBM PowerKVM on Power systems
Table 6. Common virsh command options: Network (continued) Command option
Description
virsh net-edit network_name
Opens the xml configuration file for editing.
virsh net-destroy network_name
Stops a network.
virsh net-undefine network_name
Undefines an inactive network.
Setting up a network connection with Kimchi You can use Kimchi to define network connections for your guests.
About this task To view the current network connections available, open Kimchi and select Network. From this page, you can perform the following actions: v Select Start to begin the network connection. v Select Stop to end the network connection. v Select Delete to delete the connection information. You can also create a network connection by following these steps:
Procedure 1. From the Network window, click the plus (+) icon. 2. Enter the name of the network 3. Select the network type. You can choose: Isolated Isolated mode. Guests cannot send or receive communication to or from external systems. NAT
Network Address Translation mode. Communication from guests to external systems uses the host IP address. External systems cannot initiate communication to guests.
Bridged Bridged mode. Guests can communicate with external systems and be contacted by external systems as if they were physical systems on the network. For bridged mode, you must specify additional destination and VLAN information. 4. Click Create.
Creating a network bridge with the ibm-configure-system utility You can set up a network bridge with the ibm-configure-system utility.
About this task The ibm-configure-system utility is included in the PowerKVM package. To create a network bridge with the ibm-configure-system utility, follow these steps:
Procedure 1. Connect to your system with IPMI. 2. Run ibm-configure-system. 3. Select Configure Network. Note: Navigate the menu with the up and down arrow keys or TAB.
Configuring IBM PowerKVM on Power Systems
21
4. On the Configure network window, select the device that you want to configure and select OK and press the Space bar. 5. On the Network Device Configuration window, verify the device configuration. Select Save and press Enter.
Verifying the default virtual network The default virtual network is a NAT virtual network that allows communication between guests, to the host, and to external systems with a host IP address.
About this task To verify and use this network, follow these steps:
Procedure 1. Verify that the network by running the virsh net-list command: For example, run the following command: virsh net-list
Your output is similar to this example: Name State Autostart -----------------------------------------default active yes
If the default network is not active, start it by running the virsh net-start command. For example, run the following command: virsh net-start default Network default started
2. To use the default network, select NAT as your network type in Kimchi or use the following example in your XML configuration file:
Each KVM guest needs a unique MAC address. Although defining a MAC address is optional, if you do not define one, one is automatically generated. For easier reference and consistency, consider defining a MAC address with your default network, as shown in the following example:
Configuring KVM guests to use bridge After you configure your bridge, attach virtual machines to the bridge and share the physical network port by editing any configured guests to use the configured bridge interface as a source bridge.
Before you begin Before you start, ensure that each guest operating system has an IP address or fully qualified domain name.
About this task To configure KVM guests to use a bridge, complete the following steps:
Procedure 1. Edit your guest domain definition with the virsh edit command:
22
Linux: Configuring IBM PowerKVM on Power systems
# virsh edit guest01
Note: Each KVM guest needs a unique MAC address. Otherwise, it might cause traffic disruptions to all hosts in the same subnet. To generate a new, random MAC address for a guest network interface, edit the guest XML definition, removing the mac tag for that network interface. The following is the original network configuration: # virsh dumpxml guest01
2. Restart the guest.
Adding guest para-virtualized network devices with libvirt Your guest network interfaces can be para-virtualized network devices. A para-virtualized network device improves throughput and latency.
About this task To enable the guest to use a para-virtualized network device, complete the following steps:
Procedure 1. Shut down the guest operating system. 2. Edit the guest XML definition with the virsh edit command. For example: virsh edit guest-name
3. Change the network definition to use the virtio model type. For example: ... ...
4. Start the guest operating system.
Using vhost-net for high-bandwidth applications You can use an in-kernel guest networking performance enhancement called vhost-net. vhost-net moves network packets between the guest and the host system using the Linux kernel rather than QEMU. This
Configuring IBM PowerKVM on Power Systems
23
action avoids context switches from the kernel to user space to improve overall performance. Use of vhost-net might also improve network performance and latency.
About this task To use of vhost-net support, add vhost=on to the -netdev option. For example: -net nic,model=virtio,macaddr=00:16:3e:00:01:01,netdev=nic-0 -netdev tap,id=nic-0,script=/root/ifup-br0,downscript=/root/ifdown-br0,vhost=on
Hot-plugging a network connection Hot-plugging a network device adds a network interface without stopping and starting your guest.
Before you begin v Set vhost to off for virtio network devices v Hot-plug of emulated rtl8139 and e1000 interfaces is not supported for Red Hat Enterprise Linux 6.5 guests v Ensure that you have the latest versions of the following packages: – powerpc-utils – librtas – ppc64-diag This applies to all guests, including Red Hat Enterprise Linux, Ubuntu and SUSE Linux Enterprise Server guests. Check the Service and productivity tools site for the latest version of the IBM Linux on Power Tools Repository. To hot plug a network interface, follow these steps:
Procedure 1. Create an xml file with the definition of your network interface, similar to this example. For example, create a file called hot_net.xml:
2. Hot plug the interface to the guest with the virsh command. For example, run the following command: virsh attach-device guest hot_net.xml
You can also run the virsh command with the details contained in the xml file: virsh attach-interface guest --type bridge --source virbr0 --mac 00:16:3e:1b:f7:47 --live
What to do next Before unplugging a network device from a guest, be sure to stop or unmount any filesystem processes that are using the device.
VLAN segmentation A virtual LAN (VLAN) is a method for segregating network traffic within a bridged LAN infrastructure. VLANs were designed specifically for physical environments to allow two logically separated networks to use the same physical medium. VLAN segmentation, also called 802.1q tagging, works by appending a tag identifying the VLAN ID to each Ethernet frame.
24
Linux: Configuring IBM PowerKVM on Power systems
Using VLAN in the KVM virtualization environment extends bridge interface. In the standard mode of operation, the physical interfaces, such as eth0 or eth1, are bound to the bridge, which is then used by each guest. These interfaces carry unmodified packets with or without a VLAN ID tag. You can filter out packages that do not carry a particular VLAN ID by creating subinterfaces. These subinterfaces become part of the VLAN and are defined by a specific VLAN ID. In enterprise environments, most of the existing network infrastructure supports 802.1q VLANs. Its use in the KVM virtualization environment is a good compromise for the following reasons: v Low setup cost in the KVM host – no need for firewall or routing between guests v Use of existing network access controls, methods, and processes – after the KVM Host administrator assigns a virtual machine guest to a VLAN, access control can be made external through the traditional network administrator role
Configuring VLAN segmentation Complete the steps to configure VLAN segmentation for KVM guests.
Before you begin Before you start, perform the following tasks: v Ensure that each guest operating system has an IP address or fully qualified domain name. v Ensure that the host and guest operating systems are connected to a VLAN-capable network switch and infrastructure.
Procedure 1. Identify the VLAN IDs that to assign to each guest. 2. Explicitly configure the external network infrastructure to allow traffic from those VLANs to the KVM host: a. Configure the network switch connected to the KVM Host. b. Qualify the physical port on the host as a trunk (carries multiple VLANs) and a tagged (accepts tagged frames) port. c. Allow traffic to necessary VLAN IDs. Note: If all of the guests resides in single host, then step b and step c are not required. 3. Create the virtual bridge in the KVM Host. Avoid mixing different VLANs in a single bridge. 4. Create a file named ifcg- in the /etc/sysconfig/network-scripts directory to create a permanent bridge configuration, where is the bridge interface name. The following example specifies a br_v19 bridge with a file named /etc/sysconfig/network-scripts/ifcfg-br_v19: DEVICE=br_v19 TYPE=Bridge BOOTPROTO=static STP=yes ONBOOT=yes DELAY=0
Note: The ONBOOT=yes line assures that the bridge will be available automatically after each boot. No IP address is to this bridge and Spanning Tree Protocol (STP) is enabled (STP=yes). 5. If there are multiple guests participating in the same VLAN ID (even if they use separate bridges), disable Netfilter processing in bridging devices by appending the following lines to the /etc/sysctl.conf file: net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-arptables = 0
6. Reload the kernel parameters with the sysctl command: Configuring IBM PowerKVM on Power Systems
25
# sysctl -p net.ipv4.ip_forward = 0 . . . net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-arptables = 0
7. Configure one or more subinterfaces from the main, physical network interface (the trunk). The following example configures the subinterface eth0.19 that is assigned to VLAN ID 19. The file name is /etc/sysconfig/network-scripts/ifcfg-eth0.19. #VLAN 19 in trunk eth0 DEVICE=eth0.19 VLAN=yes ONBOOT=yes BRIDGE=br_v19
Note: The value of DEVICE (that is, the subinterface name), is in the form of ., where interface-name is the name of the physical interface that this virtual bridge attaches to (eth0 in this example) while the VLAN-ID is directly taken from the subinterface name (19 in this example). In this example, the bridge strips the VLAN tags from ingress traffic and assign tags to egress packets. Note: Stripping the VLAN tags is optional. As ONBOOT is set to yes, the subinterface of the virtual bridge opens automatically on every reboot, but you can also bring them up manually by using the ifup command: # ifup br_v19 # ifup eth0.19 # brctl show bridge name bridge id STP enabled interfaces br0 8000.00145ed87f4a yes eth0 virbr0 8000.000000000000 yes br_v19 8000.00145ed87f4a yes eth0.19
This example also shows the use of the brctl command to list all of the virtual bridges and their assigned interfaces. 8. With the bridge interface running, adjust each guest configuration, assigning interfaces to their respective bridge or VLAN (as described in “Configuring KVM guests to use bridge” on page 22).
Or you can create a libvirt network and use that for guest' interface: br_v19
9. Restart the modified guests for changes to take effect. 10. Assign a separate IP address to the guest operating system for its network connection to work.
26
Linux: Configuring IBM PowerKVM on Power systems
Configuring 802.1q VLANs in Kimchi You can also configure 802.1q VLANs with the Kimchi interface.
Procedure 1. 2. 3. 4.
Log into Kimchi interface. Select Network window. Create a network by selecting the green plus (+). Enter a name for your network.
5. Select Bridged as type. 6. Select Enable VLAN and enter a VLAN ID. 7. Click Create. 8. From the action menu drop down box for the new network bridge, select Start. You can verify the network interface by running the following commands: # ifconfig enP3p5s0f0.42 enP3p5s0f0.42: flags=4163 mtu 1500 inet6 fe80::6eae:8bff:fe69:18b4 prefixlen 64 scopeid 0x20 ether 6c:ae:8b:69:18:b4 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 12 bytes 936 (936.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 # brctl show bridge name bridge id STP enabled interfaces brenP1p3s0f0 8000.6cae8b026c1c no enP1p3s0f0 dovebr_11 8000.422c7f7babf9 no dove_11 dovebr_22 8000.6a29151cfc72 no dove_22 kbP3p5s0f0-42 8000.6cae8b6918b4 no enP3p5s0f0.42 virbr0 8000.fe5400f1d886 yes vnet2 virbr1 8000.525400be8b88 yes virbr1-nic # ifconfig kbP3p5s0f0-42 kbP3p5s0f0-42: flags=4163 mtu 1500 inet6 fe80::6eae:8bff:fe69:18b4 prefixlen 64 scopeid 0x20 ether 6c:ae:8b:69:18:b4 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 6 bytes 468 (468.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
9. Assign the network to your guest by editing the guest xml configuration file:
10. Restart the guest.
PowerKVM and Quality of Service Quality of Service (QoS) is an internet standard that provides ways of giving preferential treatment to certain types of IP traffic. PowerKVM supports QoS using the bandwidth element for networks that use forwarding such as NAT. QoS is not supported for network bridges. The bandwidth element uses inbound and outbound child elements to allow incoming and outgoing traffic to be set independently of each other. You can have only one inbound and one outbound child
Configuring IBM PowerKVM on Power Systems
27
element for each network device. If you do not set one of these, then that traffic direction will not use QoS. For example, if your bandwidth element specifies values for incoming traffic only, then outgoing traffic will not use QoS. When setting bandwidth elements, you must specify the average speed. In addition, you can specify a peak and a burst. average Specifies the desired average bit rate for the interface being shaped in kilobytes/second. peak
(Optional) Specifies the maximum rate at which the interface can send data in kilobytes/second.
burst
(Optional) Specifies the amount of kilobytes that can be transmitted in a single burst at peak speed in integer values.
To set QoS in the network XML, use elements similar to the follow example:
To set QoS in the domain XML, use elements similar to this example:
For more information about network trafficking and QoS, see the following websites: v libvirt networking XML format: QoS v Linux Advanced Routing & Traffic Control
Remote management of PowerKVM with libvirt By default, you cannot manage PowerKVM guests remotely. This default condition exists because the libvirtd daemon does not create any listening sockets, and the qemu-kvm virtual network console (VNC) accepts connections only from the local host. You do have, however, several choices for secure remote management configuration.
Overview of PowerKVM remote management Packages providing virtualization support include the libvirtd daemon and the management clients of the libvirtd daemon, including virsh and kimchi. IBM PowerKVM includes the following packages: qemu, qemu-img The user space process that performs machine virtualization, namely qemu, or qemu-kvm. The qemu-img utility is used to create virtual machine disk images. libvirt, python-virtinst The user space management layer, including the following utilities: v libvirtd daemon: A system service responsible for coordinating virtual machine management
28
Linux: Configuring IBM PowerKVM on Power systems
v virsh: A full-featured, command-line virtual machine management utility that connects to the libvirtd daemon v virt-install: A command-line utility that facilitates virtual machine installations kimchi A graphical user interface to the host and guests. virsh
A command-line tool for managing your host and guests.
The libvirtd daemon is responsible for managing all things that are related to virtualization. It stores virtual machine guest configuration, creates the user space qemu-kvm processes that are responsible for machine emulation, and assigns resources such as virtual networks, disk images, or pass-through devices to the guests. Except for the VNC console, which is provided by the qemu-kvm process directly, you can perform all management of the guests through a libvirtd daemon client such as virsh. This client can be initiated remotely through various methods, including local UNIX domain sockets, Transmission Control Protocol (TCP) sockets, Secure Shell (SSH) tunnels, or secure Transport Layer Security (TLS) connections. Authentication can also be handled with Simple Authentication And Security Layer (SASL), which is a library and a protocol that is designed to add pluggable authentication support for connection-based services. Using the VNC console, management can be partially performed outside of the libvirtd daemon control because VNC clients connect directly to the QEMU process. To allow incoming connections through the PowerKVM host firewall, you are asked to open the following ports: v SSH: 22 v SASL: 16509 v TLS: 16514 v VNC connections: 5900 + VNC screen number v Kimchi: 8001 Figure 1 offers a simplified view of the architecture of a PowerKVM virtualization host. It illustrates how each of the components described connect to each other and their external connection points.
Configuring IBM PowerKVM on Power Systems
29
Figure 1. The architecture of a PowerKVM virtualization host
Remote management with SSH tunnels You can securely and directly remotely manage the libvirtd daemon plus qemu-kvm pair by using standard Secure Shell (SSH) sessions as tunnels for communicating libvirt management data. The libvirt API can perform management through an SSH tunnel, so most management software built over libvirt is also capable of using this feature. If the root user, or any other user that has permission to manage virtual machine guests, is allowed to log on to the PowerKVM host with a standard SSH session, you do not need to do any additional setup. The libvirtd daemon recognizes connections through an SSH tunnel as local. Through the SSH tunnel, you can remotely manage PowerKVM guests using the virsh command, or access your PowerKVM guest graphical consoles using the virtual network console (VNC) viewer of your choice.
Managing PowerKVM guests remotely with the virsh command You can manage your remote PowerKVM hosts by connecting to their local libvirtd daemon instance with the Secure Shell (SSH) method and running virsh commands. You must already be able to connect with SSH from your management station to the remote PowerKVM host. To connect to a remote PowerKVM host with an SSH tunnel, pass the -c flag followed by a valid Uniform Resource Identifier (URI) to the virsh command. The following command connects to the libvirtd daemon that is running on kvmhost.company.org and lists the running KVM guests:
30
Linux: Configuring IBM PowerKVM on Power systems
# virsh -c qemu+ssh://
[email protected]/system list
[email protected]’s password: Id Name State ---------------------------------1 guest01 running
The URI is similar to a common HTTP URL. The communication protocol is specified as qemu+ssh, and the host to connect is kvmhost.company.org. For more information about the libvirt URI format, see the libvirt online documentation at http://libvirt.org/uri.html. Note: In these instructions, kvmhost.company.org is used as the domain name of the PowerKVM host. Replace this domain name with the domain name of your PowerKVM host when you are using these instructions in your environment. Make sure that the PowerKVM host SSH daemon is accessible from your management station. You can run other virsh subcommands to manage your remote PowerKVM guests by including the -c flag followed by a valid URI on any virsh command invocation. However, -c and its argument are an option on the virsh command, as opposed to virsh subcommands like list, create, or reboot. Thus, order is important: the -c flag must come before any virsh subcommand.
Displaying the remote PowerKVM VNC console with any VNC client You can tunnel the virtual network console (VNC) connection manually using a standard Secure Shell (SSH) session.
About this task To tunnel the VNC connection manually, complete the following steps:
Procedure 1. Use the virsh command to query the VNC graphical console screen number for the guest you would like to connect to using the following command: # virsh -c qemu+ssh://
[email protected]/system vncdisplay guest01
[email protected]’s password: :1
2. Open an SSH session, forwarding the remote VNC port to a local port. To find out which VNC port is being used by your remote PowerKVM guest, add 5900, which the known base VNC port number, to the screen number from the previous step. In this example, the remote VNC port is 5901. The choice of local port is arbitrary. In this example, assume port 5910 (VNC port for screen number 10) is free in the management station. The following command forwards port 5901 of kvmhost.company.org to port 5910 of the host (or management station) that you are using: # ssh -L 5910:localhost:5901
[email protected] [email protected]’s password: Last login: Tue Apr 6 15:28:14 2010 from otherclient.company.org
3. At another terminal, use your VNC client of choice to connect to the local port 5910, to which the PowerKVM host 5901 port is forwarded. SSH forwards the port and recognizes the remote port as if it were a local connection attempt: # vncviewer localhost:10
Results If this configuration is successful, a VNC session of your remote PowerKVM guest graphical console opens.
Configuring IBM PowerKVM on Power Systems
31
Remote management with SASL authentication and encryption Simple Authentication and Security Layer (SASL) provides secure authentication and data encryption, but allows integration with traditional or external authentication and authorization services.
About this task In its simplest form, SASL can be used to define a database of credentials for authorization. In more complex scenarios, it can work with external authentication services such as Kerberos or Lightweight Directory Access Protocol (LDAP) to authenticate users. In both scenarios, the libvirtd daemon provides confidentiality by requiring Generic Security Services application programming interface (GSSAPI) as the SASL method if remote management requests are not running on top of a secured Transport Layer Security (TLS) connection. The libvirtd daemon can use DIGEST-MD5 instead of GSSAPI as the SASL method. However, MD5 hashes are considered unsafe and should not be used. These variations of SASL enable encryption of the data that is being pushed through. For simplicity, this example uses DIGEST-MD5 as the SASL method. To configure remote management with SASL in the simplest scenario (no external authentication or TLS security), complete the following steps:
Procedure 1. Log in to the KVM host. 2. Save a copy of the /etc/libvirt/libvirtd.conf file and the /etc/sysconfig/libvirtd file. 3. Edit the /etc/libvirt/libvirtd.conf file and make the following changes: a. Disable the listen_tls configuration directive by setting it to 0 because no TLS certificates are configured. Otherwise, the libvirtd daemon fails to start. b. Ensure that the configuration directive listen_tcp is enabled by setting it to 1. c. Set the auth_tcp configuration directive to sasl to enable SASL authentication over TCP. The following example shows these parameters in contrast with the stock libvirtd.conf file, with changes highlighted for deletions (-) and additions (+): --- libvirtd.conf.orig 2012-01-04 11:28:32.000000000 -0600 +++ libvirtd.conf 2012-01-04 11:34:02.000000000 -0600 @@ -19,7 +19,7 @@ # using this capability. # # This is enabled by default, uncomment this to disable it -#listen_tls = 0 +listen_tls = 0 # Listen for unencrypted TCP connections on the public TCP/IP port. # NB, must pass the --listen flag to the libvirtd daemon process for this to @@ -30,7 +30,7 @@ # DIGEST_MD5 and GSSAPI (Kerberos5) # # This is disabled by default, uncomment this to enable it. -#listen_tcp = 1 +listen_tcp = 1 @@ -143,7 +143,7 @@ # Don’t do this outside of a dev/test scenario. For real world # use, always enable SASL and use the GSSAPI or DIGEST-MD5 # mechanism in /etc/sasl2/libvirt.conf -#auth_tcp = "sasl" +auth_tcp = "sasl" # Change the authentication scheme for TLS sockets. #
4. Edit the /etc/sysconfig/libvirtd file and enable the --listen parameter so that the libvirtd daemon listens to TCP/IP connections:
32
Linux: Configuring IBM PowerKVM on Power systems
--- libvirtd.orig 2012-01-04 11:41:37.000000000 -0600 +++ libvirtd 2012-01-04 11:31:33.000000000 -0600 @@ -3,7 +3,7 @@ # Listen for TCP/IP connections # NB. must setup TLS/SSL keys prior to using this -#LIBVIRTD_ARGS="--listen" +LIBVIRTD_ARGS="--listen" # Override Kerberos service keytab for SASL/GSSAPI #KRB5_KTNAME=/etc/libvirt/krb5.tab
5. Restart the libvirtd daemon for the changes to take effect: # /etc/init.d/libvirtd restart Stopping libvirtd daemon: Starting libvirtd daemon:
[ OK [ OK
] ]
6. Now that the libvirtd daemon is accepting TCP connections, add some users to the SASL database. The following example uses the saslpasswd2 command to add the admin user to the libvirt credential database. # saslpasswd2 -a libvirt admin Password: Again (for verification):
Note: v The expected name of the SASL database for the libvirtd daemon authentication domain is libvirt. Do not use any other name for this database. v Keep these credentials safe as every user in this database is authorized to log on and run remote virtual machine administration. 7. If the KVM host is running a firewall, ensure that it allows incoming traffic through the libvirtd daemon TCP listening port. By default, the listening port is 16509. 8. Verify that the setup was successful by using SASL authentication to instruct libvirt enabled applications to connect with the TCP transport. The following example runs the virsh command from a remote management station and logs in as the newly created user admin to start the guest02 instance in the kvmhost.company.org system: # virsh -c qemu+tcp://kvmhost.company.org/system start guest02 Please enter your authentication name:admin Please enter your password: Domain guest02 started
Remote management with TLS TLS (Transport Layer Security) connections are made secure by digital signature verification when peers exchange certificates that were previously signed by a recognized certificate authority (CA). In the most common scenario, the client application is configured to trust a certain list of CAs. Each server must prove its identity to the client by presenting a certificate that is signed by those trusted CAs with a matching Subject Name, usually the fully qualified domain name (FQDN) of the server. One example of this scenario is when web browsers connect to web servers. In other less common scenarios that need improved security, the server also requires the client to present a digitally signed certificate to prove its identity. The example that is given in this section demonstrates how to create a local CA, use the local CA to digitally sign server and client certificates, and distribute the certificates for use. The openssl command is used to create private keys and certificates that can be used directly by libvirt. For more information about using the openssl command, run the man openssl command, or see http://www.openssl.org/docs/apps/openssl.html. Configuring IBM PowerKVM on Power Systems
33
Step 1. Creating a CA key and certificate in your PowerKVM host You can create a certificate authority (CA) key and certificate in your PowerKVM host. All certificates that are created are signed by a 2048-bit remote supervisor adapter (RSA) key and a sha256 hash algorithm.
About this task Complete the following steps:
Procedure 1. Log in to your PowerKVM host. 2. Create a temporary directory to keep the files, and change into it: # mkdir cert_files # cd cert_files
3. Using the openssl command, create a 2048-bit RSA key: openssl genrsa -out cakey.pem 2048
4. Use the key to create a self-signed certificate to your local CA: openssl req -new -x509 -days 1095 -key cakey.pem -out cacert.pem -sha256 \ -subj "/C=US/L=Austin/O=IBM/CN=my CA"
5. Check your CA certificate: # openssl x509 -noout -text -in cacert.pem Certificate: Data: Version: 3 (0x2) Serial Number: b8:fd:55:0d:39:d8:48:fc Signature Algorithm: sha256WithRSAEncryption Issuer: C=US, L=Austin, O=IBM, CN=my CA Validity Not Before: Jan 9 02:11:58 2012 GMT Not After : Jan 8 02:11:58 2015 GMT Subject: C=US, L=Austin, O=IBM, CN=my CA Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:9d:df:d0:3e:e0:ec:55:8e:e0:e5:b2:44:da:94: a5:d7:28:4d:18:2f:ea:eb:f1:a2:e3:fa:ee:a5:7c: b0:cb:db:6f:9c:4f:8a:14:ff:19:51:c3:8b:f3:ac: ce:8a:d2:c4:1d:16:03:b5:6d:94:32:8b:25:b9:75: 6a:d5:eb:f9:9f:72:68:af:02:03:47:1d:e8:07:87: fa:1a:64:d8:84:ee:62:fb:41:bb:5d:25:c7:67:8c: ad:89:89:bf:3e:bf:b3:4c:42:27:4e:44:68:cf:48: 23:6e:f3:8d:3b:62:a1:a6:e5:d2:a0:db:8b:4d:0e: 0b:5f:02:aa:52:06:49:ec:7f:ea:cd:00:6d:d2:eb: 0e:71:b3:70:98:e1:c1:81:7e:d5:1d:e4:7d:d7:e2: 22:79:24:3f:d1:0b:46:56:29:ce:ee:49:82:84:74: a8:b6:da:e0:11:35:57:46:f2:cb:11:0a:4c:c7:51: 11:be:c0:d4:73:99:fe:d2:22:e1:f4:e2:53:4a:c4: da:b0:62:b4:3d:ad:a7:63:7f:72:f2:f6:6f:b8:cc: fc:59:c5:95:50:4e:58:01:42:6f:0b:a8:ad:09:74: b6:75:e9:d1:bc:3d:76:2c:8f:7f:a9:a8:3b:b7:38: 39:6b:b1:67:ca:b8:fc:76:c1:d2:c9:2e:be:70:8d: d3:71 Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Subject Key Identifier: 0E:B0:0D:46:DF:6D:1E:05:5A:27:AF:58:AA:BC:57:57:C3:9C:83:00 X509v3 Authority Key Identifier: keyid:0E:B0:0D:46:DF:6D:1E:05:5A:27:AF:58:AA:BC:57:57:C3:9C:83:00
34
Linux: Configuring IBM PowerKVM on Power systems
X509v3 Basic Constraints: CA:TRUE Signature Algorithm: sha256WithRSAEncryption 5a:9f:28:9f:72:b1:46:f4:c7:e8:cd:c6:d4:55:80:b0:55:2b: 04:dc:b3:2b:7b:46:da:0b:55:88:34:c3:3a:90:1f:e7:b9:0b: c3:f2:82:48:eb:4e:69:da:c3:ac:df:36:18:67:4d:88:85:1c: c0:d1:bd:60:d5:8a:a6:66:d3:c7:5e:d8:ba:e5:9b:cd:37:2a: 7b:e5:a0:ed:77:72:f8:4f:15:0b:3f:47:5e:cc:e0:a2:5d:71: bd:f0:f6:e0:7b:1b:19:93:cb:84:6b:6f:75:f0:ef:3b:c7:c8: ac:d4:e2:e8:9d:a5:21:1c:c3:89:22:00:db:94:fb:0e:00:4a: 87:18:c0:dd:11:1b:a6:91:b5:90:24:24:6e:5e:a1:b6:94:31: 3e:40:b7:de:34:62:e0:a1:a3:e2:1c:c0:3c:2d:a6:3a:3f:60: 75:15:0c:51:cc:19:3e:64:37:62:cc:1f:5f:08:8b:89:ec:f0: a5:a7:7c:1d:4b:da:88:8b:f8:c0:a7:a5:9b:c7:7f:d2:a4:58: b9:63:92:c4:14:22:3f:6b:8d:d4:28:88:0a:b3:e6:60:c6:ca: 8b:eb:3a:af:a2:0d:fe:7a:7b:2e:95:dc:ef:1c:f0:9d:61:55: 4d:a3:65:5a:aa:40:b0:06:b9:c5:2e:95:84:cc:ef:52:d3:0d: 81:6b:78:ec
Step 2. Creating the client and server keys and certificates in your PowerKVM host You can create the client and server keys and certificates in your PowerKVM host.
About this task Note: To create both client and server certificates, you must create a certificate signing request. A certificate signing request (CSR) is a message that is used to apply for a certificate to a certificate authority (CA). It usually contains identification information (the certificate subject) for the certificate owner (for example, country, organization, country). It is signed by the applicant, with its private key. The format that is used by the certificate signing request that is generated by openssl is described by the PKCS#10 standard. Complete the following steps:
Procedure 1. Create the keys: # openssl genrsa -out serverkey.pem 2048 # openssl genrsa -out clientkey.pem 2048
2. Create a certificate signing request for the server. Remember to change the kvmhost.company.org address, which is used in the server certificate request, to the fully qualified domain name of your PowerKVM host: # openssl req -new -key serverkey.pem -out serverkey.csr \ -subj "/C=US/O=IBM/CN=kvmhost.company.org"
3. Create a certificate signing request for the client: # openssl req -new -key clientkey.pem -out clientkey.csr \ -subj "/C=US/O=IBM/OU=virtualization/CN=root"
4. Create client and server certificates: # openssl x509 -req -days 365 -in clientkey.csr -CA cacert.pem -CAkey cakey.pem \ -set_serial 1 -out clientcert.pem # openssl x509 -req -days 365 -in serverkey.csr -CA cacert.pem -CAkey cakey.pem \ -set_serial 94345 -out servercert.pem
5. Check the keys: # openssl rsa -noout -text -in clientkey.pem # openssl rsa -noout -text -in serverkey.pem
6. Check the certificates:
Configuring IBM PowerKVM on Power Systems
35
# openssl x509 -noout -text -in clientcert.pem Certificate: Data: Version: 1 (0x0) Serial Number: 9434242 (0x8ff482) Signature Algorithm: sha1WithRSAEncryption Issuer: C=US, L=Austin, O=IBM, CN=my CA Validity Not Before: Jan 10 19:44:06 2012 GMT Not After : Jan 9 19:44:06 2015 GMT Subject: C=US, O=IBM, OU=virtualization, CN=root Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:c1:ef:30:8e:b3:73:3a:d6:72:a3:c5:44:1f:a2: 63:23:20:2b:b9:34:04:2a:1c:12:18:8e:e5:87:ec: ff:28:ec:b1:62:e6:5e:ec:bb:67:cd:e9:18:68:c5: 51:f6:f6:fa:83:d0:0c:74:bd:72:f2:ac:a5:35:ce: 8c:84:1e:dc:a2:3d:bb:90:32:a8:14:48:2b:57:ae: d5:91:14:5e:92:ad:85:78:92:35:81:02:d0:73:9f: 4e:68:52:d3:a9:24:d5:c0:0d:1f:2f:0d:c3:57:67: 42:a3:50:b7:9b:1e:c3:25:9e:f0:35:13:f8:9c:d5: 76:5e:c4:eb:a0:d2:42:01:0c:17:f1:59:78:0d:1c: 0a:b1:3d:61:3d:89:85:7c:cd:9a:a3:07:bc:79:e3: 05:5d:97:65:51:e7:9e:26:09:d8:6d:a9:86:03:13: bd:36:af:66:fc:a7:7b:12:9a:cc:38:0d:d1:b4:a1: 9a:e7:13:50:9e:c2:b5:8e:df:b4:7c:74:6e:bb:07: 75:ef:07:8f:04:d3:2a:ee:e1:4b:ce:51:65:59:02: 3c:15:d9:d2:30:0a:0e:44:10:30:97:13:df:57:cf: 1e:df:5f:34:02:bf:8d:b8:ef:ba:25:3b:86:db:ec: 6b:d6:01:0c:09:e7:da:07:5f:47:af:27:fd:e1:a3: 58:2d Exponent: 65537 (0x10001) Signature Algorithm: sha1WithRSAEncryption 94:9c:05:53:39:7f:ae:3c:e9:14:b0:31:98:3f:df:af:05:dc: 67:73:10:bc:e5:7d:bd:20:38:af:1f:56:86:8f:e1:64:fb:ca: df:94:80:7d:78:ec:f8:bb:4e:09:10:7e:d1:2d:50:04:dc:ea: 6d:db:e0:fb:02:da:07:67:e2:06:28:fe:10:ac:9b:37:a6:8d: f3:45:07:61:18:d5:84:75:66:60:d8:fc:8d:8c:38:ce:c3:59: d0:11:d7:9e:d0:a6:eb:1c:e2:ff:5d:6b:61:bd:30:fe:6f:61: ff:2a:25:be:32:b0:31:91:be:3d:92:60:59:57:ec:9e:fd:20: 98:38:4f:6d:53:da:ce:2c:22:cd:61:de:6d:51:5b:b4:f0:91: 05:c6:e3:fe:e9:aa:43:45:a0:a8:ec:ed:4b:db:c1:fb:d0:13: 47:42:cb:38:6a:b0:10:60:ce:a7:80:ef:4b:ab:e8:0a:7e:d8: 40:7e:b4:3f:74:b3:de:d0:9c:97:31:dd:11:47:df:35:63:9f: 17:2c:e0:d7:f2:17:e1:44:50:e1:80:41:f3:54:00:3f:fe:fe: 7e:cf:c4:25:26:8a:ae:34:99:75:d6:90:52:4d:ac:ef:ea:74: e9:f6:f0:42:35:b0:eb:1f:34:6d:a3:a7:f2:bc:5c:02:10:f0: b8:e0:6a:3a # openssl x509 -noout -text -in servercert.pem Certificate: Data: Version: 1 (0x0) Serial Number: 94345 (0x17089) Signature Algorithm: sha256WithRSAEncryption Issuer: C=US, L=Austin, O=IBM, CN=my CA Validity Not Before: Jan 9 03:27:30 2012 GMT Not After : Jan 8 03:27:30 2015 GMT Subject: C=US, O=IBM, CN=kartoffel.stglabs.ibm.com Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit)
36
Linux: Configuring IBM PowerKVM on Power systems
Modulus: 00:c4:fb:0d:92:48:ae:d1:fb:e1:50:c3:32:8f:4d: fd:de:83:07:a7:cf:02:ef:10:be:3c:ad:44:cd:df: b7:52:97:fd:c2:ce:47:39:cc:e5:7d:50:4e:16:06: 48:c0:7f:12:35:b0:da:80:a9:67:7f:72:b2:c8:27: 65:f6:36:54:e1:3c:9c:2d:ac:6d:a1:a3:c1:ae:7f: 96:e1:9d:aa:56:05:85:ff:07:f5:09:29:27:d4:34: 99:3a:0b:f2:35:3a:36:dd:b0:f2:78:ca:cf:4c:21: cb:79:bd:8b:23:d6:f6:62:4f:d4:44:67:62:e5:60: 47:da:05:ae:00:02:03:84:5e:ad:e6:12:ed:ef:27: 99:72:59:46:38:f1:b9:65:fa:47:7a:29:90:1d:14: 47:06:52:da:bd:5b:91:be:42:b3:36:79:de:b2:e6: 6a:4d:01:89:51:d1:a9:3c:7e:c4:7c:37:64:2f:76: 5b:7b:26:08:d8:cc:77:07:20:02:43:53:10:c4:02: 58:f8:53:7e:51:93:66:17:68:b7:35:85:fd:58:34: 5c:3e:1d:0e:74:cf:9c:4e:28:86:1e:b0:b7:16:98: 5c:8b:a8:4e:56:e1:46:f6:dc:66:b9:76:5f:33:dd: 0a:4e:ef:f2:d7:e6:c8:a9:3e:76:50:37:03:95:c3: 4b:c3 Exponent: 65537 (0x10001) Signature Algorithm: sha256WithRSAEncryption 1a:a0:91:19:56:10:da:7c:9c:13:2a:2a:da:ae:12:15:60:71: 33:3a:2b:e0:84:f0:48:d8:d2:f7:f6:ba:08:f3:f9:9d:d8:50: fd:54:c0:ee:60:99:8d:0b:7b:21:6a:d1:9a:aa:71:df:f8:69: dd:44:96:74:2c:85:e8:b0:54:b2:7b:25:c6:06:1f:67:86:45: 0e:c6:6f:80:55:a7:43:d1:51:97:ab:80:17:16:a4:2b:ee:a1: 2b:ba:5c:7b:05:54:83:78:10:dd:42:30:68:40:7b:1c:7d:df: 60:9d:85:6e:16:ea:dc:74:3e:c6:c6:2b:17:30:0f:9c:37:bb: c2:3c:f8:ed:ea:ca:1b:b4:a4:66:30:ad:a7:85:7a:f9:94:28: b6:a5:f0:d8:af:80:5d:3a:3d:00:ee:32:6e:88:15:97:fa:ce: ba:75:70:38:d9:30:91:a3:6e:c0:52:20:a3:4e:38:bf:5a:97: 60:f6:22:4d:46:a3:a0:f1:2b:99:40:ab:c0:b3:67:6e:47:5f: 0b:40:c7:85:b5:6f:a7:76:1c:0d:d3:dd:7e:02:b4:c4:cb:e6: 8a:35:f9:c2:10:6e:13:a7:c3:c3:ec:87:b2:cd:c5:a1:d9:8e: b5:53:5c:d1:bd:d6:6d:19:44:f1:01:c3:7c:0d:a3:14:24:7e: 3e:b9:d3:f5
Step 3. Distributing keys and certificates to the PowerKVM host server When certificates and keys for both the server and the client are in a format readable by libvirt, distribute and configure them to be used by Transport Layer Security (TLS).
About this task To distribute keys and certificates to the PowerKVM host server, complete the following steps:
Procedure 1. Copy the certificate authority (CA) certificate cacert.pem file to /etc/pki/CA/cacert.pem. # cp cacert.pem /etc/pki/CA/cacert.pem
2. Create the /etc/pki/libvirt directory. Copy the servercert.pem server certificate file to /etc/pki/libvirt/servercert.pem. Create the /etc/pki/libvirt/private directory. Copy the serverkey.pem server key file to /etc/pki/libvirt/private/serverkey.pem directory. Make sure that only the root user is able to access the private key. Note: If the keys or certificates are named incorrectly or copied to the wrong directories, the authorization fails. # # # # #
mkdir /etc/pki/libvirt cp servercert.pem /etc/pki/libvirt/. mkdir /etc/pki/libvirt/private cp serverkey.pem /etc/pki/libvirt/private/. chmod -R o-rwx /etc/pki/libvirt/private
3. Verify that the files are placed correctly: Configuring IBM PowerKVM on Power Systems
37
# find /etc/pki/CA/*|xargs ls -l -rw-r--r-- 1 root root 821 Apr 9 15:10 /etc/pki/CA/cacert.pem # ls -lR /etc/pki/libvirt /etc/pki/libvirt: total 16 drwxr-x--- 2 root root 4096 Apr 9 16:35 private -rw-r--r-- 1 root root 751 Apr 9 15:11 servercert.pem /etc/pki/libvirt/private: total 8 -rw-r----- 1 root root 1040 Apr 9 15:11 serverkey.pem
Step 4. Distributing keys and certificates to clients or management stations For every configured management station, repeat the following steps. Place a copy of the client certificate (clientcert.pem) in the /etc/pki/libvirt/ directory and the key (clientkey.pem) in the /etc/pki/libvirt/private/ directory. As usual, restrict access to the client key to the root user.
About this task Restricting the client key to root access results in root-only access to the server with this certificate/key pair. This situation is acceptable if the management console requires root access to manage remote environments. Otherwise, you can disable client certificate verification in the server configuration and stack another layer of authentication with Simple Authentication And Security Layer (SASL) on top of Transport Layer Security (TLS).
Procedure 1. Log in to the management station. 2. Copy the certificate authority (CA) certificate cacert.pem from the PowerKVM host to the management station /etc/pki/CA/ directory. Do not change the file name. # scp kvmhost.company.org:/tmp/cacert.pem /etc/pki/CA/
3. Copy the client certificate clientcert.pem to the /etc/pki/libvirt/ directory, and the client key clientkey.pem to the /etc/pki/libvirt/private/ directory from the PowerKVM host. Use the default file names and make sure that only the root user is able to access the private key. Note: If the keys or certificates are named incorrectly or copied to the wrong directories, authorization fails. # # # # #
mkdir /etc/pki/libvirt/ scp kvmhost.company.org:/tmp/clientcert.pem /etc/pki/libvirt/. mkdir /etc/pki/libvirt/private scp kvmhost.company.org:/tmp/clientkey.pem /etc/pki/libvirt/private/. chmod -R o-rwx /etc/pki/libvirt/private
4. Verify that the files are placed correctly: # ls -lR /etc/pki/libvirt/ /etc/pki/libvirt/: total 8 -rw-r--r-- 1 root root 767 2010-04-09 13:54 clientcert.pem drwxr-xr-- 2 root root 4096 2010-04-09 14:00 private /etc/pki/libvirt/private: total 4 -rw-r--r-- 1 root root 1044 2010-04-09 13:55 clientkey.pem
Step 5. Editing the libvirtd daemon configuration Make sure that the libvirtd daemon is listening to network connections and that the libvirtd.conf file specifies allowed subjects and client certificates.
38
Linux: Configuring IBM PowerKVM on Power systems
About this task To edit the libvirtd daemon configuration, complete the following steps:
Procedure 1. Log in to the PowerKVM host. 2. Make a copy of the /etc/sysconfig/libvirtd file and the /etc/libvirt/libvirtd.conf file. 3. Edit the /etc/sysconfig/libvirtd file and ensure that the --listen argument is passed to the libvirtd daemon. This step ensures that the libvirtd daemon is listening to network connections. The following example shows the changes from the original file: --- libvirtd.orig 2012-01-04 11:41:37.000000000 -0600 +++ libvirtd 2012-01-04 11:31:33.000000000 -0600 @@ -3,7 +3,7 @@ # Listen for TCP/IP connections # NB. must set up TLS/SSL keys prior to using this -#LIBVIRTD_ARGS="--listen" +LIBVIRTD_ARGS="--listen" ...
4. Edit the /etc/libvirt/libvirtd.conf file and configure a set of allowed subjects with the tls_allowed_dn_list directive in the libvirtd.conf file. The following example shows the changes from the original file. It restricts acceptable client certificates to certificates with the O=IBM,OU=virtualization values, while the country (C) and common name (CN) might be assigned any value. The fields in the subject must be in the same order as was used when you create the certificate. --- libvirtd.conf.orig 2012-01-04 11:28:32.000000000 -0600 +++ libvirtd.conf 2012-01-10 11:33:02.000000000 -0600 @@ -210,7 +210,7 @@ # # By default, no DN’s are checked #tls_allowed_dn_list = ["DN1", "DN2"] +tls_allowed_dn_list = ["C=*,O=IBM,OU=virtualization,CN=*"] ...
5. Restart the libvirtd daemon service for changes to take effect: # /etc/init.d/libvirtd restart Stopping libvirtd daemon: Starting libvirtd daemon:
[ OK [ OK
] ]
Step 6. Changing the firewall configuration Allow Transport Layer Security (TLS) authenticated connections through the firewall of the PowerKVM host by opening its TCP port 16514.
About this task To allow TLS-authenticated connections through the firewall of the KVM host, complete the following steps:
Procedure 1. Access the security level configuration: 2. Add TCP port 16514 as a trusted port.
Step 7. Verifying that remote management is working Clients with a valid certificate and subject can connect to the KVM host libvirtd daemon to perform virtual machine guest administration. Configuring IBM PowerKVM on Power Systems
39
To verify that clients are able to connect, run the following commands in your management station or client: # virsh -c qemu+tls://kvmhost.company.org/system list --all Id Name State ---------------------------------- guest01 shut off - guest02 running # virsh -c qemu+tls://kvmhost.company.org/system start guest01 Domain guest01 started
User management User management varies with each distribution. Typically, you can manage users and groups with the industry standard commands. These commands include the following: Table 7. User management commands Command
Description
useradd user name
Creates a user ID
usermod user name
Modifies a user ID
userdel user name
Deletes a user ID
groupadd group name
Creates a group ID
groupmod group name
Modifies a group ID
groupdel group name
Deletes a group ID
passwd user or group name
Assigns a password to the user or group ID
Users and groups are created with the default values set up for users and groups. When you create a user or group ID, the account is created as locked. You must assign a password to unlock the ID. For example, create a user that is called NeildeGrasseTyson. useradd NeildeGrasseTyson
To see the default values created with the user, add the -D option: useradd NeildeGrasseTyson -D
To unlock the user account, add a password: passwd NeildeGrasseTyson
For more information about options for these commands, see the man page for the commands.
Password management Good password management includes the requirement that users must be required to change passwords periodically. To configure password expiration, use either the password options available in the graphical user management interface or the chage command. The chage command allows you to specify password aging values, such as the number of days between password changes. For example to specify the number of days between password changes to be 90 days for user ID NeildeGrasseTyson, issue the following command:
40
Linux: Configuring IBM PowerKVM on Power systems
chage -M 90 NeildeGrasseTyson
You can also force an immediate password change when a user ID is initially used. Use the following command: chage -d 0 NeildeGrasseTyson
For more information about options for these commands, see the man page for the commands.
KVM guest migration You can move a guest from one KVM host to another. Guests can be migrated while offline or live. Before you can migrate between any two PowerKVM host nodes v Both PowerKVM host nodes must access the shared storage in an identical manner. For NFS, the storage must be mounted at the same directory in both the systems and for Fibre Channel and iSCSI, the LUN must be accessed with the same device path name from both the hosts. v Both PowerKVM host nodes must be running same firmware and software versions, including same firmware version, same host kernel version, same QEMU version, same libvirt version, and same SLOF version. v Both PowerKVM host nodes must open the required TCP/IP ports. v Both PowerKVM host nodes must have identical network configurations. All bridging and network configurations must be the same on both hosts. v Both PowerKVM host nodes must have same number of processor threads per core. In other words, both host nodes must have identical split-core or subcore configuration. v Both PowerKVM host nodes must have processors that belong to the same processor family. Additional notes: v If you specify a number of guests to migrate simultaneously and then attempt to migrate more than that number, your migration may fail. v If you are migrating to a new host node and back again, for a guest using storage type qcow2, consider using shared storage or the migration may fail. There are no such restrictions when using storage type raw. v If the guest is running with cpuset configured, then both PowerKVM host nodes should have a matching cpuset available. v You can migrate using storage that is shared between the host nodes. Supported shared storage types include the following types: – NFS – Fibre Channel based LUNs – iSCSI v Do not run any migration tasks as a background process as timeouts and errors can occur at unpredictable times.
Common virsh command options: Migrate You can use the virsh command to perform several tasks that are related to migration. Table 8. Common virsh command options: migration Command option
Description
virsh migrate
Migrates a guest
virsh migrate --live guest_name location
Migrates a guest with live migration
--persistent
Creates a permanent copy of the guest on the destination server. Configuring IBM PowerKVM on Power Systems
41
Table 8. Common virsh command options: migration (continued) Command option
Description
--undefinesource
Undefines the guest on the source server. Using undefinesource without persistent is ill-advised.
--suspend
Pauses the guest on the destination server.
--timeout seconds
Forces the guest on the source server to suspend after a specified amount of time. After the timeout, the migration continues on the suspended guest.
--verbose
Displays the progress of the migration.
Migrating live with virsh command Migrate your guest from one guest to another on the same physical machine or migrate from one physical machine to another using the virsh command.
About this task In this example, migrate a guest, Guest_vm from the first host to the second. The first host is firstHost.example.com and the second host is secondHost.example.com.
Procedure 1. Verify that the guest is running. virsh list Id Name State ---------------------------------5 Guest_vm running
2. Migrate the guest: virsh migrate --live Guest_vm qemu+ssh://secondHost.example.com/system
3. Verify that the guest is migrated to the destination system: virsh list Id Name State ---------------------------------5 Guest_vm running
You can also run your migration in the background with this type of command: echo virsh migrate --live --domain Guest_vm qemu+ssh://secondHost.example.com/system --undefinesource --persistent --timeout 60 --unsafe > script.sh chmod 777 script.sh nohup script.sh &
Migrating a KVM guest offline with migrate option Migrate a guest offline using the virsh migrate option.
About this task In this example, migrate a guest (guest_VM) from its source server (firstHost) to a destination server (secondHost). This migration happens offline, creates a permanent copy of the guest on the destination server, and undefines the guest on the source server.
Procedure 1. Migrate the guest: virsh migrate --persistent --undefinesource guest_VM qemu+ssh://secondHost.example.com/system
42
Linux: Configuring IBM PowerKVM on Power systems
2. Verify that the guest is migrated to the destination system: virsh list Id Name State ---------------------------------Tyson running
3. Verify that the guest is undefined on the source system: virsh list Id Name State ----------------------------------
Migrating a KVM guest offline with dumpxml Migrate a guest offline using the virsh dumpxml --migratable option.
About this task To copy the guest with the virsh command, you must first create the domain information as an xml file and then create a guest with that xml file. To migrate a KM guest offline using the virsh command, follow these steps:
Procedure 1. Save the guest. 2. Run the virsh command with dumpxml --migratable option: virsh dumpxml --migratable GuestID
In this example, the information is sent to stdout. You can send it to a file instead: virsh dumpxml --migratable GuestID >guest.xml
3. Copy the XML file to the new host. 4. Create a guest with the guest.xml file: virsh create guest.xml
The sVirt service Learn about how the sVirt service isolates guests, how to optionally create static sVirt labels, and how to verify dynamic and static sVirt labeling.
Overview of the sVirt service The sVirt service is an SELinux framework included in libvirt that isolates virtual machines. The sVirt service defines unique labels for the processes and disk images of each guest operating system. These unique labels isolate each virtual machine guest from other guests. This isolation prevents malicious users of one guest operating system from accessing the host and attacking the processes and resources of other guest operating systems. sVirt labels are SELinux labels and SELinux uses mandatory access control (MAC). Therefore, the MAC security policy includes the sVirt labels. The Linux kernel enforces the MAC security policy and thus also enforces the sVirt labels. When you enable the SELinux policy, the sVirt service automatically runs and dynamically creates and manages the labels. sVirt dynamic labeling is recommended in most cases. However, you can disable dynamic labeling and create your own static labels. In this case, you are responsible for the uniqueness of the labels. Configuring IBM PowerKVM on Power Systems
43
Creating static sVirt labels You can manually assign sVirt labels to the QEMU processes and disk images of a guest operating system.
Before you begin Before you start, verify that SELinux is running in the enforcing mode by using the getenforce command: getenforce Enforcing
If the getenforce command shows that SELinux is running in permissive mode, you can change the mode to enforcing mode with the setenforce command: setenforce 1
About this task To create static sVirt labels, complete the following steps:
Procedure 1. Edit the domain.xml file of the libvirt facility by typing the following command: virsh edit domainID
In this example, domainID is the ID of the domain of the guest operating system whose processes and disk images you want to label. 2. Add the following code to the domain.xml file: system_u:system_r:svirt_t:s0:category,category system_u:object_r:svirt_image_t:s0:category,category
In this example, category,category are the categories that you want to assign to the processes and disk images of the guest operating system. For example: c32,c256 3. Label all of the virtual images that the guest operating system uses by typing the following command for each virtual image: chcon -t svirt_image_t -l s0:category,category /var/lib/libvirt/images/filename
In this example: v category,category are the categories that you defined in Step 2. For example: c32,c256 v filename is the name of a virtual image that the guest operating system uses. For example: foo.img 4. Start the guest operating system by typing the following command: virsh start domainID
In this example, domainID is the ID of the domain of the guest operating system that you want to start.
Verifying sVirt labeling by examining the labels You can verify dynamic and static sVirt labeling by examining the labels that sVirt assigns to QEMU processes and disk images.
44
Linux: Configuring IBM PowerKVM on Power systems
Before you begin Before you start, complete the following tasks: 1. Verify that SELinux is running in the enforcing mode by using the getenforce command as follows: $ getenforce Enforcing
2. Verify that the guest operating system, for which you want to examine the labels, is running. To start the guest operating system, type the following command: $ virsh start domainID
Where domainID is the ID of the domain of the guest operating system that you want to start.
About this task To verify sVirt labeling, complete the following steps:
Procedure 1. Examine the label of the QEMU process by typing the following command: $ ps -wwC qemu-system-ppc64 -o label,command
The output might look like the following output: LABEL system_u:system_r:svirt_t:s0:c12,c23
COMMAND /usr/bin/qemu-system-ppc64 ... -drive file=/var/lib/libvirt/images/rh-5.5.qcow2,if=virtio,\ index=0,boot=on -drive file=/var/lib/libvirt/images/ \ rh-5.5.img,if=virtio,index=1 ... system_u:system_r:svirt_t:s0:c132,c511 /usr/bin/qemu-system-ppc64 ... -drive file=/var/lib/libvirt/images/foo.qcow2,if=virtio, \ index=0,boot=on ...
The output shows the following information: v The type is svirt_t. v The category is unique for each QEMU process. Each row in the output represents a QEMU process. The first row in the output shows that the sVirt service assigned the c12,c23 categories to a process. The second row in the output shows that the sVirt service assigned the c132,c511 categories to another process. v The disk images that each process uses are also shown. The first process uses the /var/lib/libvirt/images/rh-5.5.qcow2 and /var/lib/libvirt/images/rh-5.5.img files. The second process uses the /var/lib/libvirt/images/foo.qcow2 file. 2. Examine the labels of the disk images by typing the following command: $ ls --scontext /var/lib/libvirt/images/{filename,filename}
Where filename,filename is a list of files whose labels you want to view. For example: rh-5.5.*,foo.qcow2 The output might look like the following output: system_u:object_r:svirt_image_t:s0:c12,c23 /var/lib/libvirt/images/rh-5.5.img system_u:object_r:svirt_image_t:s0:c12,c23 /var/lib/libvirt/images/rh-5.5.qcow2 system_u:object_r:svirt_image_t:s0:c132,c511 /var/lib/libvirt/images/foo.qcow2
The output shows the following information: v The type is svirt_image_t. Configuring IBM PowerKVM on Power Systems
45
v The sVirt service assigned the c12,c23 categories to the /var/lib/libvirt/images/rh-5.5.img and /var/lib/libvirt/images/rh-5.5.qcow2 files. v The sVirt service assigned the c132,c511 categories to the /var/lib/libvirt/images/foo.qcow2 file. 3. Verify that the labels of the disk images from Step 2 match the labels of the corresponding QEMU processes from Step 1. For example: v Step 1 shows a process that is labeled with the c12,c23 categories. That process uses the /var/lib/libvirt/images/rh-5.5.qcow2 and /var/lib/libvirt/images/rh-5.5.img files. Step 2 shows that the /var/lib/libvirt/images/rh-5.5.qcow2 and /var/lib/libvirt/images/rh-5.5.img files are labeled with the same category as the process that uses the files. v Step 1 shows a process that is labeled with the c132,c511 categories. That process uses the /var/lib/libvirt/images/foo.qcow2 file. Step 2 shows that the /var/lib/libvirt/images/foo.qcow2 file is labeled with the same category as the process that uses the file.
Verifying sVirt labeling by viewing the domain.xml file You can verify dynamic and static sVirt labeling by viewing the domain.xml file.
Before you begin Before you start, complete the following tasks: 1. Verify that SELinux is running in the enforcing mode by using the getenforce command as follows: $ getenforce Enforcing
2. Verify that the guest operating system, for which you want to examine the labels, is running. To start the guest operating system, type the following command: $ virsh start domainID
Where domainID is the ID of the domain of the guest operating system that you want to start.
About this task When you use dynamic labeling, the libvirtd daemon updates the domain.xml file to reflect dynamic labeling.
Procedure To verify sVirt labeling, view the domain.xml file by typing the following command: $ virsh dumpxml domainID | grep label
In this example, domainID is the ID of the domain for which you want to verify sVirt labeling. The output might look like the following output: system_u:system_r:svirt_t:s0:c132,c511 system_u:object_r:svirt_image_t:s0:c132,c511
The output shows the following information: v The sVirt service assigned the c132,c511 categories to the process. v The sVirt service assigned the same categories, c132,c511, to the disk images that the process uses.
Manage PowerKVM resources Learn about micro-threading and over-committing process and memory resources.
46
Linux: Configuring IBM PowerKVM on Power systems
Common virsh command options - Resources You can use the virsh command to perform several tasks that are related to resources. Note: As a rule, do not run any virsh commands as a background process as timeouts and errors can occur at unpredictable times. Table 9. Common virsh command options: resources Command option
Description
virsh attach-device guest_name xml_file
Adds a device, such as a CD/DVD, to a guest. The device needs to be specified in an XML file.
virsh attach-disk guest_name xml_file
Adds a disk, such as a hard disk, to a guest. The disk needs to be specified in an XML file.
virsh detach-device guest_name xml_file
Removes a device.
virsh detach-disk guest_name xml_file
Removes a disk.
virsh setmem guest_name count
Allocates memory for a guest. The count value cannot exceed the amount of memory that is specified for the guest.
virsh setmaxmem guest_name
Sets the maximum memory limit for a guest.
virsh vcpuinfo guest_name
Displays information about the virtual processors.
virsh setvcpus guest_name count
Sets the number of virtual processors that are assigned to a guest. The count value cannot exceed the number of processors specified for the guest.
virsh vcpupin guest_name
Controls the affinity of a virtual processor (pining).
Manage processors and memory Learn how to manage virtual processors and memory for guests.
Simultaneous Multi-Threading (SMT) Power8 systems support Simultaneous Multi-Threading (SMT). You can enable SMT to run in your guests, allowing you to run up to 8 threads from a single core. Note: SMT is not supported on the host and must be disabled. To use SMT in your guest, enable it in your guest XML configuration file by setting the number of threads per core. The number set must be a multiple of 2 (1, 2, 4, or 8). The number of virtual processors (vCPUs) must be the product of the number of threads per core and the number of cores. For example, to specify a guest with 2 cores and 8 threads per core, you need to assign 16 virtual processors to the guest (2 cores times 8 threads = 16 vCPUs). The xml configuration file for the guest includes the following markup: 16 power8
Use the ppc64_cpu command to find information about and to edit your virtual processors and SMT. For example, to find information about the processors on a system, run the ppc64_cpu --info command. On a system with 20 cores and SMT not enabled, the output is similar to this output: ppc64_cpu --info Core 0: 0* 1 2 3 4 5 6 7 Core 1: 8* 9 10 11 12 13 14 15 Core 2: 16* 17 18 19 20 21 22 23 Core 3: 24* 25 26 27 28 29 30 31 Configuring IBM PowerKVM on Power Systems
47
Core Core Core Core Core Core Core Core Core Core Core Core Core Core Core Core
4: 32* 33 34 35 36 37 38 39 5: 40* 41 42 43 44 45 46 47 6: 48* 49 50 51 52 53 54 55 7: 56* 57 58 59 60 61 62 63 8: 64* 65 66 67 68 69 70 71 9: 72* 73 74 75 76 77 78 79 10: 80* 81 82 83 84 85 86 87 11: 88* 89 90 91 92 93 94 95 12: 96* 97 98 99 100 101 102 13: 104* 105 106 107 108 109 14: 112* 113 114 115 116 117 15: 120* 121 122 123 124 125 16: 128* 129 130 131 132 133 17: 136* 137 138 139 140 141 18: 144* 145 146 147 148 149 19: 152* 153 154 155 156 157
103 110 118 126 134 142 150 158
111 119 127 135 143 151 159
Note: * indicates that the thread is enabled for the processor To determine if SMT is enabled for a guest, run the ppc64_cpu --smt command. For example: ppc64_cpu --smt SMT is on
To disable SMT, run ppc64_cpu --smt=off. Note: The default setting for a guest is to use threads='1'. When running a guest with a Linux operating system that has been enabled for POWER8, define the guest to use the maximum number of threads per core (threads='8'). If you determine that the guest workload would run more efficiently with less threads per core, then you can change the SMT dynamically with the ppc64_cpu command.
Enabling micro-threading You can enable micro-threading to better use your processors.
About this task To use PowerKVM, your Power system host must be running in with Simultaneous Multi-Threading (SMT) off. However, any guests that are running on that host can run in any supported SMT mode. To more fully use your processors, you can enable micro-threading to "split" the core into two or four subcores, allowing up to four different guests to run on that processor, with each guest that is running up to SMT set to 2. You can run in either normal or in micro-threading mode set to two or four subcores. You can change modes only when the guests are not running. When micro-threading is enabled, you need to ensure that the correct number of threads are online in the host when it is running. To enable micro-threading, follow these steps:
Procedure 1. Ensure that all guests are not running. 2. Set the number of subcores to 1: ppc64_cpu --subcores-per-core=1
3. Enable Simultaneous Multi-Threading (SMT) on the host: ppc64_cpu --smt=on
4. Set the number of subcores to 4: ppc64_cpu --subcores-per-core=4
48
Linux: Configuring IBM PowerKVM on Power systems
5. Turn off Simultaneous Multi-Threading (SMT): ppc64_cpu --smt=off
Results You can verify that micro-threading is enabled by viewing information about the system: ppc64_cpu --info
Note: These settings are persistent across a host restart.
What to do next You can disable microthreading by following these steps: v Ensure that all guests are not running. v Set the cores to full core mode by running this command: ppc64_cpu --subcores-per-core=1
v Reset the thread topology by turning SMT on and then off again. Run these commands: ppc64_cpu --smt=on ppc64_cpu --smt=off
You can verify that micro-threading is disabled by viewing information about the system: ppc64_cpu --info
Processor pinning Workloads that run in the guest operating systems can benefit from processor pinning. Processor pinning is when you pin the virtual processors of a guest operating system to one or more physical processors on the host.
Benefits of processor pinning One of the benefits of processor pinning is that you can pin a virtual processor to one or more physical processors to improve caching. The data, or footprint, of the workload is already loaded in the cache and because no other workload footprints are running on the core, the data is not overwritten. This action improves your cache response time and reduces the need to access more memory. Without this pinning, the virtual processor can be scheduled to run on a different physical core and then must fetch the data into the cache again, causing it to run slower. In one set of lab tests, the SPECjbb2005 workload ran in a guest operating system of KVM. The guest operating system virtual processors were pinned to the corresponding physical processors in the host. The SPECjbb2005 workload was configured with 1 JVM per core, with SMT threads used for garbage collection. One test iteration ran without virtual processor pinning. Another test iteration ran with the processor pinning. The test iteration that used processor pinning resulted in an 18% performance improvement compared to the test iteration that did not use processor pinning. The processor pinning works at two different levels: the KVM operating system and the guest operating system. Pinning one or more virtual processors to one or more physical processors for a guest ensures that the KVM operating system schedules the guest on those pinned physical processors only. However, the second level of pinning comes within the guest, where processes (or threads) can be pinned to virtual processors. Another advantage of processor pinning is the ability to guarantee processor availability to a guest operating system. For example, you have a guest operating system that must run. However, the host system does not have enough physical processors to run all of the virtual processors of all the guest operating systems. You can pin the virtual processors that the guest operating system uses to a subset of physical processors. You then pin the virtual processors that the other guest operating systems use to the Configuring IBM PowerKVM on Power Systems
49
remainder of the physical processors. Therefore, the high priority guest operating system always has the subset of physical processors available on which to run.
Drawbacks of processor pinning Sometimes you can decrease performance when you pin a virtual processor to a set of physical processors. One situation is when physical processors are idle because they are pinned to inactive workloads. For example, a system contains four physical processors. Guest operating system 1 uses one virtual processor to read the stock market ticker. Guest operating system 2 uses four virtual processors to run a web server. You want the stock market ticker to always run so that the system can respond to the stock market data in real time. Therefore, you pin the virtual processor that is used by Guest operating system 1 to one physical processor. You pin the four virtual processors that Guest operating system 2 uses to the remaining three physical processors. In this configuration, the stock market ticker always runs. However, the web server has fewer physical processors on which to run, which might decrease the performance of the web server. You might accept this performance risk when the stock market is open. However, when the stock market closes, the stock market ticker no longer needs to run. Therefore, the physical processor that is pinned to the stock market ticker is idle instead of providing additional resource to the web server.
Over-committing processor and memory resources Memory overcommit allocates more memory space to guest systems than the physical server actually has, and then dynamically reallocates unused memory from idle guests to other guests that require it. For example, some workloads stress the storage subsystem while other workloads stress the network. Some workloads are active during the business day and other workloads are active at night. To fully use your system resources, you can overcommit the processor and memory resources and then control the ability of the guest operating systems to access those resources. When you use overcommit, use these best practices: Target system use at 80% or lower Target overall system use at 80% or lower to maximize resource use while maintaining performance. Allocate the minimum amount of processor resources To maximize performance, allocate the minimum number of virtual processors necessary for each guest operating system to successfully operate. Do not allocate virtual processors to guest operating systems that the guest operating systems do not need. Overcommit memory resources by using page sharing or ballooning Use page sharing, ballooning, or swapping to overcommit memory resources.
Configuring huge pages To increase performance, you can configure the PowerKVM host to use huge pages.
About this task Configuring huge pages can improve performance by backing the guest memory with 16 MB pages, allowing them to reserve these huge pages for applications to use. A PowerKVM guest can be backed with either base (64k) or huge (16M) pages. You can only use huge (16M) pages in the guest if it is backed by huge (16M) pages in the host. If guest is configured to use huge pages, then the memory settings in the guest specification should be in multiples of 16MB. Using huge pages with memory ballooning is not supported. To configure huge pages in the PowerKVM host, complete the following steps: Note: The examples in the following procedure are written in the bash scripting language.
50
Linux: Configuring IBM PowerKVM on Power systems
Procedure 1. Determine the amount of memory to allocate as huge pages. For example: HUGE_SIZE_MB=2048
Tip: Use units of megabytes because huge page sizes are in multiples of megabytes. 2. Identify the size of a huge page by viewing the Hugepagesize entry in the /proc/meminfo file. For example: HUGE_PAGE_SIZE_KB=`grep Hugepagesize /proc/meminfo | awk ’{print $2}’` HUGE_PAGE_SIZE_MB=$((HUGE_PAGE_SIZE_KB/1024))
3. Calculate the number of huge pages that you need. For example: HUGE_PAGES_NEEDED=$(((HUGE_SIZE_MB+HUGE_PAGE_SIZE_MB-1)/HUGE_PAGE_SIZE_MB))
In this example, you divide the amount of huge page memory that you need by the size of a huge page. 4. Allocate the memory as huge pages by writing the number of huge pages to the /proc/sys/vm/ nr_hugepages file. For example: echo $HUGE_PAGES_NEEDED >/proc/sys/vm/nr_hugepages HUGE_PAGES_ALLOCATED=`cat /proc/sys/vm/nr_hugepages`
5. Set the maximum size of any discrete portion of shared memory by writing the size of the huge page memory, in bytes, to the /proc/sys/kernel/shmmax file: For example: SHMMAX_NEEDED=$((HUGE_PAGES_ALLOCATED*HUGE_PAGE_SIZE_KB*1024)) echo $SHMMAX_NEEDED > /proc/sys/kernel/shmmax
6. Set the total amount of shared memory that you want to allocate: a. Identify the size of a page, not a huge page, by using the getconf command. For example: PAGE_SIZE=`getconf PAGE_SIZE`
b. Write the number of pages, not huge pages, of the huge page memory to the /proc/sys/kernel/ shmall file. For example: SHMALL_NEEDED=$((SHMMAX_NEEDED/PAGE_SIZE)) echo $SHMALL_NEEDED > /proc/sys/kernel/shmall
7. The hugetlbfs file system is mounted by default by PowerKVM. Confirm that the hugetlbfs file system is mounted by running the following command: mount -l | grep hugetlbfs
The output of this command is similar to this example output: hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
8. Indicate that the guest memory is backed with huge pages: For example, add this section to the XML configuration file for the guest:
Memory ballooning Memory ballooning allows the guest memory to be changed dynamically by the host, depending on the amount of free memory available. Automatic memory ballooning is especially important when using memory overcommit. Without the ability to grow and shrink memory automatically, the process must be monitored constantly. Automation allows the host to adjust memory of a guest as needed. Memory ballooning is enabled by default. You can enable it manually by adding the following section to the domain XML configuration file for the guest: Configuring IBM PowerKVM on Power Systems
51
You can verify that memory ballooning has been enabled by running the virsh qemu-monitor-command, similar to the following example: virsh qemu-monitor-command --domain ubuntu --hmp ’info balloon’
If memory ballooning is enabled, output similar to the following is returned: allon: actual=2048
If memory ballooning is not enabled, the following message is returned: Device balloon has not been activated
Enabling Kernel Same-page Merge (KSM) Kernel Same-page Merge (KSM) is a process that allows guests to share identical memory pages. By sharing pages, the combined memory usage of the guest is reduced. The savings are especially increased when multiple guests are running similar base operating system images. Note: This process is not instantaneous and takes several cycles to locate and merge the pages.
The KSM tuning service The KSM tuning service (ksmtuned) controls and tunes the KSM service (ksm) by starting and stopping the ksm service as . With only the ksmtuned daemon running, KSM is limited to sharing only 2000 pages. The ksmtuned service is started automatically by PowerKVM. To verify that the service is started, run the following command: systemctl status ksmtuned
You should see output similar to the following example: ksmtuned.service - Kernel Samepage Merging (KSM) Tuning Daemon Loaded: loaded (/usr/lib/systemd/system/ksmtuned.service; enabled) Active: active (running) since Sat 2014-05-10 10:55:52 EDT; 2 days ago Main PID: 18420 (ksmtunedd) CGroup: name=systemd:/system/ksmtuned.service 17510 sleep 60 18420 /bin/bash /usr/sbin/ksmtuned
If the daemon is not running, start it by running the following command: service ksmtuned start
You can change options, such as how often the service should run, by editing the /etc/ksmtuned.conf file. If you create or destroy guests, update the ksmtuned service by running the retune parameter: service ksmtuned retune
The KSM service To fully exploit KSM, enable the KSM (ksm) service. The KSM service is included in the PowerKVM package but is not automatically enabled. Check to see if ksm is enabled by running the following command: cat /sys/kernel/mm/ksm/run
If the output returned is a 1, then ksm is enabled. If the output is a 0, then enable ksm by running this command: echo 1 > /sys/kernel/mm/ksm/run
52
Linux: Configuring IBM PowerKVM on Power systems
KSM options You can set options for KSM in the /etc/ksmtuned.conf file. The default configuration file is similar to the following example: # Configuration file for ksmtuned. # How long ksmtuned should sleep between tuning adjustments # KSM_MONITOR_INTERVAL=60 # Millisecond sleep between ksm scans for 16Gb server. # Smaller servers sleep more, bigger sleep less. # KSM_SLEEP_MSEC=10 # KSM_NPAGES_BOOST is added to the npages value, when free memory is less than thres. # KSM_NPAGES_BOOST=300 # KSM_NPAGES_DECAY Value given is subtracted to the npages value, when free memory is greater than thres. # KSM_NPAGES_DECAY=-50 # KSM_NPAGES_MIN is the lower limit for the npages value. # KSM_NPAGES_MIN=64 # KSM_NAGES_MAX is the upper limit for the npages value. # KSM_NPAGES_MAX=1250 # KSM_TRES_COEF - is the RAM percentage to be calculated in parameter thres. # KSM_THRES_COEF=20 # KSM_THRES_CONST - If this is a low memory system, and the thres value is less than KSM_THRES_CONST, then reset thres value to KSM_THRES_CONST value. # KSM_THRES_CONST=2048 # uncomment the following to enable ksmtuned debug information # LOGFILE=/var/log/ksmtuned # DEBUG=1
Monitoring KSM You can monitor KSM by looking at sys/kernel/mm/ksm files. For example, to find the number of pages that have been merged, look at the /sys/kernel/mm/ksm/pages_shared file: cat /sys/kernel/mm/ksm/page_shared 2976
The files in sys/kernel/mm/ksm include: pages_shared The number of pages that have been merged. pages_sharing The number of virtual pages that are sharing a single page. pages_unshared Number of pages that are candidate to be shared but are not currently shared. pages_volatile Number of pages that are candidate to be shared but are changed frequently. These pages are not merged. full_scan Number of times that KSM has scanned for duplicated content merge_across_nodes Allows merging across NUMA nodes.
Configuring IBM PowerKVM on Power Systems
53
pages_to_scan Number of pages to scan at a time. Setting this to a high value can impact performance. sleep_milisecs Amount of time between scans.
Disabling KSM You can disable KSM by stopping the ksmtuned and the ksm service: # service ksmtuned stop Stopping ksmtuned: # service ksm stop Stopping ksm:
[ OK ] [ OK ]
Enabling PCI passthrough PowerKVM allows for PCI passthrough. PCI passthrough allows guests to have devices that are exclusively assigned to them. Using PCI passthrough, these devices behave as though they were physically attached to the guest operating system.
About this task All device function must be assigned to the guest; individual function passthrough is not supported. For details about which devices are supported for PCI passthrough, see Supported input/output (I/O) adapters. To enable PCI passthrough, follow these steps:
Procedure 1. Stop the guest. 2. Identify the PCI device. Run this command: lspci | grep Ethernet The output should look similar to this output: 0003:05:00.0 0003:05:00.1 0003:05:00.2 0003:05:00.3
Ethernet Ethernet Ethernet Ethernet
controller: controller: controller: controller:
Broadcom Broadcom Broadcom Broadcom
Corporation Corporation Corporation Corporation
NetXtreme NetXtreme NetXtreme NetXtreme
BCM5719 BCM5719 BCM5719 BCM5719
Gigabit Gigabit Gigabit Gigabit
Ethernet Ethernet Ethernet Ethernet
PCIe PCIe PCIe PCIe
3. Identify the ports. Run this command: virsh nodedev-list --cap pci This card has four functions (ports): [snip] pci_0003_05_00_0 pci_0003_05_00_1 pci_0003_05_00_2 pci_0003_05_00_3 [snip]
4. Open the device details in the guest XML. Run this command: virsh nodedev-dumpxml pci_0003_05_00_0 pci_0003_05_00_0 /sys/devices/pci0003:00/0003:00:00.0/0003:01:00.0/0003:02:09.0/0003:05:00.0 pci_0003_02_09_0 tg3 3 5 0 0 NetXtreme BCM5719 Gigabit Ethernet PCIe
54
Linux: Configuring IBM PowerKVM on Power systems
(rev (rev (rev (rev
01) 01) 01) 01)
Broadcom Corporation
5. Assign all the functions of the Ethernet card to the guest by adding the following XML entries in the guest XML.
6. Start the guest.
What to do next Additional considerations for PCI passthrough include the following items: v Booting from a PCI passthrough device is not supported. v Consider using Enhanced Host Controller Interface (ehci) or Extensible Host Controller Interface (xhci) for USB passthrough rather than Open Host Controller Interface (ohci). v Kdump works for guests that are not using PCI passthrough, that is, that are using only virtual or emulated I/O devices. For guests that use PCI passthrough devices, the kernel prints error messages during the transition from the crashed kernel to the kdump kernel, but still boots the kdump kernel successfully. The kdump kernel cannot use any PCI passthrough devices successfully. It is, however, able to access virtual and emulated I/O devices. v If you are seeing the following error when you are using virsh dump for the guest with a PCI passthrough: # virsh dump 6 myguest error: Failed to core dump domain 6 to myguest error: internal error: unable to execute QEMU command ’migrate’: State blocked by non-migratable device ’pci@800000020000005:05.0/vfio-pci’
Use --memory-only option to dump the core file for the PCI passthrough guest: # virsh dump 6 myguest --memory-only Domain 6 dumped to myguest
v If you add an adapter that uses PCI passthrough and is capable of using 64-bit DMA, the system creates a huge DMA window. To get the best performance, especially when you add more than one adapter of this type, update the memory xml of the guest to increase the hard_limit value. The Configuring IBM PowerKVM on Power Systems
55
hard_limit parameter sets the maximum memory that the guest domain can use. To calculate this value for huge 64-bit DMA windows with PCI passthrough, use the formula NUMBER_IOMMU_GROUPS * (2GB + currentMemory) where NUMBER_IOMMU_GROUPS is the number of input/output memory management unit (IOMMU) groups and currentMemory is the memory available to the guest. For example, if you have a guest that is assigned 4 GB of memory and has 3 IOMMU groups, two of which can use 64-bit DMA, set the hard_limit as follows: 3*2 GB + 3*4 GB = 18 GB. Note that while 18 GB of memory is assigned to the guest, this does not mean that all 18 GB is immediately locked. The 14 GB is used as a counter and the guest will more than likely lock less than 4 GB, or the actual guest memory size. v When assigning memory to a PCI passthrough device, the amount of memory that is assigned to the guest must be less than 128 GB. PCI Device Passthrough for PowerKVM is constantly changing and the functional limitations and semantics are subject to change. Check the PowerKVM wiki for additional information.
Update PowerKVM You can update your system with the latest PowerKVM package. Updates are available through Fix Central or the IBM yum repository for PowerKVM. If your system has Internet access, you can use Kimchi or yum. If you do not have Internet access, use the ibm-update-system utility.
Updating PowerKVM with Kimchi Use Kimchi to update your system.
Before you begin Before you update PowerKVM, power off any guest virtual machines and stop any workloads. You are required to restart your system after you apply any updates. To update PowerKVM with the Kimchi, follow these steps:
Procedure 1. Log in to Kimchi. 2. Select Host. 3. Select Software Updates. 4. Inspect all the package updates available. 5. Click Update All.
Updating PowerKVM with yum If your system has Internet access, you can use yum to update your system.
Before you begin Before you update PowerKVM, power off any guest virtual machines and stop any workloads. You are required to restart your system after you apply any updates. To update PowerKVM with yum, follow these steps:
56
Linux: Configuring IBM PowerKVM on Power systems
Procedure 1. Verify that your system has connectivity to the Internet. 2. Use the yum command to create a list of available updates. For example: yum update 3. Review the list of packages and enter Y to install them.
Updating PowerKVM with the ibm-update-system utility If your system is not connected to the Internet, use the ibm-update-system utility to update PowerKVM.
Before you begin Before you update PowerKVM, power off any guest virtual machines and stop any workloads. You are required to restart your system after you apply any updates. To update PowerKVM with the ibm-update-system utility, follow these steps:
Procedure 1. Download the IBM PowerKVM updates ISO image from Fix Central with a PC or notebook that has Internet access. 2. Upload this ISO image, to your system, in a location of your choice. 3. Run the following command: ibm-udpate-system --iso-path=path_to_file
Replace path_to_file with the path to the file. 4. Review the packages marked for update and enter Y to agree.
Upgrade PowerKVM Use this information to upgrade your PowerKVM instance from one release to another or to upgrade your firmware.
Updating the firmware Learn how to update the firmware on your Power System that is running PowerKVM.
About this task Before updating your firmware, ensure all the following items are true: 1. PowerKVM is installed on your system and running. 2. All guests have been stopped. Firmware updates are installed first to a temporary side. This action allows you to book the system in the temporary side to verify that update is successful. You can then move the fix to the permanent side. By default, the server firmware is installed on the temporary side only after the existing contents of the temporary side are permanently installed on the permanent side.
Procedure 1. Your current firmware level is displayed in the ASMI window, for example: FW810.20 (SV810_079). You can also verify your firmware levels by connecting to your PowerKVM host using SSH or through an IPMI console and running the command lsmcode. The output of this command is similar to the following example: Version of System Firmware is FW810.20 (SV810_079) (t) FW810.10 (SV810_076) (p) FW810.20 (SV810_079) (b)
Configuring IBM PowerKVM on Power Systems
57
v The first value (t) is the temporary level. The temporary level is sometimes referred to as the installed level. v The second value (p) is the permanent level. The permanent level is sometimes referred to as the backup level. v The third value (b) is the booted level. The booted level is sometimes referred to as the activated level. To update your firmware level, continue with this task. 2. Download the image file for the firmware update. Fixes are available from Fix Central. Follow the steps available from the site to find and download your image file. For example, select Power from the Product Group, your machine type or model from the Product list and then select the fix that you want to download and press Continue. The on-screen prompts assist you to download the file. The image file must be loaded onto the PowerKVM server or else made available through a network connection. To load the image file from a USB device, follow these steps: a. Connect to the PowerKVM host using an IPMI console or SSH. b. Insert the USB device into the USB port on the server. c. Locate the USB device by running the following command: fdisd -l. The device is named similar to disk /dev/sdy. Note: If your USB device is not recognized, try a different one. d. Create a mount point: cd /mnt. e. Mount the USB device, for example: mount /dev/sdy usb/. f. Verify that the file is on the USB device, for example: ls -l /dev/sdy usb/01vs810_076_076.img. g. Copy the file from the USB device to /root, for example: cp /dev/sdy usb/01vs810_076_076.img /root. 3. Verify the image by running this command: update_flash -v -f . For example: update_flash -v -f 01VS810_076_076.img. Verify that the projected results are what you expect. 4. Update your firmware using the update_flash command. For example: update_flash -f 01VS810_076_076.img Note: Do not interact with your system while the update is running. Your system will reboot when the update is complete. 5. Verify the update by running the command lsmcode or through the ASMI interface. If successful, your system displays the new firmware level. You may want to use the new level of firmware for a period of time to verify that it works correctly. When you are sure that the new level of firmware works correctly, you can permanently install the firmware fix. v To reject the update, run the following command: update_flash -r Note: This command only rejects the temporary side. 6. To permanently install the firmware fix, run the following command: update_flash -c. v Be aware that if you install the firmware fix permanently, you cannot return to the level that was previously on the permanent side. v Moving your firmware to the permanent side does not cause the system to reboot. v After committing your firmware, update_flash will not display the correct Permanent side level until after a system reboot. If you reject a level, update_flash will not display the correct Temporary side level until after a system reboot.
Upgrading PowerKVM to a new release Use the ibm-update-system command to upgrade your PowerKVM system to a new release; for example, to update from PowerKVM 2.1.0 to IBM PowerKVM 2.1.1.
58
Linux: Configuring IBM PowerKVM on Power systems
Before you begin Before you upgrading PowerKVM, power off any guest virtual machines and stop any workloads. You are required to restart your system after you apply any updates. To upgrade to a new release, follow these steps:
Procedure 1. Determine your currently level of PowerKVM by running the following command: rpm -qa | grep release. Look at the output, similar to this output: pkvm2_1-release-19_02.1.0.2.32.0.pkvm2_1.noarch. If the returned release level is 0-2.1.0.2.32.0 or higher, then you are at the necessary level for upgrading your PowerKVM release. If the displayed release is lower, then you need to update your system before upgrading. See “Update PowerKVM” on page 56 for more information. 2. Download the IBM PowerKVM 2.1.1.0 GA or newer ISO image from the Entitled System Support site at http://www-304.ibm.com/servers/eserver/ess/index.wss. Provide your IBM Web ID to sign into the site. You will need your registered customer number in order to access . 3. Provide the PowerKVM 2.1.1.0 ISO image to your existing PowerKVM 2.1.0.1 system. 4. Run the following command: ibm-update-system --upgrade --iso-path=, where is the location of the ISO image that you uploaded in step 3. 5. Review the packages marked for upgrade and then enter Y to start the installation.
Monitor and debug information Monitoring and debugging tools are available from RAS. For more information about these tools, see Managing your server with service and productivity tools. You can also use Kimchi to create debug reports. To create debug reports through Kimchi, follow these steps: 1. Log in to Kimchi 2. Select the Host page 3. In the Debug reports section, select from options to generate a new report, or rename, remove, or download an existing report. The debug report is generated with the sosreport command. The command generates a .tar file that contains configuration and diagnostic information, such as the running kernel version, loaded modules, and system and service configuration files. The command also runs external programs to collect further information and stores this output in the resulting archive.
Install and update packages You can install and update packages through the Kimchi management interface or with the Linux repositories directly. To update your software through Kimchi, follow these steps: 1. Log in to Kimchi 2. Select the Host page 3. In the Software Updates section, information for all of the packages that have updates available is displayed. To update all of them, select Update All. You cannot select individual packages for updates. Configuring IBM PowerKVM on Power Systems
59
For information about updating your software through the Linux repositories, see Managing your server with service and productivity tools.
Migrating to IBM PowerKVM from IBM PowerVM You can migrate your system from PowerVM to IBM PowerKVM.
About this task To migrate your system, follow these steps:
Procedure 1. Log in to the ASMI interface. 2. From the main menu, select Power/Restart Control > Power On/Off System. 3. Verify your setting and click Save settings and power off. 4. After your system powers off, select System Configuration > Hardware Management Consoles. 5. Remove all connections to Hardware Management Consoles (HMC). 6. Select System Service Aids > Factory Configuration. 7. Select Reset server firmware settings. 8. Select System Configuration > Firmware Configuration. 9. Select OPAL as your Hypervisor Mode and set up a password for IPMI session. Note: If the IPMI password is not available on Firmware Configuration, follow these steps: a. From the main menu, select Login Profile -> Change Passwords. b. Select IPMI from the list of user IDs. c. Enter the current password for the administrator (set in step 2) and then enter and confirm a password for IPMI. d. Click Continue.
What to do next You can install IBM PowerKVM or power on your server with IPMI. Go to “Overview of installation steps for PowerKVM” on page 1 for instructions.
Migrating to IBM PowerVM from IBM PowerKVM Use this task to migrate your IBM POWER system from running PowerKVM virtualization to PowerVM virtualization, managed by a Hardware Management Console (HMC).
About this task Follow these steps:
Procedure 1. Log in to the ASMI interface. 2. From the main menu, select Power/Restart Control > Power On/Off System. 3. Verify your setting and click Save settings and power off. 4. After your system powers off, select System Configuration > Firmware Configuration. 5. Select PowerVM as your Hypervisor Mode and click Continue. 6. Select Network Services > Network Configuration and check the IP address of the Service Processor.
60
Linux: Configuring IBM PowerKVM on Power systems
7. Connect to your HMC. For information about configuring an HMC, see Configuring the HMC. 8. From the HMC user interface, select Servers > Connections > Add Managed System. 9. Enter the IP address of the Service Processor and the Password. 10. If the server falls into Recovery Status, select the server and recover it with Servers > Recover Partition Data. 11. Select if you want to restore the profile data from HMC Backup Data or if you want to initialize the server (No partition).
Configuring IBM PowerKVM on Power Systems
61
62
Linux: Configuring IBM PowerKVM on Power systems
Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not grant you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing IBM Corporation North Castle Drive Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Licensees of this program who wish to have information about it for the purpose of enabling: (i) the exchange of information between independently created programs and other programs (including this one) and (ii) the mutual use of the information which has been exchanged, should contact: IBM Corporation Dept. LRAS/Bldg. 903 11501 Burnet Road Austin, TX 78758-3400 U.S.A. Such information may be available, subject to appropriate terms and conditions, including in some cases, payment of a fee. The licensed program described in this document and all licensed material available for it are provided by IBM under terms of the IBM Customer Agreement, IBM International Program License Agreement or any equivalent agreement between us.
© Copyright IBM Corp. 2014
63
For license inquiries regarding double-byte (DBCS) information, contact the IBM Intellectual Property Department in your country or send inquiries, in writing, to: IBM World Trade Asia Corporation Licensing 2-31 Roppongi 3-chome, Minato-ku Tokyo 106-0032, Japan IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental.
Privacy policy considerations IBM Software products, including software as a service solutions, (“Software Offerings”) may use cookies or other technologies to collect product usage information, to help improve the end user experience, to tailor interactions with the end user or for other purposes. In many cases no personally identifiable information is collected by the Software Offerings. Some of our Software Offerings can help enable you to collect personally identifiable information. If this Software Offering uses cookies to collect personally identifiable information, specific information about this offering’s use of cookies is set forth below. This Software Offering does not use cookies or other technologies to collect personally identifiable information. If the configurations deployed for this Software Offering provide you as the customer the ability to collect personally identifiable information from end users via cookies and other technologies, you should seek your own legal advice about any laws applicable to such data collection, including any requirements for notice and consent. For more information about the use of various technologies, including cookies, for these purposes, see IBM’s Privacy Policy at http://www.ibm.com/privacy and IBM’s Online Privacy Statement at http://www.ibm.com/privacy/details the section entitled “Cookies, Web Beacons and Other Technologies” and the “IBM Software Products and Software-as-a-Service Privacy Statement” at http://www.ibm.com/software/info/product-privacy.
Trademarks IBM, the IBM logo, and ibm.com® are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® and ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information
64
Linux: Configuring IBM PowerKVM on Power systems
was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at Copyright and trademark information at www.ibm.com/legal/copytrade.shtml Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. Java™ and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Other company, product, or service names may be trademarks or service marks of others.
Notices
65
66
Linux: Configuring IBM PowerKVM on Power systems
IBM®
Printed in USA
