CS 423 – Operating Systems Design
Lecture 34 – Virtualization – Part 1 Klara Nahrstedt Fall 2011 Based on slides by Andrew S. Tanenbaum; Mendel Rosenblum slides and talk at ASPLOS Keynote “Impact of Virtualization on Computer Architecture and Operating Systems”, White Paper “Understanding Full Virtualizatoin, Paravirtualization, and Hardware Assist”, 2007 VMware
CS 423 - Fall 2011
Administrative MP4 is out Deadline – December 2 + bonus days if you have left any Interviews for Linux projects – Monday, December 5 Interviews of Android projects – Monday, December 5 Android Competition on Tuesday, December 6 – final 6 projects selected for competition
CS 423 - Fall 2011
Introduction
Virtualization as a concept known over the last 40 years (IBM/370) 1998 – VMware figures out how to virtualize x86 platform Solution ◦ Combination of binary translation and direct execution on the processor that allowed multiple guest OSes to run in full isolation on the same computer with affordable overhead ◦ Dynamic partitioning and sharing available physical resources such as CPU, storage, memory and I/O devices CS 423 - Fall 2011
What is Virtualization ?
A hypervisor running four virtual machines.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639
Benefits of Virtualization Energy (Multiple Heterogeneous Jobs on the same machine Performance (Bandwidth, High Utilization) Reliability, Robustness, Security (Isolation) Scale Sharing of resources Simplification of software development and testing Enabling server consolidation Enhancement of data center agility and business continuity
CS 423 - Fall 2011
Benefits of Virtualization (2)
Servers can run in extremely fault tolerant configurations on virtual infrastructures 24 x 7 x 365 with no downtime needed for backups or hardware maintenance.
CS 423 - Fall 2011
Selected Timeline of x86 Virtualization Technologies Xen v1 Linux w/ Paravirtualization 1998
2002
VMware Founded
One Million VMware Users
2003
2004
Xen v3 Windows w/ Hardware Assist 2005/2006
2007
1st generation Hardware Assist
Source: VMware, “Understanding Full Virtualization,2007 Para-virtualization, and CS 423 - Fall 2011 Hardware Assist”, 2007.
Virtualization Layer with Hypervisor Application Application OS
VMM
Application Application OS
VMM
Application Application OS
VMM
Enhanced Functionality
Base Functionality (e.g., scheduling) HYPERVISOR
Hardware CS 423 - Fall 2011
Virtualization Layer Software responsible for hosting and managing all virtual machines on virtual machine monitors Functionality of hypervisor (virtualization layer component) varies greatly based on architecture and implementation Each VMM running on hypervisor implements VM hardware abstraction and is responsible for running guest OS Each VMM has to partition and share CPU, memory, I/O devices
CS 423 - Fall 2011
Challenges of x86 Hardware Virtualization (x86 privileged architecture without virtualization) Ring 3
User Applications Direct Execution Of User And OS Requests
Ring 2
Ring 1 Ring 0
OS
Host Computer System Hardware
CS 423 - Fall 2011
Challenges of x86 Hardware Virtualization x86 OS is designed to run directly on bare metal hardware x86 OS assumes to fully own computer hardware x86 assumes direct access to memory and hardware to execute its privileged instructions (in Ring 0)
◦ Set of instructions that cause trap if executed in user mode
Virtualizing x86 architecture requires placing virtualization layer under OS to create and manage VMs that deliver shared resource Some sensitive instructions can’t be effectively virtualized as they have different semantics when they are not executed in Ring 0
◦ Set of Instructions executed only in kernel mode - I/O change, MMU settings
Difficulty in trapping and translating these sensitive and privileged instruction requests at runtime was challenge !!!
CS 423 - Fall 2011
CPU Virtualization
VMware developed binary translation techniques ◦ Allow VMM to run in Ring 0 for isolation and performance while moving OS to user level ring with greater privilege than applications in Ring 3, but less privilege than VMM (Virtual Machine Monitor)
Using binary translation for full virtualization approach is de facto standard today ◦ No open standards to define and manage virtualization CS 423 - Fall 2011
Alternative Techniques for CPU Virtualization (handling sensitive and privileged instructions)
Full virtualization using binary translation OS assisted virtualization or Paravirtualization Hardware-assisted virtualization (first generation)
◦ Virtualization in CPU
CS 423 - Fall 2011
Technique 1- Full Virtualization using Binary Translation Ring 3 User Mode
Ring 2
Ring 1 Kernel Mode
User Applications
Ring 0
Guest OS VMM
Host Computer (Host Operating System) System Hardware
CS 423 - Fall 2011
Direct Execution Of User Requests
Binary Translation Of OS Request
Full Virtualization - Type 2 Hypervisors Hosted architecture installs and runs virtualization layer (Type 2 Hypervisor) as an application on top of OS (e.g., Linux) Type 2 Hypervisor supports the broadest range of hardware configurations All sensitive instructions are replaced by calls to procedures that emulate these instructions
◦ No sensitive instructions issued by guest OS are ever executed by true hardware ◦ Sensitive instructions are caught and replaced with a call to a VMM procedure(s) that handle it. ◦ This technique is called binary translation. Sensitive instructions are turned into calls to hypervisor which then emulates them. CS 423 - Fall 2011
Full Virtualization – Type 2 Hypervisor
VMware can virtualize any x86 OS using ◦ Combination of binary translation and direct execution techniques ◦ Translation of kernel code to replace non-virtualize-able instructions with new sequences of instructions that have intended effect on virtual hardware ◦ User level code is directly executed on processor for high performance virtualization
Each VMM provides each VM services of physical system, including virtual BIOS, virtual devices and virtual memory management Guest OS does not need any modification No need for any hardware assist or OS assist to virtualize sensitive and privileged instructions Hypervisor translates all OS instructions on fly and caches results for future use while user level instructions run unmodified at native speed. Best isolation and security for VM Simplified migration and portability CS 423 - Fall 2011
Technique 2 - OS Assisted Virtualization or Paravirtualization Ring 3
User Applications
Ring 2
Direct Execution Of User Requests
Ring 1 Ring 0
Para-virtualized Guest OS (e.g., Modified Linux) Virtualization Layer (microkernel)
Host Computer System Hardware CS 423 - Fall 2011
‘Hypercalls’ to Virtualization Layer replace Non-virtualizable OS instructions
Technique 2 - OS Assisted Virtualization or Para-virtualization
Paravirtualization refers to communication between guest OS and hypervisor to improve performance and efficiency It involves modifying OS kernel to replace non-virtualizable instructions with hypercalls that communicate directly with virtualization layer hypervisor Hypervisor provides APIs to critical kernel operations ◦ Memory management, interrupt handling, time keeping CS 423 - Fall 2011
Paravirtualization (1)
Figure 8-27. A hypervisor supporting both true virtualization and paravirtualization.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639
Paravirtualization (2) Value of paravirtualization is in lower virtualization overhead Performance advantage of paravirtualization over full virtualization varies greatly depending on workload Its compatibility and portability is poor It introduces significant support and maintainability issues in production
◦ Requires deep OS kernel modifications
Xen – open source project is example of paravirtualization ◦ Virtualizes processor and memory using modified Linux kernel ◦ Virtualizes I/O using custom guest OS device drivers
CS 423 - Fall 2011
Paravirtualization (3)
Figure 8-28.VMI Linux running on (a) the bare hardware (b) VMware (c) Xen.
VMI – Virtual Machine Interface - Low Level Interface that interfaces with Hardware or hypervisor Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639
Technique 3 – Hardware Assisted Virtualization Ring 3
Non-root Mode Privilege Levels
User Applications
Ring 2
Direct Execution Of User Requests
Ring 1 Ring 0
Root Mode Privilege Levels
OS Requests Trap to VMM Without Binary Translation or Paravirtualization
Guest OS VMM
Host Computer System Hardware CS 423 - Fall 2011
Hardware-Assisted Virtualization - Type 1 Hypervisors
Hypervisor architecture (bare-metal) – virtualization layer directly on clean x86 system Type 1 Hypervisor – more efficient, greater scalability, robustness, performance When the operating system in a virtual machine executes a kernel-only instruction, it traps to the hypervisor if virtualization technology is present. VMware ESX Server, Citrix XenServer, Microsoft Virtual Server – Type 1 hypervisor
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639
Hardware-assisted Virtualization (1)
Hardware vendors develop new features to simplify virtualization techniques 1st generation enhancements
◦ Intel Virtualization Technology (VT-x) ◦ AMD’s AMD-V ◦ Both target privileged instructions with new CPU execution mode feature that allows VMM to run in new root mode below Ring 0
Privileged and sensitive calls are set to automatically trap to hypervisor, removing need for either binary translation or paravirtualization CS 423 - Fall 2011
Hardware-assisted Virtualization (2) Guest state stored in Virtual Machine Control Structures (VT-x) or Virtual Machine Control Blocks (AMD-V) Processors with Intel VT-x and AMD-V became available in 2005/2006 Due to high hypervisor-to-guest transition overhead and rigid programming model
◦ VMware’s binary translation approach outperforms 1st generation hardware-assisted implementation CS 423 - Fall 2011
Conclusion Virtualization – concept embraced by industry and it is to stay here Virtualization - A lot of benefits CPU Virtualization
◦ Full Virtualization ◦ Paravirtualization ◦ Hardware-assisted Virtualization
CS 423 - Fall 2011
