HP BladeSystem ProLiant firmware management best practices General overview
Introduction......................................................................................................................................... 2 Firmware updates ............................................................................................................................ 2 HP BladeSystem................................................................................................................................... 2 Overview ........................................................................................................................................ 2 HP BladeSystem and firmware ........................................................................................................... 2 Firmware management......................................................................................................................... 3 What options are available for firmware deployment? ......................................................................... 3 Firmware deployment overview ............................................................................................................. 4 Available tools to update HP BladeSystem firmware................................................................................. 4 HP Smart Update Manager (HP SUM) ................................................................................................ 4 Virtual Connect Support Utility ........................................................................................................... 6 Single components ........................................................................................................................... 6 Obtaining the latest available HP firmware update deliverables ................................................................ 7 Deciding which deployment deliverable to use ........................................................................................ 8 HP BladeSystem Firmware Update Bundles ....................................................................................... 10 HP BladeSystem Firmware Deployment Tool ...................................................................................... 11 HP Firmware Maintenance CD ........................................................................................................ 13 Recommended firmware deployment strategies ..................................................................................... 14 Updating an enclosure and blades with an operating system already installed using the HP BladeSystem Firmware Update Bundles or HP Firmware Maintenance CD................................................................... 16 Automatic updates of multiple enclosures including the OA, VC, and server blades ................................... 19 Limitations in VMware ESX/ESXi environments .................................................................................. 20 General best practices........................................................................................................................ 20 Recommended installation order ...................................................................................................... 21 If an operating system has already been installed:.......................................................................... 21 If an operating has not yet been installed: ..................................................................................... 21 If Virtual Connect Ethernet firmware is older than v1.34:................................................................. 21 Acronyms and abbreviations............................................................................................................... 22 For more information.......................................................................................................................... 24
Introduction Firmware updates Updating firmware has become critical to maintaining the health of server systems. As new hardware is released, updating is becoming more cost-effective than replacing the hardware. Consequently, firmware has become as important as software and driver updates to the operating system. This document details firmware best practices for HP c-Class BladeSystem products including: • What firmware to update • Which tools to use • What dependencies to consider for the update process • How to deploy firmware updates
HP BladeSystem Overview When updating HP BladeSystem c-Class enclosures, the following firmware must be considered: • System ROM • iLO 2 firmware • Broadcom, Intel®, and NetXen NIC firmware • Power PIC firmware • Smart Array Controller firmware • SAS and SATA hard drive firmware • Emulex, Qlogic, and Brocade Fibre Channel Host Bus Adapter firmware • Tape blade firmware • Onboard Administrator firmware • Virtual Connect firmware • EVA firmware* • MSA2000 firmware* • BladeSystem Interconnect Switches* *These firmware types are not included in this whitepaper. For information on how to update the firmware for these devices, consult the product documentation. In this white paper, you will learn about firmware types, recommended tools, processes, installation steps, known dependencies, and limitations based on the currently available information.
HP BladeSystem and firmware A good place to start the discussion is with what firmware needs to be deployed and the supported versions of this firmware. The HP BladeSystem Compatibility Chart provides a list of which firmware and software versions of the diffferent BladeSystem products were tested together. This chart provides the current and previous versions of this list. HP recommends that the firmware in your enclosures be at one of these two levels to ensure proper operation of your enclosures. The chart is updated approximately once a quarter. When determining which firmware and software to update for your BladeSystem products, start with this chart.
2
For the latest revision of the HP BladeSystem Compatibility Chart, see http://www.hp.com/go/bladesystemupdates/.
Firmware management What options are available for firmware deployment? The HP ProLiant BladeSystem series of server blades were invented, designed, and created by the same teams that designed the ProLiant DL rack-based and ML stand-alone servers and associated management tools. This is a very reassuring fact from a customer standpoint because this is a maturing technology, and this technology can be understood quickly because the reliability and design confidences that have been built up over many years with both ProLiant and Integrity products are present in HP BladeSystem server blades. As HP BladeSystem server blades were adopted in many environments and subsequent feedback was received, these teams developed additional tools specific to HP BladeSystem solutions. The following chart shows how HP provides individual smart components to update individual firmware, how HP SUM and other tools are provided to manage and deploy these updates, and how the components and tools are integrated into customer deliverables.
With multiple tools comes flexibility, but also the potential for confusion. This section specifies the recommended tools to use to upgrade firmware components within an HP BladeSystem solution, and demystifies which tool is appropriate for which situation. Unfortunately, there is currently not a single tool to update all the firmware within the BladeSystem environment so depending on the firmware that needs to be updated, different tools may be needed to complete the firmware update process.
3
Firmware deployment overview Four key terms are used when describing firmware update use case scenarios; local, remote, online, and offline. • A local installation means the installation tool runs on the physical hardware being updated. An example is a utility that is executed on a server to update its system ROM. • A remote installation means the installation tool runs on one system but updates other physical targets. An example is where the OA firmware can be updated across the network by uploading a firmware .bin file to OA through a web browser. The physical tool runs on a workstation, but remotely updates the OA. • An online installation means the installation takes place while the host processor is running in the customer’s normal environment. For instance, if the customer runs Microsoft® Windows Server® 2003, the update takes place under this environment and does not require the user to boot to a special environment to update the firmware. An online installation does not mean that a reboot may not be necessary for the firmware to be activated. • An offline installation means the installation takes place while the server is booted to a special environment that prevents the user from doing normal operations until the firmware update operations are completed and server is rebooted. An example is when a customer boots to the Firmware Maintenance CD to deploy Emulex, QLogic, and Brocade Fibre Channel HBA firmware updates. These updates require that no I/O operations are done through the adapters while they are being updated, and the only way to ensure this is to boot to a special environment. These terms can be used interchangeably to designate the type of environment required for firmware updates to occur, such as local-online or remote-online.
Available tools to update HP BladeSystem firmware The recommended tools to use for deployment of firmware are as follows: • HP Smart Update Manager (HP SUM) • Virtual Connect Support Utility for Virtual Connect Firmware • Individual Smart Components
HP Smart Update Manager (HP SUM) HP Smart Update Manager (HP SUM) is the primary deployment tool that should be used to update firmware on HP BladeSystem enclosures. HP SUM is designed for maximum flexibility and is shipped within the HP ProLiant Support Pack, Firmware Maintenance CD, HP BladeSystem Firmware Deployment Tool, and HP BladeSystem Online Firmware Update Bundles for Windows® and Linux. HP SUM provides a Graphical User Interface (GUI) and a command-line, scriptable interface for deployment of firmware for single or one-to-many servers and network-based targets such as iLOs, OAs, and Virtual Connect Ethernet and Fibre Channel modules. HP SUM has an integrated hardware and software discovery engine that discovers the installed hardware, current versions of firmware in use on a target, and software versions on target servers. This prevents extraneous network traffic by only sending the components to a target host that are needed. HP SUM also has logic to install updates in the correct order and ensure all dependencies are met before deployment of a firmware update. It also contains logic to prevent version-based dependencies from derailing an installation and ensures firmware updates are handled in a manner that reduces any downtime required for the firmware update process.
4
HP SUM does not require an agent for remote installations as it copies a small, secure SOAP server to the target server for the duration of the installation. After the installation is complete, the SOAP server and all remote files associated with the installation except installation log files are removed. HP SUM copies the log files from the remote targets back to the system where HP SUM is executed. The key features of HP SUM include: • Dependency checking, which ensures appropriate install order and dependency checking between components. • Intelligent deployment deploys only required updates. • Simultaneous firmware and software deployment. • Improved deployment performance. • Local or remote (one-to-many) online deployment. • Local offline deployments with the HP Firmware Deployment Tool and Firmware Maintenance CD. • Remote offline deployment when used with the SmartStart Scripting Toolkit or iLO Virtual Media. • GUI or CLI/scriptable with extensive logging. • Remote command-line deployment. HP SUM provides updates for the following firmware types. Firmware types
Supported by HP Smart Update Manager (HP SUM)
System ROM
Yes
iLO2 firmware
Yes, both server-based and directly to the iLO Management port
Broadcom NIC firmware
Yes
Intel NIC firmware
Yes, after firmware is available
NetXen NIC firmware
Yes, in offline mode only because of NetXen NIC firmware limitations
Power PIC firmware
Yes, beginning with version 3.5.1
Smart Array controller firmware
Yes
SAS and SATA hard drive firmware behind Smart Array controllers
Yes, some older SATA hard drives require offline mode
SAS and SATA hard drive firmware behind nonSmart Array controllers
No
Emulex, QLogic, and Brocade Fibre Channel Host Bus Adapter firmware
Yes, in offline mode only because of HBA firmware limitations
Tape Blade firmware
Yes
Onboard Administrator firmware
Yes, online only
Virtual Connect firmware
Yes, online only beginning with version 3.5.0
Note: The long-term goal is for HP SUM to address all of the firmware components included in an enclosure including partner switches.
The latest HP SUM User Guide is included in the Firmware CD User’s Guide. To obtain a copy of this PDF file, see ftp://ftp.hp.com/pub/c-products/servers/management/smartstart/ and select the latest version of the FWCDUsersGuidex.xx.pdf.
5
Virtual Connect Support Utility HP BladeSystem c-Class Virtual Connect Support Utility (VCSU) enables administrators to: • Upgrade VC Ethernet and VC-FC firmware modules • Perform other maintenance tasks remotely on Virtual Connect Ethernet and Fibre Channel modules installed in both the HP BladeSystem c-Class c7000 and c3000 enclosures using a standalone Windows or Linux-based command-line utility When the utility initiates a firmware upgrade process, all modules can be updated at the same time, or the updates can be alternated between “left” and “right” modules so that network and SAN connectivity is not disrupted during the upgrades. The utility displays a progress message indicating that an update is in progress and the percentage completed. After the module firmware updates are complete, the utility activates all modules. The VCSU also minimizes any outage time in the network fabric and might eliminate it altogether if the VC modules are installed in redundant pairs.
Single components HP provides almost all firmware updates as individual smart components. In most cases, Individual smart components are available for both Windows and Linux environments. Individual smart components, except Virtual Connect firmware, have an embedded installer that checks basic dependencies including supported operating system, presence of required hardware, and enforcement of limited advanced dependencies known for the individual component. The smart components for VC firmware update must use HP Smart Update Manager. Individual smart components can be used in almost any deployment scenario. However, certain limitations, such as some firmware updates needing to be handled offline, exist and can limit effectiveness. Most smart components support only local deployment, but can be integrated with other deployment tools to support deployment on remote servers. The following table illustrates the capabilities of different firmware smart components. Firmware types
Online
Offline
Local
System ROM
X
X
X
iLO2 firmware
X
X
X
Broadcom NIC firmware
X
X
X
Staged
Intel NIC firmware
X
X
X
Staged
X
X
Staged
NetXen NIC firmware
Remote
Is firmware staged for next reboot or to activate immediately? Staged
X (Can be deployed to iLO 2 when using HP SUM to deploy this firmware)
Immediate
Power PIC firmware
X
X
X
Staged
Smart Array controller firmware
X
X
X
Staged
SAS and SATA hard drive firmware behind Smart Array controllers
X (Some older SATA drives may be offline only)
X
X
Staged
SAS and SATA hard drive firmware behind nonSmart Array controllers Emulex, QLogic, and Brocade Fibre Channel
Not currently supported
X
X
Staged
6
Firmware types
Online
Offline
Local
Remote
Is firmware staged for next reboot or to activate immediately?
Tape Blade firmware
X
X
X
Onboard Administrator firmware
X
X (Can be updated stand-alone or by HP SUM)
Immediate
Virtual Connect firmware
X
X (Can be updated by VCSU, HP SUM, or through web browser)
Immediate
Host Bus Adapter firmware Immediate
*Individual components normally require another application like VCSU or HP SUM to deploy them to remote target servers and hosts. In some instances, components have embedded support to deploy themselves remotely. NOTE: Staged firmware means the firmware is written to the chipset, but not activated until the next reboot. Any firmware that is staged can be installed together and activated at a later time.
It is not a best practice to use individual components to upgrade HP BladeSystem enclosure firmware. Use installation tools such as HP SUM and VCSU to deploy components because these tools understand the dependencies between components and installation limitations. These limitations and dependencies can include certain components supporting only offline deployment, validating that the correct driver is installed, ordering Virtual Connect module updates to minimize any network outages.
Obtaining the latest available HP firmware update deliverables In fall 2009, HP will introduce the concept of firmware recipes so that customers can be aware of what HP has tested together as a solution set. Previously, firmware updates had been released as needed by the various product teams. However, because of the complexity of integrating the various firmware deliverables, bundles are now the recommended way to update HP BladeSystem firmware. The firmware recipes are based on the HP BladeSystem Compatibility Chart (www.hp.com/go/bladesystemupdates) and are made approximately once a quarter. To obtain firmware updates for HP BladeSystem products, use the following deliverables: • HP BladeSystem Firmware Update Bundles • HP BladeSystem Firmware Deployment Tool (FDT) • HP Firmware Maintenance CD NOTE: Individual Smart Components might still be released when critical updates are required and will be integrated into the next release of the Firmware Update Bundles, Firmware Deployment Tool, and Firmware Maintenance CD.
7
Deciding which deployment deliverable to use Use the following table to determine the recommended deployment. Deployment type Local
Remote
Upgrade
Bare-metal (no operating system installed on server) Single server
1-to-many
Network-based targets (OA, VC)
Online support
Offline support
•
Firmware Maintenance CD
• Firmware Maintenance CD
•
BladeSystem Firmware Update Bundles
• BladeSystem Firmware Deployment Tool
•
Firmware Maintenance CD
•
BladeSystem Firmware Update Bundles
•
Firmware Maintenance CD
• Firmware Maintenance CD
•
BladeSystem Firmware Update Bundles
• BladeSystem Firmware Deployment Tool
Not applicable
None
• Firmware Maintenance CD • BladeSystem Firmware Deployment Tool
• Firmware Maintenance CD
• Firmware Maintenance CD
• BladeSystem Firmware Update Bundles
• BladeSystem Firmware Deployment Tool
• Firmware Maintenance CD • BladeSystem Firmware Update Bundles
BladeSystem Firmware Deployment Tool (using iLO Virtual Media or c3000 DVD drive)
• Firmware Maintenance CD
None
• BladeSystem Firmware Update Bundles
From another perspective, the following table can help determine the correct deliverable to use depending on the update scenario and also provides limitations of each scenario. Scenario
Description
Limitations
I want to update the server firmware offline automatically.
Recommended deployment deliverable to use:
Using this method updates all firmware in an enclosure with the exception of OA and VC Ethernet and Fibre Channel modules.
•
HP BladeSystem Firmware Deployment Tool (FDT)
Use this scenario when you: • Want an automated way to update a blade firmware
Do not need to update infrastructure firmware at the same time •
Using the HP SUM pull-from-web feature to pull the latest updates is not supported in offline environments.
• Want to leverage the BladeSystem enclosure ability to load an ISO image to multiple server concurrently •
Do not need feedback during the update process
• Do not need log files at the conclusion of the installation for archival or debug purposes •
Need to support devices that are only updateable
offline • Need to update firmware on multiple server blades simultaneously
I want to update server firmware offline but want a graphical installation or manual method.
Recommended deployment deliverable to use: •
HP Firmware Maintenance CD (FWCD)
Use this scenario when you: •
Want to manually update server blade firmware
Do not want to do this under your normal operating environment •
• Need to flash firmware for devices that are only updateable offline
Using this method updates all firmware in an enclosure with the exception of OA and VC Ethernet and Fibre Channel modules. Using the HP SUM pull-from-web feature to pull the latest updates is not supported in offline environments.
8
Scenario
Description
Limitations
• Do not need to update infrastructure firmware at the same time •
Are not familiar with the OA web interface
Only need to update a single blade server within an enclosure • •
I want to update the server firmware online to a single server or network-based host such as the OA, iLO2 and Virtual Connect Ethernet and Fibre Channel modules.
Only need to update firmware for a single device
Recommended deployment deliverables to use: •
HP BladeSystem Bundle for Windows
•
HP BladeSystem Bundle for Linux
•
HP Firmware Maintenance CD (FWCD)
Use this scenario when you: •
Want to manually update server blade firmware
Do not want to do this under your normal operating environment •
• Need to flash firmware for devices that are only updateable offline • Do not need to update infrastructure firmware at the same time •
Using this method, all firmware except the following can be updated: •
NetXen NIC
•
Emulex FC HBA
•
QLogic FC HBA
•
Brocade FC HBA
These types of firmware must be updated offline using either the BladeSystem Firmware Deployment Tool or the HP Firmware Maintenance CD.
Are not familiar with the OA web interface
Only need to update a single blade server within an enclosure •
I want to update the server firmware online to multiple servers and networkbased host such as OA and iLO 2.
Recommended deployment deliverable to use: •
HP BladeSystem Bundle for Windows
•
HP BladeSystem Bundle for Linux
•
HP Firmware Maintenance CD (FWCD)
Use this scenario when you: Want to manually update server blade, OA, or iLO 2 firmware •
• Want the option to script updates to a single or multiple servers and enclosures •
Need to update infrastructure firmware
•
Are not familiar with the OA web interface
•
Need to minimize downtime across the enclosure
Using this method, all firmware except the following can be updated: •
NetXen NIC
•
Emulex FC HBA
•
QLogic FC HBA
•
Brocade FC HBA
These types of firmware must be updated offline using either the BladeSystem Firmware Deployment Tool or the HP Firmware Maintenance CD.
• Want to stage updates so blades can be rebooted at a later time to activate the new firmware •
I want to update the Virtual Connect firmware.
Need to update many server simultaneously
Recommended deployment deliverables to use: •
Virtual Connect Support Utility (VCSU)
•
HP BladeSystem Bundle for Windows
•
HP BladeSystem Bundle for Linux
Use this scenario when you: • Want to manually update Virtual Connect Ethernet and Fibre Channel module firmware •
If updating from VC firmware versions before 1.34, the VC firmware must be updated before the OA firmware. For VC firmware versions 1.34 and later, the update order between the OA and VC is not important.
Want to update a single VC domain (VCSU)
Want to update multiple VC domains simultaneously (HP BladeSystem Bundles) •
• Are not familiar with the OA and VC web interface
9
HP BladeSystem Firmware Update Bundles The HP BladeSystem Firmware Update Bundles for Windows and Linux are a collection of HP Smart Update Manager (HP SUM), firmware smart components and the latest available driver smart components from the Windows and Linux ProLiant Support Packs (PSPs) that have been tested together to ensure compatibility. The driver smart components are provided as a convenience for those customers who might not want to deploy a full PSP but need the drivers to perform firmware update operations. BladeSystem Firmware Update Bundles are for use by those customers who already have an installed operating system on a server and just want to update the firmware to the latest available level. These BladeSystem Firmware Update Bundles require an operating system installed on each target server. These bundles are for online deployment in either a local or remote scenario using HP Smart Update Manager to deploy the updates, enforce limitations, and move the components to remote targets. The BladeSystem Firmware Update Bundles are meant to be deployed from a Windows or Linux workstation, PC, or server to servers running a supported version of the same operating system. Deployment from a Windows workstation to a Linux server or from a Linux server to a Windows workstation is not supported. For workgroup-level deployments of up to 29 servers or network-based devices such as OA and iLO 2, HP SUM should be used to update firmware. The HP SUM team plans to increase this limit in a future release. For enterprise-level deployment scenarios requiring deployment to more than 29 servers concurrently, these BladeSystem Firmware Update Bundles can be integrated with the Version Control software provided by HP SIM. However, additional limitations exist when including these bundles with HP SIM version control infrastructure beyond what is listed in the following table. These limitations include the inability to deploy Onboard Administrator and Virtual Connect firmware updates and the inability to deploy iLO2 firmware through the iLO management port. The HP SIM team is addressing these limitations and plans to resolve them by late 2009 by integrating HP SUM into their version control infrastructure. NOTE: HP supports the current and one previous version of the BladeSystem Firmware Update Bundles as indicated in the HP BladeSystem Compatibility Chart.
10
These BladeSystem Firmware Update Bundles enable you to update the following BladeSystem firmware. Firmware types
Supported by HP BladeSystem Firmware Update Bundle (Online only, supports both local and remote deployment)
System ROM
Yes
iLO 2 firmware
Yes, both server-based and directly to the iLO Management port
Broadcom NIC firmware
Yes
Intel NIC firmware
Yes, beginning with bundle version 1.70 planned for fall 2009 release
NetXen NIC firmware
No
Power PIC firmware
Yes, beginning with bundle version 1.60 released in July 2009
Smart Array controller firmware
Yes
SAS and SATA hard drive firmware behind Smart Array controllers
Yes, some older SATA drives might not be supported if they are for offline only deployment
SAS and SATA hard drive firmware behind nonSmart Array controllers
No
Emulex, QLogic, and Brocade Fibre Channel Host Bus Adapter firmware
No
Tape blade firmware
Yes
Onboard Administrator firmware
Yes
Virtual Connect firmware
Yes, beginning with bundle version 1.60
NOTE: Because HP BladeSystem Firmware Update Bundles do not provide the ability to update all possible firmware types, additional deployment deliverables must be used to ensure all firmware is updated within a BladeSystem enclosure and to align with the HP BladeSystem Compatibility Chart. To update the firmware that cannot be updated by HP BladeSystem Firmware Update Bundles, use the BladeSystem Firmware Deployment Tool or the HP Firmware Maintenance CD. The latest version can be obtained from http://www.hp.com/go/bladesystemupdates/.
HP BladeSystem Firmware Deployment Tool The HP BladeSystem Firmware Deployment Tool is an offline only ISO image containing a collection of HP SUM and firmware smart components that have been tested together to ensure compatibility. The Firmware Deployment Tool has been designed for those customers who do not have an installed operating system on their server blades yet or who want an unattended method of deploying firmware in an offline environment. Because the FDT requires the user boot to it, the server must be taken offline for the period of time needed to complete the firmware update.
11
When a user boots to the FDT, all feedback is provided through the UID lights. While the update process is running, the UID light blinks. Upon completion, the UID light is set to one of two states: • If the UID light is off, the update process has completed and the server operating system can be installed or the server restarted to its previous operating system. • If the UID light is solid, a firmware update failure has occurred requiring user attention. The user should either plug in the KVM dongle or use iLO Remote Console support to browse into the affected server to determine the cause of the failure. The FDT loads the error log into a “vi” editor window for review to assist those who might not be Linux savvy. HP highly recommends that the issue causing the failure be resolved before installing or restarting the operating system to prevent issues that could affect server operation. For more information on how to handle various firmware update failure scenarios, see the “Troubleshooting HP BladeSystem Firmware Deployment Tool” section. The FDT was designed with limited user intervention and feedback in mind. Administrators who want log files or feedback beyond the UID lights should use other firmware update deliverables such as the HP Firmware Maintenance CD. NOTE: Use OA version 2.51 or later to get a list of all major firmware versions deployed in your enclosure using the “Rack Firmware” link on the main OA web page.
NOTE: HP supports the current and one previous version of the BladeSystem Firmware Deployment Tool.
The HP BladeSystem Firmware Deployment Tool provides enables you to update the following BladeSystem firmware. Firmware types
Supported by HP BladeSystem Firmware Deployment Tool
System ROM
Yes
iLO 2 firmware
Yes
Broadcom NIC firmware
Yes
Intel NIC firmware
Yes, beginning with version 1.70
NetXen NIC firmware
Yes, beginning with version 1.70
Power PIC firmware
Yes, beginning with version 1.70
Smart Array controller firmware
Yes
SAS and SATA hard drive firmware behind Smart Array controllers
Yes
SAS and SATA hard drive firmware behind non-Smart Array controllers
No
Emulex, QLogic, and Brocade Fibre Channel Host Bus Adapter firmware
Yes
Tape blade firmware
Yes
Onboard Administrator firmware
No
Virtual Connect firmware
No
12
The latest version can be obtained from http://www.hp.com/go/bladesystemupdates.
HP Firmware Maintenance CD The HP ProLiant Firmware Maintenance CD provides a collection of HP SUM and firmware in an ISO image that can be used either in an offline, interactive mode, or online in either a scripted or interactive mode. The Firmware Maintenance CD provides firmware for supported HP servers and options. The Firmware CD contains all the firmware for the BladeSystem enclosures with the exception of the Virtual Connect firmware. The VC firmware is not available on the Firmware Maintenance CD because of its size. In offline mode, the Firmware Maintenance CD boots a small Linux kernel and enables firmware updates to take place on a single server using the embedded HP SUM software. Because of the special boot environment, support for remote servers and hosts is not available. In online mode, users can leverage the autorun utility to launch HP SUM or browse the CD to the \compaq\swpackages directory and execute it directly. All firmware smart components are placed in the \compaq\swpackages directory for use by HP SUM. If additional firmware smart components are needed, the Firmware Maintenance CD can be copied to a USB key, and these additional components added to the \compaq\swpackages directory. If HP SUM supports the type of firmware added, it is automatically picked up the next time HP SUM is executed. NOTE: Use OA version 2.51 or later to get a list of all major firmware versions deployed in your enclosure using the “Rack Firmware” link on the main OA web page.
The Firmware Maintenance CD provides the means to update the following BladeSystem firmware. Firmware types
Supported by HP Firmware Maintenance CD
System ROM
Yes
iLO 2 firmware
Yes, local and remote
Broadcom NIC firmware
Yes
Intel NIC firmware
No
NetXen NIC firmware
No
Power PIC firmware
Yes, beginning with Firmware Maintenance CD 8.60
Smart Array controller firmware
Yes
SAS and SATA hard drive firmware behind Smart Array controllers
Yes
SAS and SATA hard drive firmware behind nonSmart Array controllers
No
Emulex, QLogic, and Brocade Fibre Channel Host Bus Adapter firmware
Yes, offline only
Tape blade firmware
Yes
Onboard Administrator firmware
Yes, online only
Virtual Connect firmware
Yes, online only and components must be downloaded from the web and added to the CD
13
NOTE: Because firmware might be able to update only in online or offline mode for some components, you might need to execute the HP Firmware Maintenance CD both online and offline to update all the firmware in an enclosure.
NOTE: Virtual Connect firmware components are not shipped on the HP Firmware Maintenance CD because of their size (approximately 50 MB per component). However, if the Virtual Connect firmware components are downloaded from www.hp.com, they can be added to the Firmware Maintenance CD by creating a USB key using the USB Key Utility. The latest version can be obtained from http://www.hp.com/go/bladesystemupdates.
Recommended firmware deployment strategies Over the past several years, many different firmware deployment strategies have been put forth. There are few hard installation order requirements. The known requirements are: • TOA upgrade from version 1.x to 2.x requires an intermediate step and; • The Power PIC firmware v3.4(C) requires iLO 2 firmware version 1.78 or greater. • If the Virtual Connect firmware version is older than version 1.34, update VC firmware before OA firmware to prevent a network outage. • Starting with VC firmware version 1.34 and later, the VC firmware should be updated after the OA firmware. The main differences found in the various installation orders are with certain features, especially with OA, not supporting all features until everything is updated to a given solution. An example of this is the dynamic power capping support. This required updates to the iLO 2, Power PIC, system ROM, and OA firmware to enable this feature. If some of the updates were implemented, various errors would be reported by the OA until the full solution was installed. Many people would not continue with the installation of the other updates after they encountered errors and would report them to HP Support. Because many errors would disappear after the full solution was installed, HP Support would recommend upgrading to the full HP BladeSystem Compatibility Chart. This would lead to a circle of issues where a customer would not upgrade because of an error encountered, and HP Support would require the upgrade in order to receive support. This was exacerbated by a lack of information about the various errors that could be encountered during the upgrade process and whether they could be ignored or not. You can resolve this dilemma by doing the full installation. In an effort to move forward, we document firmware updates and their relative severity in the release notes for new releases of the firmware recipes starting in fall 2009. The recommended firmware update strategies as documented in the following sections are intended to minimize downtime and provide the least intrusive method of updating the OA, Virtual Connect Ethernet and Fibre Channel Modules, and servers in a given enclosure or enclosures that have been daisy-chained together. Depending on the deployment deliverable used, some of these will scale better to larger environments than others. In each scenario, at least two options are provided. For more information, see “Deciding which deployment deliverable to use.”
14
Scenario I want to update a single server under Windows or Linux to the latest firmware versions.
Description Recommended deployment deliverable to use: •
HP BladeSystem Firmware Update Bundles for Windows or Linux
•
NOTE:
Because HP BladeSystem Firmware Update Bundles do not provide the ability to update all possible firmware types, additional deployment deliverables must be used to ensure all firmware is updated within a BladeSystem enclosure and to align with the HP BladeSystem Compatibility Chart. To update the firmware that cannot be updated by HP BladeSystem Firmware Update Bundles, use the BladeSystem Firmware Deployment Tool or the HP Firmware Maintenance CD. The latest version can be obtained from http://www.hp.com/go/bladesystemupdates/. •
I want to update the OA firmware only.
I want to update the Virtual Connect firmware only.
•
HP BladeSystem Firmware Deployment Tool (FDT)
•
HP Firmware Maintenance CD
Recommended deployment deliverable to use: •
HP BladeSystem Firmware Update Bundles for Windows or Linux
•
Use the OA web interface
Recommended deployment deliverable to use: •
Virtual Connect Support Utility (VCSU)
•
HP BladeSystem Firmware Update Bundles for Windows or Linux
I want to update an entire enclosure including OA, VC, and server blades and have an operating system already installed.
Recommended deployment deliverable to use:
I want to update an entire enclosure including the OA, VC and server blades and the blades do not have an operating system already installed.
Recommended deployment deliverable to use:
I want to automate the update of multiple enclosures including the OA, VC and server blades.
•
HP BladeSystem Firmware Update Bundles for Windows or Linux
•
HP Firmware Maintenance CD
•
NOTE:
• Because HP BladeSystem Firmware Update Bundles do not provide the ability to update all possible firmware types, additional deployment deliverables must be used to ensure all firmware is updated within a BladeSystem enclosure and to align with the HP BladeSystem Compatibility Chart. To update the firmware that cannot be updated by HP BladeSystem Firmware Update Bundles, use the BladeSystem Firmware Deployment Tool or the HP Firmware Maintenance CD. The latest version can be obtained from http://www.hp.com/go/bladesystemupdates/. •
HP BladeSystem Firmware Deployment Tool (FDT)
•
HP Firmware Maintenance CD
Recommended deployment deliverable to use: •
NOTE:
• Because HP BladeSystem Firmware Update Bundles do not provide the ability to update all possible firmware types, additional deployment deliverables must be used to ensure all firmware is updated within a BladeSystem enclosure and to align with the HP BladeSystem Compatibility Chart. To update the firmware that cannot be updated by HP BladeSystem Firmware Update Bundles, use the BladeSystem Firmware Deployment Tool or the HP Firmware Maintenance CD. The latest version can be obtained from http://www.hp.com/go/bladesystemupdates/. • HP BladeSystem Firmware Deployment Tool (FDT) and RIBCL scripts for server updates • HP BladeSystem Firmware Update Bundles for Windows or Linux to update OA and VC
15
*For more information on how to implement these scenarios, see the HP BladeSystem ProLiant Firmware Management Best Practices Implementer’s Guide.
Updating an enclosure and blades with an operating system already installed using the HP BladeSystem Firmware Update Bundles or HP Firmware Maintenance CD NOTE: If you have an enclosure with blades with operating systems installed and a new blade is being added, the infrastructure should be updated first, followed by the new server blades.
NOTE: If some blades are already inserted and have an operating system installed, HP recommends that the existing blades be updated using this scenario before adding new server blades.
NOTE: If the Virtual Connect firmware version is older than 1.34, the Virtual Connect firmware must be updated first to prevent a network outage. If the VC firmware is 1.34 or later, the VC should be updated last as described in these instructions.
When you update an enclosure and blades where an operating system is already installed, use a systematic approach to ensure the firmware updates are installed and minimize enclosure downtime. No single tool can update the firmware for server blades and network-based devices such as OA and VC. So, you must execute the updates multiple times. HP SUM can update almost all enclosure and server blade infrastructures. But, to update Emulex, QLogic, and Brocade FC HBA or NetXen NIC firmware, you might need to boot HP SUM in an offline mode using either HP BladeSystem Firmware Deployment Tool or HP Firmware Maintenance CD. This section provides steps to minimize downtime and ensure the firmware is updated in tandem between the various devices and hardware in an enclosure. 1. Use HP BladeSystem Firmware Update Bundles to update firmware on the server blades using
HP SUM.
Update the OA firmware with the server blades to minimize the time it takes to update firmware in the enclosure and to ensure that the OA firmware matches the requirements of the HP BladeSystem Compatibility Chart. All firmware updates can be performed without interfering with normal server operation, with the exception of NIC firmware updates or Virtual Connect firmware updates from versions older than 1.34. 2. Reboot the server after you have installed all updates. The installation loads all the firmware that updates the underlying hardware, but does not activate the updates until the server reboots.
16
3. If a server blade requires offline-mode firmware updates such as Emulex, QLogic, Brocade
adapters, or NetXen NICs, take the server blade offline and update using one of the following deployment tools:
– HP BladeSystem Firmware Deployment Tool (preferred) – HP Firmware Maintenance CD Offline updates are performed after running the HP BladeSystem Firmware Update Bundles for several reasons: – The HP Firmware Maintenance CD depends on the iLO Virtual Media. Updating the offline items first might lead to a situation where the iLO firmware is being updated as part of the installation set, causing HP SUM to lose access to the media. Unexpected results might occur. – Minimizing offline updates reduces server downtime. – Update Virtual Connect and other networking infrastructures using VCSU. Caution: Virtual Connect and other devices can take a long time to update (approximately 1–2 hours) and can cause networking and server outages.
NOTE: If the VC firmware version is older than 1.34, perform step 4 first to prevent a network outage. If the VC firmware is version 1.34 or newer, follow the steps as recommended.
17
Updating an enclosure and blades without an existing operating system installed using the HP BladeSystem Firmware Deployment Tool or HP Firmware Maintenance CD NOTE: If you have an enclosure with server blades with operating systems installed and a new server blade is being added, the infrastructure should be updated first, followed by the new server blades.
NOTE: If some blades are already inserted and have an operating system installed, HP recommended the existing blades be updated using the “Updating an enclosure and blades with an operating system already installed using the HP BladeSystem Firmware Update Bundles or HP Firmware Maintenance CD” scenario before adding new server blades.
NOTE: If the Virtual Connect firmware version is older than 1.34, update it first to prevent a network outage. If the VC firmware is 1.34 or later, update the VC firmware last.
Updating an enclosure and server blades where an operating system is not installed (called bare-metal deployment) is usually only executed one time when a server blade first arrives from the factory and is ready for deployment into a customer’s environment. Two deployment deliverables can be used to install firmware updates when there is no operating system present. Each provides updates to the server blades, but do not take into account the enclosure firmware. In these scenarios, HP recommends that a two-phased approach leveraging the HP BladeSystem Firmware Update Bundles and one of these two deliverables be used in conjunction to deploy firmware updates to fresh enclosures. An enclosure might not have a supported level of firmware on the OA, VC, and server blades, especially if they have sat on a shelf for a while before deployment. Therefore, HP recommends that the OA firmware be updated first. After the OA firmware has been updated, update the VC firmware next. These recommendations are made because out-of-date OA and VC firmware can cause issues that prevent blade servers from powering on. Therefore, using BladeSystem Firmware Update Bundles to update the OA firmware should be the first step in updating the enclosure firmware. The OA firmware update can be done from a network attached client. The second step should be updating the VC firmware in the firmware update strategy for enclosures without an existing operating system installed. The next step in the strategy is to update the blade servers themselves using the HP BladeSystem Firmware Deployment Tool or the HP Firmware Maintenance CD.
18
NOTE: This contradicts the recommendation for enclosures where an operating system is already installed on the server blades.
To recap: 1. Update the OA first using the BladeSystem Firmware Update Bundles for Windows and Linux. 2. Use the VCSU to update the VC firmware after all other updates have been applied, servers have
been rebooted, and all firmware activated.
3. Use the HP BladeSystem Firmware Deployment Tool or HP Firmware Maintenance CD to update
all servers. NOTE: If the VC firmware version is older than 1.34, it is best to reverse steps 1 and 2 to prevent a network outage. If the VC firmware is version 1.34 or later, follow the steps as recommended.
Automatic updates of multiple enclosures including the OA, VC, and server blades To update an existing enclosure automatically, scripts must be used. Scripts can be deployed in a number of ways including the use of OS-based scripting, Remote Deployment Program (RDP), HP SIM, or the HPONCFG utility to send RIBCL scripts. Whatever scripting method is used, there are multiple ways to update the c-Class infrastructure, but only the recommended method is discussed here. The recommended method is to update everything that can be done online, then switch to an offline method to update the firmware that must be updated offline such as Fiber Channel HBAs and certain NIC adapters. This approach minimizes downtime, allows updates to be staged on a production server and activated at the next reboot, and can scale to multiple enclosures. For the online updates, the HP BladeSystem Firmware Update Bundles should be scripted in two batches. The first batch contains the OAs that may need to be updated. The second batch contains all the servers that need firmware updates. When the online updates have been completed successfully, the servers can be rebooted as part of a RIBCL script using the HPONCFG utility that then launches the HP BladeSystem Firmware Deployment Tool to handle the offline updates for each server. The HP BladeSystem Firmware Deployment Tool automatically reboots the servers if the firmware updates are completed successfully or if no updates are needed. After the reboot occurs, the server can be placed back into production mode. Last, the VCSU can be scripted to update all Virtual Connect firmware within the enclosures. To recap: 1. Script the update of the OA firmware using the BladeSystem Firmware Update Bundles for
Windows and Linux. 2. Script the update of the server blades using the BladeSystem Firmware Update Bundles for
Windows and Linux.
3. Script the HPONCFG utility to drive a RIBCL script to launch the HP BladeSystem Firmware
Deployment Tool to update any offline only firmware. This script can also be driven through HP SIM, RDP, and other scripting environments as well.
19
4. Script the VCSU to update the VC firmware after all other updates have been applied, servers
rebooted and all firmware activated.
NOTE: If the VC firmware version is older than 1.34, perform step 4 first to prevent a network outage. If the VC firmware is version 1.34 or newer, follow the steps as recommended.
T his methodology allows most of the firmware updates to be staged and the servers rebooted at a later time to complete steps 2 and 3.
Limitations in VMware ESX/ESXi environments Because of the lack of a console-based interface and the lockdown of SSH support that HP SUM would normally use to connect to Linux-based servers, none of the online HP tools are able to update firmware on server blades running VMware ESX or ESXi. HP recommends evacuating virtual machine guests to an alternate host, and then using either the HP BladeSystem Firmware Deployment Tool (FDT) or HP Firmware Maintenance CD ISO to update the server blade firmware. The steps to use to update firmware in these environments are to update the blade server first, and then the OA and VC.
General best practices • For specific instructions on implementing any of the discussed scenarios in this document, see the HP BladeSystem ProLiant Firmware Management Best Practices Implementer’s Guide. • Always align firmware versions with one of the columns of the HP BladeSystem Compatibility Chart. • Bare-metal deployment should use the HP BladeSystem Firmware Deployment Tool – Offline for unattended installation and HP Firmware Maintenance CD for attended installations. • An entire enclosure should be updated together. Even when multiple enclosures need to be updated, the recommended approach is to update a single enclosure at a time. • Updates to existing servers should use the HP BladeSystem Firmware Update Bundles for Windows and Linux. • The HP BladeSystem Firmware Bundle – Online can have group creation scripted if this is a large environment to ease this potentially time-consuming task. • To get appropriate support from the HP Post Sales support teams when registering an issue, make sure the firmware revision of the components included in the enclosure align with a current column in the HP Blade System Compatibility Chart. • It is acceptable to deploy firmware and software together using HP SUM. • Test your setup in a lab environment before touching production servers. • For those customers who do not have access to a test environment, pick one server and update it. Allow the updated server run for a few days, and if everything continues to work, update others starting with small groups of similar servers. • Make sure customers are within the support window of current and one previous release. If problems occur, it is best to use the most current release. • If you have a problem with a component installing, check the HP website and the ITRC forums to see if there is a new component available or if others are reporting the same problem. If unable to obtain the information to resolve the problem, contact HP Support. • It is acceptable to flash firmware for an entire enclosure at one time as long as appropriate testing has been completed.
20
• If installing a new server, update the firmware first before installing the operating system. This ensures that any firmware issues with the operating system installation have been addressed. • Always ensure you have a backup of the server in case the firmware update is unsuccessful. • Normally, HP does not recommend downgrading or rewriting firmware unless there is a specific need to do so. Downgrading firmware can cause incompatibilities between devices if the downgraded firmware does not mesh well with another devices firmware. • Always reboot the server after server-based driver or firmware upgrades. This allows everything to be flushed out, memory cleared, and so on, so the new firmware/driver starts out fresh.
Recommended installation order If the Virtual Connect Ethernet firmware is version 1.34 or later, or you are not using Virtual Connect in your blade enclosures, the following is the recommended installation order based on whether an operating system has been installed or not. If an operating system has already been installed: 1. Update the blades and the OA first using the BladeSystem Firmware Update Bundles for Windows
and Linux.
2. Use the HP BladeSystem Firmware Deployment Tool or HP Firmware Maintenance CD to update
any offline only firmware.
3. Use the VCSU to update the VC firmware after all other updates have been applied, servers
rebooted, and all firmware activated
If an operating has not yet been installed: 1. Update the OA first using the BladeSystem Firmware Update Bundles for Windows and Linux. 2. Use the VCSU to update the VC firmware after all other updates have been applied, servers
rebooted, and all firmware activated.
3. Use the HP BladeSystem Firmware Deployment Tool or HP Firmware Maintenance CD to update
all server-specific firmware.
If Virtual Connect Ethernet firmware is older than v1.34: If the Virtual Connect Ethernet firmware is older than version 1.34, the following is the recommended installation order regardless of whether an operating system has been installed or not. 1. Update the VC firmware first using the VCSU. 2. Update the server blades and the OA next using the BladeSystem Firmware Update Bundles for
Windows and Linux. 3. Use the HP BladeSystem Firmware Deployment Tool or HP Firmware Maintenance CD to update
any offline only firmware.
21
Acronyms and abbreviations CLI command-line interface EVA Enterprise Virtual Array FC Fiber Channel FDT Firmware Deployment Tool GUI graphical user interface HBA host bus adapter HPONCFG HP Lights-Out Online Configuration Utility HP SIM HP Systems Insight Management HP SUM HP Smart Update Manager iLO 2 Integrated Lights Out 2 ISO International Organization for Standardization KVM keyboard, video, and mouse NIC network interface controller OA Onboard Administrator
22
OS operating system POST Power-On Self Test PSP ProLiant Support Pack RDP Remote Deployment Program RIBCL Remote Insight Board Command Language ROM read-only memory SAS Serial attached SCSI SATA Serial ATA SOAP Simple Object Access Protocol UID unit identification USB universal serial bus VC Virtual Connect VCSU Virtual Connect Support Utility VM virtual machine
23
For more information • About supported hardware, software, and firmware versions, see the HP BladeSystem Compatibility Chart at http://www.hp.com/go/bladesystemupdates/. • About the latest management software and driver updates and known issues, see the HP BladeSystem Matrix Release Notes at http://h18004.www1.hp.com/products/blades/components/matrix/. • For best practices, see the HP BladeSystem Matrix Getting Started Guide at http://h18004.www1.hp.com/products/blades/components/matrix/.
© Copyright 2009 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Microsoft and Windows are U.S. registered trademarks of Microsoft Corporation. UNIX is a registered trademark of The Open Group. 584746-001, September 2009
