CA Application Performance Management
for IBM z/OS Guide Release 9.6
This Documentation, which includes embedded help systems and electronically distributed materials, (hereinafter referred to as the “Documentation”) is for your informational purposes only and is subject to change or withdrawal by CA at any time. This Documentation is proprietary information of CA and may not be copied, transferred, reproduced, disclosed, modified or duplicated, in whole or in part, without the prior written consent of CA. If you are a licensed user of the software product(s) addressed in the Documentation, you may print or otherwise make available a reasonable number of copies of the Documentation for internal use by you and your employees in connection with that software, provided that all CA copyright notices and legends are affixed to each reproduced copy. The right to print or otherwise make available copies of the Documentation is limited to the period during which the applicable license for such software remains in full force and effect. Should the license terminate for any reason, it is your responsibility to certify in writing to CA that all copies and partial copies of the Documentation have been returned to CA or destroyed. TO THE EXTENT PERMITTED BY APPLICABLE LAW, CA PROVIDES THIS DOCUMENTATION “AS IS” WITHOUT WARRANTY OF ANY KIND, INCLUDING WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. IN NO EVENT WILL CA BE LIABLE TO YOU OR ANY THIRD PARTY FOR ANY LOSS OR DAMAGE, DIRECT OR INDIRECT, FROM THE USE OF THIS DOCUMENTATION, INCLUDING WITHOUT LIMITATION, LOST PROFITS, LOST INVESTMENT, BUSINESS INTERRUPTION, GOODWILL, OR LOST DATA, EVEN IF CA IS EXPRESSLY ADVISED IN ADVANCE OF THE POSSIBILITY OF SUCH LOSS OR DAMAGE. The use of any software product referenced in the Documentation is governed by the applicable license agreement and such license agreement is not modified in any way by the terms of this notice. The manufacturer of this Documentation is CA. Provided with “Restricted Rights.” Use, duplication or disclosure by the United States Government is subject to the restrictions set forth in FAR Sections 12.212, 52.227-14, and 52.227-19(c)(1) - (2) and DFARS Section 252.227-7014(b)(3), as applicable, or their successors. Copyright © 2014 CA. All rights reserved. All trademarks, trade names, service marks, and logos referenced herein belong to their respective companies.
CA Technologies Product References This document references the following CA Technologies products and features: ■
CA Application Performance Management (CA APM)
■
CA Application Performance Management ChangeDetector (CA APM ChangeDetector)
■
CA Application Performance Management ErrorDetector (CA APM ErrorDetector)
■
CA Application Performance Management for CA Database Performance (CA APM for CA Database Performance)
■
CA Application Performance Management for CA SiteMinder® (CA APM for CA SiteMinder®)
■
CA Application Performance Management for CA SiteMinder® Application Server Agents (CA APM for CA SiteMinder® ASA)
■
CA Application Performance Management for IBM CICS Transaction Gateway (CA APM for IBM CICS Transaction Gateway)
■
CA Application Performance Management for IBM WebSphere Application Server for z/OS (CA APM for IBM WebSphere Application Server for z/OS)
■
CA Application Performance Management for IBM WebSphere for Distributed Environments (CA APM for IBM WebSphere for Distributed Environments)
■
CA Application Performance Management for IBM WebSphere MQ (CA APM for IBM WebSphere MQ)
■
CA Application Performance Management for IBM WebSphere Portal (CA APM for IBM WebSphere Portal)
■
CA Application Performance Management for IBM WebSphere Process Server (CA APM for IBM WebSphere Process Server)
■
CA Application Performance Management for IBM z/OS® (CA APM for IBM z/OS®)
■
CA Application Performance Management for Microsoft SharePoint (CA APM for Microsoft SharePoint)
■
CA Application Performance Management for Oracle Databases (CA APM for Oracle Databases)
■
CA Application Performance Management for Oracle Service Bus (CA APM for Oracle Service Bus)
■
CA Application Performance Management for Oracle WebLogic Portal (CA APM for Oracle WebLogic Portal)
■
CA Application Performance Management for Oracle WebLogic Server (CA APM for Oracle WebLogic Server)
■
CA Application Performance Management for SOA (CA APM for SOA)
■
CA Application Performance Management for TIBCO BusinessWorks (CA APM for TIBCO BusinessWorks)
■
CA Application Performance Management for TIBCO Enterprise Message Service (CA APM for TIBCO Enterprise Message Service)
■
CA Application Performance Management for Web Servers (CA APM for Web Servers)
■
CA Application Performance Management for webMethods Broker (CA APM for webMethods Broker)
■
CA Application Performance Management for webMethods Integration Server (CA APM for webMethods Integration Server)
■
CA Application Performance Management Integration for CA CMDB (CA APM Integration for CA CMDB)
■
CA Application Performance Management Integration for CA NSM (CA APM Integration for CA NSM)
■
CA Application Performance Management LeakHunter (CA APM LeakHunter)
■
CA Application Performance Management Transaction Generator (CA APM TG)
■
CA Cross-Enterprise Application Performance Management
■
CA Customer Experience Manager (CA CEM)
■
CA Embedded Entitlements Manager (CA EEM)
■
CA eHealth® Performance Manager (CA eHealth)
■
CA Insight™ Database Performance Monitor for DB2 for z/OS®
■
CA Introscope®
■
CA SiteMinder®
■
CA Spectrum®
■
CA NetQoS® Performance Center
■
CA Performance Center
Contact CA Technologies Contact CA Support For your convenience, CA Technologies provides one site where you can access the information that you need for your Home Office, Small Business, and Enterprise CA Technologies products. At http://ca.com/support, you can access the following resources: ■
Online and telephone contact information for technical assistance and customer services
■
Information about user communities and forums
■
Product and documentation downloads
■
CA Support policies and guidelines
■
Other helpful resources appropriate for your product
Providing Feedback About Product Documentation If you have comments or questions about CA Technologies product documentation, you can send a message to
[email protected]. To provide feedback about CA Technologies product documentation, complete our short customer survey which is available on the CA Support website at http://ca.com/docs.
Contents Chapter 1: Overview of CA APM for IBM z/OS
11
CA APM for IBM z/OS ................................................................................................................................................. 11 CA APM for IBM z/OS Architecture ............................................................................................................................ 11 CA APM for IBM z/OS Functionality ........................................................................................................................... 12
Chapter 2: Installing CA APM for IBM z/OS
15
System Requirements ................................................................................................................................................ 15 Verifying the CA Introscope® Installation................................................................................................................... 15 Preparing for the Installation ..................................................................................................................................... 16 Planning the Installation Locations ..................................................................................................................... 17 Download the Installation Files ........................................................................................................................... 17 Directory and File Structure ................................................................................................................................ 18 How to Deploy CA APM for IBM z/OS ........................................................................................................................ 19 Install CA APM for IBM z/OS ...................................................................................................................................... 20 Integrate CA APM for IBM z/OS with IBM WebSphere for z/OS ......................................................................... 22 Toggle Metrics with PBD File Settings ................................................................................................................. 22 Install the CA APM for IBM z/OS Management Module and Tab Views ............................................................. 25 Verifying the Installation ............................................................................................................................................ 25 Verifying CA APM for IBM z/OS ........................................................................................................................... 25 Verifying the System Overview Service Metrics ................................................................................................. 25 Uninstall CA APM for IBM z/OS .................................................................................................................................. 26
Chapter 3: Configuring the APM Extension for z/OS
27
CA APM for IBM z/OS Configuration .......................................................................................................................... 27 About the System Overview Service .......................................................................................................................... 27 Monitor Additional Classes or Processes ................................................................................................................... 28 Sample Files: Non-Blame and Blame Versions ........................................................................................................... 28
Chapter 4: Using CA APM for IBM z/OS
29
View z/OS Performance Metrics in CA Introscope® ................................................................................................... 29 Analyzing Historical Data for a Specific Metric ................................................................................................... 30 Alerts .......................................................................................................................................................................... 30
Chapter 5: Using the CA Introscope® Console and z/OS Dashboards
31
Overview Dashboard .................................................................................................................................................. 31
Contents 7
The CA Introscope® Console and z/OS Dashboards ................................................................................................... 31 Java Entity Beans Dashboard ..................................................................................................................................... 32 Java Server Pages Dashboard ..................................................................................................................................... 32 Java Servlets Dashboard ............................................................................................................................................. 33 Java Session Beans Dashboard ................................................................................................................................... 33
Chapter 6: Using CA Introscope® Investigator Tab Views
35
CA Introscope® Investigator Tab Views ...................................................................................................................... 35 Servlets Graphical Tab View ....................................................................................................................................... 36 Servlets Tabular Tab View .......................................................................................................................................... 36 JSP CPU Times Tab View ............................................................................................................................................. 36 EJB Graphical Tab View .............................................................................................................................................. 37 EJB Tabular Tab View ................................................................................................................................................. 37 DB2 Backends Graphical Tab View ............................................................................................................................. 37 DB2 Backends Tabular Tab View ................................................................................................................................ 38 JDBC Tabular Tab View ............................................................................................................................................... 38 WebServices CPU Time Aggregates Client Tab View .................................................................................................. 38 WebServices CPU Time Aggregates Server Tab View................................................................................................. 38 WebServices Engine Handler Tab View ...................................................................................................................... 39 Process Overview Tab View ....................................................................................................................................... 39 System Overview Tab View ........................................................................................................................................ 39 WebSphereMQ Graphical Tab View ........................................................................................................................... 40 WebSphereMQ Tabular Tab View .............................................................................................................................. 40 CTG Frontends CPU Time Graphical Tab View ........................................................................................................... 41 CTG Frontends CPU Time Tabular Tab View .............................................................................................................. 41 CTG Backends CPU Time Graphical Tab View ............................................................................................................ 41 CTG Backends CPU Time Tabular Tab View ................................................................................................................ 42
Appendix A: CA APM for IBM z/OS Metrics
43
Tracer Metric .............................................................................................................................................................. 43 Process Overview Service Metrics.............................................................................................................................. 43 System Overview Service Metrics .............................................................................................................................. 44 CPU Usage Metrics - J2EE Based ................................................................................................................................ 46 CPU Usage Metrics - SQL Agent ................................................................................................................................. 48 CPU Usage Metrics - Web Services ............................................................................................................................ 49 CPU Usage Metrics - Web Services Unique to CA Introscope® .................................................................................. 50 CPU Usage Metrics - WebSphere MQ ........................................................................................................................ 51 CPU Usage Metrics - CTG ........................................................................................................................................... 52
8 for IBM z/OS Guide
Appendix B: Troubleshooting
55
Management Module is Not Installed ........................................................................................................................ 55 Dashboards Have Empty Panels ................................................................................................................................. 55
Contents 9
Chapter 1: Overview of CA APM for IBM z/OS This chapter provides an overview of CA APM for IBM z/OS and the requirements for its use. This section contains the following topics: CA APM for IBM z/OS (see page 11) CA APM for IBM z/OS Architecture (see page 11) CA APM for IBM z/OS Functionality (see page 12)
CA APM for IBM z/OS CA APM for IBM z/OS is an extension to the CA Introscope® performance management product. CA Introscope® provides a real-time Java thread monitoring so you can determine the number of CPU cycles Java applications consume. If you have service level requirements to meet, CA APM for IBM z/OS can help you ensure a high level of availability and performance for your enterprise applications. Systems programmers and DBAs can properly tune and run their online production systems. Network analysts and capacity planners can monitor overall systems health and view resource use. This capability makes it possible to prevent bottlenecks by better balancing the workload and adding more capacity when it is cost effective. In addition to the default metrics CA APM provides, you can alter metrics that CA APM for IBM z/OS reports.
CA APM for IBM z/OS Architecture CA APM for IBM z/OS consists of a component that operates on the IBM mainframe and another that runs on the Enterprise Manager. These components provide extended reporting on Java APIs. The components also report on the performance impact that all jobs running in the z/OS environment have on the Java environment. The mainframe component performs data collection by using a custom tracer. CA Introscope® uses this custom tracer to gather the Java component level CPU timing information for CA Introscope®-enabled components running under z/OS.
Chapter 1: Overview of CA APM for IBM z/OS 11
CA APM for IBM z/OS Functionality
The Enterprise Manager component: ■
Management Modules—A set of management modules that run on the Enterprise Manager. The Enterprise Manager can run in the z/OS mainframe environment. However, typically the Enterprise Manager runs on a Windows server or a UNIX Server.
■
Tab Views—A set of graphical modules that runs the Enterprise Manager and are displayed in the Workstation.
The following diagram depicts the software layers that CA APM for IBM z/OS requires on the z/OS system. z/OS JVM – Application (WebSphere) Instrumented Java classes Servlets
JSPs
Entity Beans
Session Beans
MDBs
Introscope Workstation
System Overview Service Thread
CpuMethodTimer Support
Introscope Agent
TCP/IP
Introscope Enterprise Manager
CA APM for IBM z/OS Functionality CA APM for IBM z/OS is designed to meet performance tuning and capacity planning requirements in organizations where the z/OS mainframe platform is in use. CA APM for IBM z/OS helps transaction environments using various Java-enabled Web servers. To gather information for this environment, CA APM for IBM z/OS uses the CPUMethodTracer. CA APM for IBM z/OS collects data by using a custom tracer. CA Introscope® uses this custom tracer to gather Java component level CPU timing information for CA Introscope®-enabled components running under z/OS. The custom tracer operates in both Normal and Blame modes. Note: In CA Introscope®, Blame mode describes tracking component interactions and component resource usage in an instrumented application. Normal mode is when Blame mode is disabled.
12 for IBM z/OS Guide
CA APM for IBM z/OS Functionality
The System Overview Service provides high-level system information regarding related job processes and system performance data. Using both the custom tracer for Java component level CPU timing information and the System Overview Service for high-level system information allows Java component performance to be viewed. This Java component performance data is viewed in context with the rest of the system resources currently being utilized. Use Blame mode with CPU Method traces infrequently or only for critical issues, because Blame mode has higher overhead. For typical operation, use normal mode for CPU Method tracers.
Chapter 1: Overview of CA APM for IBM z/OS 13
Chapter 2: Installing CA APM for IBM z/OS This chapter presents information about the z/OS environment that is required for the CA APM for IBM z/OS and installation instructions. This section contains the following topics: System Requirements (see page 15) Verifying the CA Introscope® Installation (see page 15) Preparing for the Installation (see page 16) How to Deploy CA APM for IBM z/OS (see page 19) Install CA APM for IBM z/OS (see page 20) Verifying the Installation (see page 25) Uninstall CA APM for IBM z/OS (see page 26)
System Requirements The infrastructure that is required to run CA APM for IBM z/OS is as follows: ■
z/OS 1.9, 1.10, 1.11, or 1.12
■
UNIX System Services enabled
■
WebSphere Application Server, any version 7.0 through 8.5
■
Support for 32-bit and 64-bit JVMs
■
Java 1.4.2, 1.5, or 1.6
■
Enterprise Manager version 9.1
■
CA Introscope® agents version 8.x or 9.x
Note: The Japanese version of the CA APM for IBM z/OS only works with CA Introscope® 9.0.5.
Verifying the CA Introscope® Installation Before you install CA APM for IBM z/OS, CA Introscope® must be installed and operational. For further information about CA Introscope® architecture and installation requirements, see the following guides: ■
CA APM Installation and Upgrade Guide
■
CA APM Java Agent Implementation Guide
Chapter 2: Installing CA APM for IBM z/OS 15
Preparing for the Installation
■
CA APM Sizing and Performance Guide
■
The remainder of the CA APM documentation.
The CA Introscope® environment requires the interaction of several components that are installed on various hardware platforms. The following is a brief explanation of the location and purpose of the CA Introscope® components: CA Introscope® Component
Hardware Platform
Function
Enterprise Manager
Windows Server, or UNIX Server
Receives measurement data from the CPUMethodTracer
CA Introscope® Workstation
PC Workstation
Connects to the Enterprise Manager to view data
SmartStor database
z/OS Mainframe, Windows Server, or UNIX Server
Receives data from the Enterprise Manager. Stores data for historical reporting, trend analysis, and determining Service Level performance
CA APM for IBM z/OS
z/OS Mainframe
Gathers platform-specific metrics
Preparing for the Installation Perform several steps before you install CA APM for IBM z/OS:
16 for IBM z/OS Guide
■
Plan where to install your components.
■
Collect the information that is required for installation.
■
Review the installation and configuration process.
Preparing for the Installation
Planning the Installation Locations It is important to plan the installation location for each of the CA APM for IBM z/OS components. The following table identifies the location of each component: Components
Required Location
CA APM for IBM z/OS
Install the CA APM for IBM z/OS on the z/OS computer.
Enterprise Manager
Enterprise Manager may reside on a Windows server or a UNIX server.
CA APM for IBM z/OS Management Module
Install the Management Module on the same host computer as the Enterprise Manager. zOSExtension/config/modules/ zOSExtensionManagementModule.jar
Download the Installation Files The CA APM for IBM z/OS release files are part of the Java agent installer. All extensions, .PBD files, and .PBL files are located in the /examples/PowerPackForIBMzOSExtension directory. The in the filename refers to the version of the CA APM for IBM z/OS. Follow these steps: 1.
After your CA APM representative provides the location, download the appropriate product file (CAWilyAPMExtensionForzOSvebcdic.tar) to the z/OS computer.
2.
Extract the files from the installer package using the tar command: tar -xvof CAWilyAPMExtensionForzOSvebcdic.tar Unpacking the release package creates a top-level directory named: zOSExtension.
3.
Select a directory on the z/OS mainframe into which to place the CA APM for IBM z/OS files for installation.
4.
Review the installation overview (see page 19).
Chapter 2: Installing CA APM for IBM z/OS 17
Preparing for the Installation
Directory and File Structure The system administrator selects the directory location for CA APM for IBM z/OS. The following table lists the files included in the z/OS Extension directory of the installation bundle when the CA APM for IBM z/OS TAR file is extracted. CA APM for IBM z/OS File
Description
/examples/PowerPackForIBMzOSExt ension/ext/libsysinfo32.so
Native shared library for 32-bit z/OS versions used to extract performance data
/examples/PowerPackForIBMzOSExt ension/ext/libsysinfo64.so
Native shared library for 64-bit z/OS versions used to extract performance data
/examples/PowerPackForIBMzOSExt ension/ext/zOSExtension.jar
Java class files implementing reporting through the tracer and service facilities
/examples/config/modules/zOSExtensi onManagementModule.jar
Default Management Module for Component Metrics
/core/config/zos-typical.pbl
CA APM for IBM z/OS directive files
/core/config/zos-full.pbl /core/config/zos-toggles-typical.pbd /core/config/zos-toggles-full.pbd /core/config/zos-required.pbd /core/config/zos-j2ee.pbd /core/config/zos-java2.pbd /core/config/zos-ctg.pbd /core/config/zos-mq.pbd /core/config/zos-webservices8x.pbd /core/config/zos-webservices9x.pbd /core/config/zos-sqlagent.pbd
/examples/PowerPackforIBMzOSExtens The file that enables tab views. ion/ext/xmltv/zosext.typeviewers.xml
18 for IBM z/OS Guide
How to Deploy CA APM for IBM z/OS
CA APM for IBM z/OS includes support for more extensions and other features that are included as part of CA Introscope®. CA Introscope® uses a separate PBD to support CPU Times for each extension or other feature area. Enable the PBD to view the associated metrics for that extension or feature area. The following table lists each PBD that CA Introscope® uses to support the corresponding item. Extension or feature area
Associated PBD
WebSphereMQ
zos-mq.pbd
WebServices (CA Introscope® 8.x)
zos-webservices8x.pbd
WebServices (CA Introscope® 9.x)
zos-webservices9x.pbd
CICS Transaction Gateway (CTG)
zos-ctg.pbd
SQL Agent
zos-sqlagent.pbd
The PBDs in CA APM for IBM z/OS depend on the content of the associated PBD for the target extension. Install the target extension in the CA Introscope® agent directory (/wily/) and include the PBD in the agent directives file. This requirement also applies to WebServices and SQL agent, which are included by default in the product but not automatically enabled. If you do not include the target extension PBD, ProbeBuilder fails due to missing references. Then, the agent is unable to provide metrics for either extension. By default, the new CPU Time tracers for the target extensions are turned off in the zos-full.pbl and zos-typical.pbl files. To turn on the CPU Time tracers, uncomment the associated entry in these PBL files.
How to Deploy CA APM for IBM z/OS After CA Introscope® is operational and the pre-installation requirements are met, install CA APM for IBM z/OS. CA APM for IBM z/OS is operational immediately upon the completion of its installation. After a successful installation, the performance data appears immediately in the CA Introscope® Investigator tree. For existing PowerPack for IBM z/OS users, CA APM for IBM z/OS is a direct upgrade of the ComponentAgent, with no loss in functionality. Install in a new separate directory. Then, re-direct the associated products using PowerPack for IBM z/OS to use the CA APM for IBM z/OS directory instead. Note: You can decide to run the default set of metrics for a period before customizing CA APM for IBM z/OS. This method can provide a baseline indicator for valid threshold values for your specific environment.
Chapter 2: Installing CA APM for IBM z/OS 19
Install CA APM for IBM z/OS
Note: Installation procedures for the CA APM for IBM z/OS assume that CA Introscope® is already installed and operational. Follow these steps: 1.
Install and configure CA APM for IBM z/OS (see page 20).
2.
Integrate CA APM for IBM z/OS with IBM WebSphere for z/OS (see page 22).
3.
Configure which metrics (see page 22) will be automatically enabled.
4.
Install CA APM for IBM z/OS (see page 25) Management Module on the Enterprise Manager host.
5.
Verify the installation (see page 25).
6.
(Optional) Configure CA APM for IBM z/OS (see page 27).
7.
Verify the installation (see page 25) after performing customizations.
Install CA APM for IBM z/OS Install the Introscope agent before installing CA APM for IBM z/OS. Note: For more information, see the CA APM Installation and Upgrade Guide and the CA APM Java Agent Implementation Guide. CA APM for IBM z/OS reports on a default set of Java APIs. If you want to report only on the default Java API set in the installation, you do not have to perform more configurations.
20 for IBM z/OS Guide
Install CA APM for IBM z/OS
Follow these steps: 1.
Copy the following files from the Enterprise Manager installation: ■
\examples\PowerPackForIBMzOSExtension\config\modules\MM.j ar
■
\examples\PowerPackForIBMzOSExtension\xmltv\*.xml
Put the copied files into the respective directories in the actual Enterprise Manager installation. For example, move the \examples\PowerPackForIBMzOSExtension\config\modules\MM.jar into the \config\modules directory. 2.
Copy the following files from the agent installation into the respective directories: ■
\examples\ext\zOSExtension.jar
■
\examples\ext\libsysinfo32.so
■
\examples\ext\libsysinfo64.so
Verify that the files have global read and execute permissions. Note: CA Introscope® uses the correct libsysinfoXX.so file, if you are running in 32or 64-bit mode. You can copy both the 32- and 64-bit files into the core/ext folder for the PowerPack to select the appropriate file and run. 3.
Edit the IntroscopeAgent.profile file to include necessary .PBD files in the directive property.
4.
Review the status of the following properties in the appropriate z/OS toggles file: ■
HttpAppServerAutoProbeServletTracing
■
ZosHttpAppServerAutoProbeServletTracing
If you are using the recommended JVM AutoProbe, they are both disabled as follows: #TurnOn: HttpAppServerAutoProbeServletTracing #TurnOn: ZosHttpAppServerAutoProbeServletTracing
If you are using Application Server AutoProbe, they are as follows: #TurnOn: HttpAppServerAutoProbeServletTracing TurnOn: ZosHttpAppServerAutoProbeServletTracing
Chapter 2: Installing CA APM for IBM z/OS 21
Install CA APM for IBM z/OS
Integrate CA APM for IBM z/OS with IBM WebSphere for z/OS To monitor EJBs, servlets, and other components that run under WebSphere, perform the steps in this procedure to integrate the CA APM for IBM z/OS with WebSphere for z/OS. Follow these steps: 1.
Copy the CA APM for IBM z/OS .JAR files into the /wily directory, as specified previously.
2.
Use the WebSphere Admin console and update the WebSphere regular classpath with the path to the CA APM for IBM z/OS classes (/wily directory).
3.
Put the zOSExtension.jar and libsysinfoXX.so files (where XX is 32 or 64, depending on your OS) in the /wily/core/ext directory.
4.
In the WebSphere VM’s Custom Properties configuration screen, define the com.wily.introscope.default.encoding parameter with a value of Cp1047. This formats any log information properly.
Toggle Metrics with PBD File Settings The CA APM for IBM z/OS lets a user get detailed CPU timing information for any Servlet, JSP, or EJB running under WebSphere for z/OS. By applying settings in the PBD file, you automatically switch on or off sets of metrics. Toggle the full set of metrics, or a subset of typical metrics. Toggling sets of metrics saves time over toggling individual metrics. The following procedures describe how to set up both types of PBD files. Select the appropriate procedure for using the JVM AutoProbe or the Application Server AutoProbe. Note: CA Technologies highly recommends using JVM AutoProbe as the preferred method for instrumentation.
22 for IBM z/OS Guide
Install CA APM for IBM z/OS
To apply PBD file settings for JVM AutoProbe instrumentation using -javaagent or -Xbootclasspath 1.
In the IntroscopeAgent.profile file (located in the \wily\core\config directory), edit the introscope.autoprobe.directivesFile parameter to include one of these .PBL files: ■
zos-full.pbl
To automatically instrument all of your WebSphere Servlets, JSPs, and EJBs, add the appropriate zos-full.pbl file to the introscope.autoprobe.directivesFile parameter of the WebSphere IntroscopeAgent.profile. ■
zos-typical.pbl
To automatically instrument the typical set of metrics that is used, add the appropriate zos-typical.pbl file to the introscope.autoprobe.directivesFile parameter of the WebSphere IntroscopeAgent.profile. The directives ideally already have one of the following entries:
2.
■
websphere-full.pbl
■
websphere-typical.pbl
Verify that you have the following entries in the toggles-full.pbd or toggles-typical.pbd file: TurnOn: HTTPServletTracing #TurnOn: HTTPAppServerAutoProbeServletTracing
3.
Verify that you have the following entries in the zos-toggles-full.pbd or zos-toggles-typical.pbd file: #TurnOn: ZosHTTPAppServerAutoProbeServletTracing TurnOn: ZosHTTPServletTracing
4.
(Optional) Create a customized PBD file to add more customer-defined classes or processes that the CA APM for IBM z/OS monitors for CPU usage times. For an overview, see Monitoring additional classes or processes (see page 28). Note: For more information about PBD modification, see the CA APM Java Agent Implementation Guide and the CA APM .NET Agent Implementation Guide.
5.
Restarting all managed applications reflects the new changes.
Chapter 2: Installing CA APM for IBM z/OS 23
Install CA APM for IBM z/OS
To apply PBD file settings for Application Server AutoProbe instrumentation (Plug-in method) 1.
In the IntroscopeAgent.profile file (located in the \wily\core\config directory), edit the introscope.autoprobe.directivesFile parameter to include one of these .PBL files: ■
zos-full.pbl
To automatically instrument all of your WebSphere Servlets, JSPs, and EJBs, add the appropriate zos-full.pbl file to the introscope.autoprobe.directivesFile parameter of the WebSphere IntroscopeAgent.profile. ■
zos-typical.pbl
To automatically instrument the typical set of metrics that is used, add the appropriate zos-typical.pbl file to the introscope.autoprobe.directivesFile parameter of the WebSphere IntroscopeAgent.profile. The directives ideally already have one of the following two entries:
2.
■
websphere-full.pbl
■
websphere-typical.pbl
Verify that you have the following entries in the toggles-full.pbd or toggles-typical.pbd file: TurnOn: HTTPServletTracing #TurnOn: HTTPAppServerAutoProbeServletTracing
3.
Verify that you have the following entries in the zos-toggles-full.pbd or zos-toggles-typical.pbd file: TurnOn: ZosHTTPAppServerAutoProbeServletTracing #TurnOn: ZosHTTPServletTracing
Important! The HTTPAppServerAutoProbeServletTracing flag conflicts with ZosHTTPAppServerAutoProbeServletTracing and must be disabled by commenting it out when using the CA APM for IBM z/OS. 4.
(Optional) Create a customized PBD file to add more customer-defined classes or processes that the CA APM for IBM z/OS monitors for CPU usage times. Note: For more information about PBD modification, see the CA APM Java Agent Implementation Guide and the CA APM .NET Agent Implementation Guide.
5.
24 for IBM z/OS Guide
Restarting all managed applications reflects the new changes.
Verifying the Installation
Install the CA APM for IBM z/OS Management Module and Tab Views Install the CA APM for IBM z/OS Management Module and tab views on the Enterprise Manager host computer. Follow these steps: 1.
Copy zOSExtensionManagementModule.jar to the /config/modules directory.
2.
Copy PowerPackForIBMzOSExtension..typeviewers.xml to the /ext/xmltv directory.
3.
Restart the Enterprise Manager which loads the new Management Module and tab views.
Verifying the Installation Verifying that CA APM for IBM z/OS has been successfully installed and configured requires that you verify the following items: ■
CA APM for IBM z/OS is configured properly and operational.
■
CA Introscope® displays the System Overview Service metrics.
Verifying CA APM for IBM z/OS Verify successful installation of the CA APM for IBM z/OS by viewing the metrics reported in the CA Introscope® Investigator tree. Alternatively, verify successful installation on the z/OS Java Dashboards (Java Servlets, Java Entity Beans, Java Session Beans, or JSPs). If no metrics are reported, verify the following items: ■
All components are operational (Enterprise Manager, CA Introscope® Workstation, and CA Introscope®-enabled applications are all running).
■
All installation steps have been properly completed.
Verifying the System Overview Service Metrics Verify that the CA APM for IBM z/OS is reporting the systemwide metrics in the CA Introscope® Investigator tree. If the metrics are not being reported, verify that PowerPackForIBMzOSExtension.componentagent.overview.disable=false has been set in the Introscope agent profile (located in the \wily\core\config directory).
Chapter 2: Installing CA APM for IBM z/OS 25
Uninstall CA APM for IBM z/OS
Uninstall CA APM for IBM z/OS To remove the CA APM for IBM z/OS, you must be a system administrator with the authority to remove directories. Uninstall CA APM for IBM z/OS and then delete the zOSExtension directory. Follow these steps:
26 for IBM z/OS Guide
1.
Remove the PowerPackForIBMzOSExtension.componentagent.overview.disable entry from the Introscope agent profile (located in the \wily\core\config directory) if the System Overview Service has been active.
2.
Shut down the managed application.
3.
Remove the appropriate files that are based on the installation tool that is utilized in the initial installation: ■
For an AutoProbe installation: zos-typical.pbl or zos-full.pbl file and the custom PBD file if created from the introscope.autoprobe.directivesFile property in the Introscope agent profile from the CA Introscope® installation.
■
For a ProbeBuilder Wizard or Command-Line ProbeBuilder installation: CA Introscope®-enabled .JAR files from the classpath for the application.
4.
Remove the zOSExtension.jar file from the /wily/core/ext directory under the application server (that is, WebSphere).
5.
Restart the managed application.
6.
Delete the zOSExtensionManagementModule.jar file using the CA Introscope® Workstation.
7.
Remove the zOSExtension directory.
8.
Restart the Enterprise Manager.
Chapter 3: Configuring the APM Extension for z/OS This chapter presents information about configuring CA APM for IBM z/OS to run in your z/OS environment. This section contains the following topics: CA APM for IBM z/OS Configuration (see page 27) About the System Overview Service (see page 27) Monitor Additional Classes or Processes (see page 28) Sample Files: Non-Blame and Blame Versions (see page 28)
CA APM for IBM z/OS Configuration After you change the CA APM for IBM z/OS configuration, verify the installation (see page 25). After running the default or base configuration for a period, configure CA APM for IBM z/OS to fit your operating environment better and reporting requirements. For example, you can do the following actions: ■
Use the System Overview Service function and report basic system information.
■
Create a custom tracer and monitor the CPU usage times of more classes or processes.
Note: For more information, see the CA APM Java Agent Implementation Guide.
About the System Overview Service The System Overview Service reports overall system performance data. This report allows the Java component performance to be viewed in context with the total system resources in use. The System Overview Service is always on.
Chapter 3: Configuring the APM Extension for z/OS 27
Monitor Additional Classes or Processes
Monitor Additional Classes or Processes Create a custom PBD file that monitors CPU usage of classes or processes that are not in the default set of CA APM for IBM z/OS metrics. The custom PBD file gathers the Java component level CPU timing information for instrumented components running under z/OS. Custom PBD files can operate in Blame or Normal mode. Note: In CA Introscope®, Blame mode describes tracking component interactions and component resource usage in an instrumented application. Normal mode is when Blame mode is disabled. For more information about custom PBD files, see the CA APM Java Agent Implementation Guide or contact CA Technologies Professional Services. Follow these steps:: 1.
Create a PBD file. Note: For details, see the CA APM Java Agent Implementation Guide.
2.
In the PBD file, use standard ProbeBuilder Directives to specify the classes or process that you want to monitor. Custom tracers can be in Non-Blame and Blame (see page 28) mode.
3.
Add the PBD file to your CA Introscope® installation.
4.
Verify that CA Introscope® displays the custom metrics. After the new metric is included in CA Introscope®, you can include it in a dashboard.
Sample Files: Non-Blame and Blame Versions The custom tracer operates in non-Blame or Blame mode. The two methods are mutually exclusive. In Blame mode, CA Introscope® provides a preconfigured set of metrics to use. In non-Blame mode, specify each metric individually. Here are examples of custom ProbeBuilder directives for each mode: Non-blame version: TraceOneMethodOfClass: com.sun.petstore.catalog.Catalog search ZosCpuMethodTimer "Petstore|Catalog|search:Average Method Invocation CPU Time (us)
Blame version: TraceOneMethodOfClass: com.sun.petstore.catalog.Catalog search BlamedZosCpuMethodTimer "Petstore|Catalog|search:Average Method Invocation CPU Time (us)
28 for IBM z/OS Guide
Chapter 4: Using CA APM for IBM z/OS This chapter provides information and instructions about managing and using CA APM for IBM z/OS in a z/OS environment. This section contains the following topics: View z/OS Performance Metrics in CA Introscope® (see page 29) Alerts (see page 30)
View z/OS Performance Metrics in CA Introscope® Enterprise Manager must be running on a distributed machine (Windows, AIX, Windows) to view the performance data that CA APM for IBM z/OS provides. The CA APM for IBM z/OS default configuration determines the process and agent names. Illustrations in this section show some default metrics sets provided with the CA APM for IBM z/OS. If your CA Introscope® administrator has customized the CA APM for IBM z/OS metrics (see page 43) for your current environment, the Workstation displays those environment-specific metrics also. The following options let you view data from the CA APM for IBM z/OS in CA Introscope®: ■
Raw metrics in the Investigator—This option provides you a technical view of the underlying performance of all resources and components of the z/OS computer.
■
Tab views in the Investigator—This option provides you aggregate views of performance and resources and components of the z/OS system.
■
Dashboards in the Console—This option provides an easy-to-use interface for users who lack familiarity with details of the z/OS architecture.
■
Alerts in the Investigator—This option shows alerts that CA APM for IBM z/OS dashboards generate and alerts you create.
Follow these steps: 1.
Start your managed application.
2.
Launch Enterprise Manager.
Chapter 4: Using CA APM for IBM z/OS 29
Alerts
3.
Launch and log in to the Workstation.
4.
Open an Investigator window. All the CA APM for IBM z/OS-specific metrics appear under several nodes in the metric browser tree. Note: The metrics (see page 43) that you see depend on the resources that your applications use. Only those metrics managed by your Java applications appear.
Analyzing Historical Data for a Specific Metric To analyze historical performance data for a specific metric, set up a Persistent Collection. Note: For information about setting up a Persistent Collection, see the CA APM Configuration and Administration Guide.
Alerts Alerts are set with default caution/danger thresholds for many of the performance metrics displayed in the dashboards. Two types of thresholds exist: Warning and Danger. A metric that exceeds a Warning or Danger threshold triggers an alert. What makes a Danger threshold value different from a Warning are the value and severity settings for each. Values are per 15-second interval. The z/OS CPU Times Overview dashboard contains alerts with the following threshold values: Alert
Warning Threshold Value
Danger Threshold Value
Java Entity Beans Alert
300,000 µ(0.3 second)
500,000 µ (0.5 second)
Java Session Beans Alert
300,000 µ (0.3 second)
500,000 µ (0.5 second)
Java Server Pages Alert
300,000 µ (0.3 second)
500,000 µ (0.5 second)
Java Servlets Alert
300,000 µ (0.3 second)
500,000 µ (0.5 second)
Note: For information about customizing these alerts and thresholds for your environment, see the CA APM Workstation User Guide.
30 for IBM z/OS Guide
Chapter 5: Using the CA Introscope® Console and z/OS Dashboards This section contains the following topics: Overview Dashboard (see page 31) The CA Introscope® Console and z/OS Dashboards (see page 31) Java Entity Beans Dashboard (see page 32) Java Server Pages Dashboard (see page 32) Java Servlets Dashboard (see page 33) Java Session Beans Dashboard (see page 33)
Overview Dashboard The z/OS CPU Times Overview dashboard provides general system information and system status. The z/OS CPU Times Overview dashboard is the first line of defense in monitoring system performance. When a metric exceeds an alert threshold, the z/OS CPU Times Overview dashboard directs you to related dashboards. These dashboards provide resource usage details.
The CA Introscope® Console and z/OS Dashboards CA APM for IBM z/OS provides a number of preconfigured dashboards for displaying various z/OS performance metrics. You can see data in some or all of the dashboards, depending on the type of resources the monitored applications use. All of the dashboards for the CA APM for IBM z/OS begin with Extension for z/OS to distinguish them from dashboards for any other Management Modules installed in CA Introscope®. Consult CA Technologies Professional Services about customizing these dashboards for use in your environment. The primary CA APM for IBM z/OS dashboards are as follows: Primary Dashboard
Description
Extension for z/OS Overview
General system status
Extension for z/OS Java Entity Beans
Java Entity Bean activity
Extension for z/OS Java JSPs
Java JSP activity
Chapter 5: Using the CA Introscope® Console and z/OS Dashboards 31
Java Entity Beans Dashboard
Primary Dashboard
Description
Extension for z/OS Java Servlets
Java Servlet activity
Extension for z/OS Java Session Beans
Java Session Bean activity
The CA Introscope® Workstation console displays the CA APM for IBM z/OS dashboards.
Java Entity Beans Dashboard The z/OS Java Entity Beans dashboard provides information about the current ten highest CPU-using Java Entity Beans and their average over time. Use the z/OS Java Entity Beans dashboard when the z/OS CPU Times Overview dashboard indicates that an Entity Bean has exceeded the alert threshold. Use this dashboard for monitoring the impact of new Entity Beans and determine capacity and resource requirements. During periods of low activity, the CPU time values can be zero. In this case, CA Introscope® lists the Java Entity Beans in order of first in, first out (FIFO).
Java Server Pages Dashboard The z/OS Java Server Pages dashboard provides information about the current ten highest CPU-using Java Server Pages and their average over time. Use the z/OS Java Server Pages Dashboard when the z/OS CPU Times Overview dashboard indicates that a JSP has exceeded the alert threshold. Use this dashboard when monitoring the impact of new JSPs. During periods of low activity, the CPU time values can be zero. In this case, CA Introscope® lists the Java Server Pages in order of first in, first out (FIFO).
32 for IBM z/OS Guide
Java Servlets Dashboard
Java Servlets Dashboard The z/OS Java Servlet dashboard provides information about the current ten highest CPU-using Java servlets and their average over time. Use the z/OS Java Servlet dashboard when the z/OS CPU Times Overview dashboard indicates that a servlet has exceeded the alert threshold. Use this dashboard and monitor the impact of new servlets. Note: In the Investigator tree, the initial CPU times for a servlet can be high due to the initial Just-in-Time (JIT) compiling. The initial high CPU times proceed much lower values. These values can affect the scale in such a way that the subsequent lower values all appear to be merely above zero. After the high value from the initial Just-in-Time compiling has scrolled off the display, the scale returns to normal. The CPU time values can be zero during periods of low activity. In this case, CA Introscope® lists the Java servlets in order of first in, first out (FIFO).
Java Session Beans Dashboard The z/OS Java Session Beans dashboard provides information about the current 10 highest CPU using Java Session Beans and their average over time. Use the z/OS Java Session Beans dashboard when the z/OS CPU Times Overview dashboard indicates that a Session Bean has exceeded the alert threshold. Use this dashboard for monitoring the impact of new Session Beans. During periods of low activity, the CPU time values can be zero. In this case, CA Introscope® lists the Java Session Beans in order of first in, first out (FIFO).
Chapter 5: Using the CA Introscope® Console and z/OS Dashboards 33
Chapter 6: Using CA Introscope® Investigator Tab Views You can change the view that is displayed in the CA Introscope® Investigator viewer pane by using the tabs at the top of the pane. Many of the z/OS-specific nodes on the tree have special prebuilt summary views. This section contains the following topics: CA Introscope® Investigator Tab Views (see page 35) Servlets Graphical Tab View (see page 36) Servlets Tabular Tab View (see page 36) JSP CPU Times Tab View (see page 36) EJB Graphical Tab View (see page 37) EJB Tabular Tab View (see page 37) DB2 Backends Graphical Tab View (see page 37) DB2 Backends Tabular Tab View (see page 38) JDBC Tabular Tab View (see page 38) WebServices CPU Time Aggregates Client Tab View (see page 38) WebServices CPU Time Aggregates Server Tab View (see page 38) WebServices Engine Handler Tab View (see page 39) Process Overview Tab View (see page 39) System Overview Tab View (see page 39) WebSphereMQ Graphical Tab View (see page 40) WebSphereMQ Tabular Tab View (see page 40) CTG Frontends CPU Time Graphical Tab View (see page 41) CTG Frontends CPU Time Tabular Tab View (see page 41) CTG Backends CPU Time Graphical Tab View (see page 41) CTG Backends CPU Time Tabular Tab View (see page 42)
CA Introscope® Investigator Tab Views The following tabs are standard and available in the CA Introscope® Workstation, regardless of the extensions you use. ■
General—The General tab is the default tab that displays when you select any item below an agent in the CA Introscope® Investigator metric browser tree. When you select a metric, the General tab is a visualization of the metric—either live data or for a selected historical period. For nodes in the tree, the General tab shows the path to that node object in the CA Introscope® Investigator hierarchy.
■
Overview—The Overview tab presents high-level health indicators and a log of related events and historical metric information. This tab is available when you select an agent in the CA Introscope® Investigator metric browser tree, enables application monitoring.
Chapter 6: Using CA Introscope® Investigator Tab Views 35
Servlets Graphical Tab View
■
Search—The Search tab allows you to find metrics quickly. This tab is available when you select a node in the CA Introscope® Investigator metric browser tree that contains metrics.
■
Trace—The Trace tab lists the Transaction Traces in which the currently selected resource or component participated. This tab is similar to the Trace Viewer and is available when you select a resource or component in the CA Introscope® Investigator metric browser tree.
■
Error—The Error tab lists errors and error detail for the selected item. This error displays when you select a resource or component in the CA Introscope® Investigator metric browser tree.
Note: For more information about standard CA Introscope® tabs, see the CA APM Workstation User Guide.
Servlets Graphical Tab View When you click a Servlet name under the Servlets node, CA Introscope® Investigator displays the CPU Time Graphic tab, showing that servlet’s CPU Time related metrics: ■
CPU Timer used: CPU time that is executed on the main z/OS CP processor
■
zAAP Time used: CPU time that is executed on the zAAP processor
■
zIIP Time used: CPU time that is executed on the zIIP processor
Servlets Tabular Tab View When you click the Servlets node, CA Introscope® Investigator displays the Servlet CPU Time Tabular Summary tab. These results display a list of all the servlets, and the following CPU Time related metrics: ■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
JSP CPU Times Tab View When you click the JSP node, CA Introscope® Investigator displays the JSP CPU Time Tabular Summary tab, showing the following CPU Time-related metrics:
36 for IBM z/OS Guide
■
CPU Timer usec: CPU Time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
EJB Graphical Tab View
EJB Graphical Tab View When you click an EJB name under the EJB node, CA Introscope® Investigator displays the CPU Time Graphic tab, showing that EJB's CPU Time related metrics: ■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
EJB Tabular Tab View When you click the Entity, Session, or Message-Driven Bean node under the EJB node, CA Introscope® Investigator displays the EJB Session CPU Time Tabular Summary tab. This tab displays a summary list of all the Entity, Session, or Message Driven EJBs, and their CPU Time related metrics: ■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
DB2 Backends Graphical Tab View Opening the DB2 node and selecting the individual method (CreateStatement, Execute, ExecuteQuery, ExecuteUpdate, GetResultSet, PrepareStatement) displays the DB2 backend graphs. The CA Introscope® Investigator displays a set of graphs showing the CPU Times for each of the major methods that Java DB2 application invokes. The following CPU Time related metrics are displayed for each primary method: ■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
Chapter 6: Using CA Introscope® Investigator Tab Views 37
DB2 Backends Tabular Tab View
DB2 Backends Tabular Tab View Select the DB2 node and CA Introscope® Investigator displays a summary table showing the CPU Times for each of the major methods Java DB2 application invoke. This summary table shows the CPU overhead for each of the major methods. The following CPU Time related metrics are displayed: ■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
JDBC Tabular Tab View Select the JDBC node and CA Introscope® Investigator displays a summary table showing the summary CPU Times for each JDBC driver (DB2 Universal, Derby, and so forth). CA Introscope® Investigator displays the following CPU Time-related metrics: ■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
WebServices CPU Time Aggregates Client Tab View Select the Server node under the WebServices|CPU Time Aggregates node of the tree and CA Introscope® Investigator displays the Web Services CPU Time Graphical Summary tab. This graph shows the following CPU Time related metrics: ■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
WebServices CPU Time Aggregates Server Tab View Select the Server node under the WebServices|CPU Time Aggregates node of the tree and CA Introscope® Investigator displays the Web Services CPU Time Graphical Summary tab. This graph shows the following CPU Time related metrics:
38 for IBM z/OS Guide
■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
WebServices Engine Handler Tab View
WebServices Engine Handler Tab View Select the WebServices Engine node under the WebServices|CPU Time Aggregates tree node and CA Introscope® Investigator displays the Web Services CPU Time Graphical Summary tab. This graph shows the following CPU Time related metrics: ■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
Process Overview Tab View When you open the Process Overview node, the CA Introscope® Investigator displays a set of graphs showing the key information for the currently running Java Process. For example, if the CA APM for IBM z/OS is running inside WebSphere, CA Introscope® Investigator displays the process information for WebSphere. If CA APM for IBM z/OS is running inside CTG, it displays the process information for the CTG Server. The primary fields that display on the Process Overview view are as follows: ■
Total cumulative CPU time (on a CP engine) the process uses.
■
Latest delta of CPU time (on a CP engine) the process uses in the last interval.
■
Total cumulative zAAP CPU time the process uses.
■
Latest delta of zAAP CPU time the process uses in the last interval.
■
The maximum amount of OS-related memory the process that is allocated below the 16-MB line.
■
The maximum amount of OS-related memory the process that is allocated above the 16-MB line.
■
The maximum amount of User (heap) related memory the process that is allocated below the 16-MB line.
■
The maximum amount of User (heap) related memory the process that is allocated above the 16-MB line.
System Overview Tab View When you open the System Overview node, the CA Introscope® Investigator displays a set of graphs showing the following z/OS System Level information: ■
The overall average CPU use in the system
■
The current MSU capacity of the system
Chapter 6: Using CA Introscope® Investigator Tab Views 39
WebSphereMQ Graphical Tab View
■
The number of active main (CP engine) CPUs
■
The number of online/active zAAP processors
■
The number of online/active zIIP processors
■
The name of the system
■
The ID of the system (SYS ID)
WebSphereMQ Graphical Tab View Open the WebSphereMQ node's Connector_CPU_Time node or the JMS_CPU_Time node and select the individual method (Get, Put, Send, Receive) to display a WebSphereMQ graph. CA Introscope® Investigator displays a set of graphs showing the CPU Times for each of the major methods that invoke a Java MQ application. The following CPU Time related metrics display for each primary method: ■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
WebSphereMQ Tabular Tab View Select the WebSphereMQ node's Connector_CPU_Time node or the JMS_CPU_Time node and a WebSphereMQ graph displays. CA Introscope® Investigator displays a summary table showing the CPU Times for each of the major methods which invoke a Java MQ application. For the Connector_CPU_Time node, it shows the method times for standard MQ Puts and Gets. These MQ Puts and Gets issue either to the Queue Manager or to the Queue itself. For the JMS_CPU_Time node, it shows the method times for standard JMS sends and receives. The CA Introscope® Investigator displays the following CPU Time-related metrics:
40 for IBM z/OS Guide
■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
CTG Frontends CPU Time Graphical Tab View
CTG Frontends CPU Time Graphical Tab View Open the Client_to_CTG Aggregates node's CPU Time Aggregates node and select an individual method (JCA_ECI_Execute, JCA_ECI_Open, TCP Flow, and so forth) to display a CPU graph. CA Introscope® Investigator displays a set of graphs showing the CPU Times for each of the major methods that invoke a Java CTG application. The following CPU Time related metrics are displayed for each primary method: ■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
CTG Frontends CPU Time Tabular Tab View Select the Client_to_CTG Aggregates node's CPU Time Aggregates node and CA Introscope® Investigator displays a summary table. This table shows the CPU Times for each of the major methods which invoke a Java CTG application. CA Introscope® Investigator provides a summary table, showing the CPU overhead for each of the major methods. CA Introscope® Investigator displays following CPU Time related metrics: ■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
CTG Backends CPU Time Graphical Tab View Open the CTG_to_CICS_ECI_IPIC node's CPU Time Aggregates node and select an individual method (Execute, Read Packet, Write Packet) to display a graph. CA Introscope® Investigator displays a set of graphs showing the CPU Times to process CTG requests being sent to/from CICS. CA Introscope® Investigator displays following CPU Time related metrics for each primary method: ■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
Chapter 6: Using CA Introscope® Investigator Tab Views 41
CTG Backends CPU Time Tabular Tab View
CTG Backends CPU Time Tabular Tab View Select the CTG_to_CICS_ECI_IPIC node's CPU Time Aggregates node and CA Introscope® Investigator displays a summary table. This table shows the CPU Times to process CTG requests being sent to/from CICS. CA Introscope® Investigator provides a summary table, showing the CPU overhead for each of the major methods, and displays the following CPU Time-related metrics:
42 for IBM z/OS Guide
■
CPU Timer usec: CPU time that is executed on the main z/OS CP processor
■
zAAP Time usec: CPU time that is executed on the zAAP processor
■
zIIP Time usec: CPU time that is executed on the zIIP processor
Appendix A: CA APM for IBM z/OS Metrics This appendix outlines the metrics CA APM for IBM z/OS collects. The metrics are organized within the groups. This section contains the following topics: Tracer Metric (see page 43) Process Overview Service Metrics (see page 43) System Overview Service Metrics (see page 44) CPU Usage Metrics - J2EE Based (see page 46) CPU Usage Metrics - SQL Agent (see page 48) CPU Usage Metrics - Web Services (see page 49) CPU Usage Metrics - Web Services Unique to CA Introscope® (see page 50) CPU Usage Metrics - WebSphere MQ (see page 51) CPU Usage Metrics - CTG (see page 52)
Tracer Metric The resource structure is the same as the corresponding components in the CA Introscope® PBDs and suffixed with the metric name: CPU Time (ms). Actual (Central CP) CPU time used.
Process Overview Service Metrics Process Overview|Process ASCB id (Hex):{derived from control block} Address space identifier for job (hex number) Process Overview|Process Job Name:{derived from control block} Name of job that this process is running under Process Overview|Process SRB CPU Time Cumulative (ms):{derived from control block} Cumulative job/process CPU SRB time used
Appendix A: CA APM for IBM z/OS Metrics 43
System Overview Service Metrics
Process Overview|Process TCB CPU Time Cumulative (ms):{derived from control block} Cumulative job/process CPU TCB time used Process Overview|Process Total CPU Time Cumulative (ms):{derived from control block} Cumulative job/process total CPU time used Process Overview|Process OS Memory Allocated above 16-MB (kb) High water mark:{derived from control block} System Area memory (ECSA,ESQA,ELSQA) for job/process above 16-MB line (High water mark) Process Overview|Process OS Memory Allocated below 16-MB (kb) High water mark:{derived from control block} System Area memory (CSA,SQA,LSQA) for job/process below 16-MB line (High water mark) Process Overview|Process User Memory Allocated above 16-MB (kb) High water mark:{derived from control block} Private Area memory for job/process above 16-MB line (High water mark) Process Overview|Process User Memory Allocated below 16-MB (kb) High water mark:{derived from control block} Private Area memory for job/process below 16MB line (High water mark)
System Overview Service Metrics System Overview|LPAR Name:{derived from control block} Name of LPAR under which job/process is running. System Overview|Operating System:{derived from control block} Name of operating system under which job/process is running (OS/390, z/OS, ...). System Overview|Operating System Version Level:{derived from control block} Version of operating system under which job/process is running (z/OS). System Overview|Sys Id:{derived from control block} Four character System Identifier. System Overview|Sysplex Name:{derived from control block} Name of sysplex on which this system resides. System Overview|System Approximate MIPs Capacity:{derived from control block} Approximate processor capacity in MIPS (Millions of Instructions Per Second).
44 for IBM z/OS Guide
System Overview Service Metrics
System Overview|System Average CPU Utilization (%):{derived from control block} Average System CPU utilization over last five intervals %. System Overview|System MSU Capacity:{derived from control block} Approximate processor capacity in MSUs (Millions of Service Units). System Overview|LPAR Name:{derived from control block} Name of LPAR under which job/process is running. System Overview|Operating System:{derived from control block} Name of operating system under which job/process is running (OS/390, z/OS, ...). System Overview|Operating System Version Level:{derived from control block} Version of operating system under which job/process is running (z/OS). System Overview|Sys Id:{derived from control block} Four character System Identifier. System Overview|Sysplex Name:{derived from control block} Name of sysplex on which this system resides. System Overview|System Approximate MIPs Capacity:{derived from control block} Approximate processor capacity in MIPS (Millions of Instructions Per Second). System Overview|System Average CPU Utilization (%):{derived from control block} Average System CPU utilization over last five intervals %. System Overview|System MSU Capacity:{derived from control block} Approximate processor capacity in MSUs (Millions of Service Units). System Overview|LPAR Name:{derived from control block} Name of LPAR under which job/process is running. System Overview|Operating System:{derived from control block} Name of operating system under which job/process is running (OS/390, z/OS, ...). System Overview|Operating System Version Level:{derived from control block} Version of operating system under which job/process is running (z/OS). System Overview|Sys Id:{derived from control block} Four character System Identifier. System Overview|Sysplex Name:{derived from control block} Name of sysplex on which this system resides. System Overview|System Approximate MIPs Capacity:{derived from control block} Approximate processor capacity in MIPS (Millions of Instructions Per Second).
Appendix A: CA APM for IBM z/OS Metrics 45
CPU Usage Metrics - J2EE Based
System Overview|System Average CPU Utilization (%):{derived from control block} Average System CPU utilization over last five intervals %. System Overview|System MSU Capacity:{derived from control block} Approximate processor capacity in MSUs (Millions of Service Units). System Overview|LPAR Name:{derived from control block} Name of LPAR under which job/process is running. System Overview|Operating System:{derived from control block} Name of operating system under which job/process is running (OS/390, z/OS, ...). System Overview|System Name:{derived from control block} Name of System. System Overview|System Number of Active CPUs:{derived from control block} Current Number of Online CPUs, zIIPs, and zAAPs. System Overview|System Number of Online zAAPs:{derived from control block} System-wide Number of online zAAPs. System Overview|System Number of Online zIIPs:{derived from control block} System-wide Number of online zIIPs. System Overview|System Paging Rate (pages/sec) :{derived from control block} System-wide paging rate.
CPU Usage Metrics - J2EE Based Servlets:Average Method Invocation CPU Time (us) Aggregate CPU time for all servlets Servlets|{classname}:Average Method Invocation CPU Time (us) CPU time for a specific Servlet JSP:Average Method Invocation CPU Time (us) Aggregate CPU time for all JSPs JSP|{classname}:Average Method Invocation CPU Time (us) CPU time for a specific JSP JSP TagLib:Average Method Invocation CPU Time (us) Aggregate CPU time for all Taglib components
46 for IBM z/OS Guide
CPU Usage Metrics - J2EE Based
JSP TagLib|{classname}|{method} Average Method Invocation CPU Time (us) CPU time for a specific Taglib component EJB|Message-driven:Average Method Invocation CPU Time (us) Aggregate CPU time for all MDBs EJB|Message-driven|{classname}: Average Method Invocation CPU Time (us) CPU time for a specific MDB EJB|Message-driven|{classname}| {method}:Average Method Invocation CPU Time (us) CPU time for a specific MDB method call EJB|Session:Average Method Invocation CPU Time (us) Aggregate CPU time for all EJB Session Beans EJB|Session|{classname}:Average Method Invocation CPU Time (us) CPU time for a specific EJB Session Bean class invocation EJB|Session|{classname}|{method} Average Method Invocation CPU Time (us) CPU time for a specific EJB Session Bean method call EJB|Entity:Average Method Invocation CPU Time (us) Aggregate CPU time for all EJB Entity Beans EJB|Entity|{classname}:Average Method Invocation CPU Time (us) CPU time for a specific EJB Entity Bean class invocation EJB|Entity|{classname}|{method}: Average Method Invocation CPU Time (us) CPU time for a specific EJB Entity Bean method call CORBA|{classname}:Average Method Invocation CPU Time (us) CPU time for a specific CORBA class Struts|{classname}|{method}:Average Method Time (us) CPU time for a specific Struts method call XML|SAX|{classname}:Average Method Invocation CPU Time (us) CPU time for a specific XML SAX class XML|XSLT|{classname}:Average Method Invocation CPU Time (us) CPU time for a specific XML XLST class JTA|{classname}:Average Method Invocation CPU Time (us) CPU time for a specific JTA class
Appendix A: CA APM for IBM z/OS Metrics 47
CPU Usage Metrics - SQL Agent
JNDI|Context|{classname}:Average Method Invocation CPU Time (us) CPU time for a specific JNDI class Java Mail|Transport|{classname}: Average Method Invocation CPU Time (us) CPU time for a specific Mail Transport class J2EE Connector|{classname}:Average Method Invocation CPU Time (us) CPU time for a specific J2EE Connector class JMS|Message Listener|{classname}: Average Method Invocation CPU Time (us) CPU time for a specific JMS Listener class JMS|Message Consumer|{classname}: Average Method Invocation CPU Time (us) CPU time for a specific JMS Consumer class JMS|Topic Publisher|{classname}: Average Method Invocation CPU Time (us) CPU time for a specific JMS Publisher class JMS|Queue Sender|{classname}:Average Method Invocation CPU Time (us) CPU time for a specific JMS Queue Sender class RMI|Client|{classname}|{method}: Average Method Invocation CPU Time (us) CPU time for a specific RMI Client class RMI|Server|{classname}|{method}: Average Method Invocation CPU Time (us) CPU time for a specific RMI Server class JDBC|{drivername}:Average Method Invocation CPU Time Query (us) Aggregate CPU time for all JDBC Execute Query calls JDBC|{drivername}:Average Method Invocation CPU Time Update (us) Aggregate CPU time for all JDBC Execute Update calls
CPU Usage Metrics - SQL Agent Backends|DB2|CreateStatement: Average Method Invocation CPU Time (us) CPU time for a JDBC CreateStatement call Backends|DB2|PrepareStatement: Average Method Invocation CPU Time (us) CPU time for a JDBC PrepareStatement call Backends|DB2|PrepareCall: Average Method Invocation CPU Time (us) CPU time for a JDBC Prepare call
48 for IBM z/OS Guide
CPU Usage Metrics - Web Services
Backends|DB2|Commits: Average Method Invocation CPU Time (us) CPU time for a JDBC Commit call Backends|DB2|Rollbacks: Average Method Invocation CPU Time (us) CPU time for a JDBC Rollback call Backends|DB2|Execute: Average Method Invocation CPU Time (us) CPU time for a JDBC Execute call Backends|DB2|ExecuteQuery: Average Method Invocation CPU Time (us) CPU time for a JDBC ExecuteQuery Backends|DB2|ExecuteUpdate: Average Method Invocation CPU Time (us) CPU time for a JDBC ExecuteUpdate Backends|DB2|GetResultSet: Average Method Invocation CPU Time (us) CPU time for a JDBC GetResultSet call Backends|DB2|GetStatement: Average Method Invocation CPU Time (us) CPU time for an SQLJ GetStatement Backends|DB2|ExecuteRTQuery: Average Method Invocation CPU Time (us) CPU time for an SQLJ ExecuteRTQuery
CPU Usage Metrics - Web Services WebServices|Client|Invoke: Average Method Invocation CPU Time (us) CPU time for a Web Services Client static Invoke (two way exchange) call. WebServices|Client|InvokeOneWay: Average Method Invocation CPU Time (us) CPU time for a Web Services Client static Invoke (one way send only) call. WebServices|Client|InvokeEngine: Average Method Invocation CPU Time (us) CPU time for a Web Services Client to run through the common Web Services invocation request logic. WebServices| WebServicesEngine Handler|Invoke: Average Method Invocation CPU Time (us) CPU time for a Web Services Client or Server running through the Web Services engine (run-time support). WebServices|Server|Invoke: Average Method Invocation CPU Time (us) CPU time for a Web Services Server to process an incoming Invoke (two-way exchange) call.
Appendix A: CA APM for IBM z/OS Metrics 49
CPU Usage Metrics - Web Services Unique to CA Introscope®
CPU Usage Metrics - Web Services Unique to CA Introscope® WebServices|Server|InvokeBusinessLogic Average Method Invocation CPU Time (us) CPU time for a Web Services Server to process generic BusinessLogic calls. WebServices|Client| JAX-WS|Invoke:Average Method Invocation CPU Time (us) CPU time for a Web Services Client static Invoke (two-way exchange) call using JAX-WS APIs. WebServices|Client| JAX-WS| InvokeOneWay:Average Method Invocation CPU Time (us) CPU time for a Web Services Client static Invoke (one-way send only) call using JAX-WS APIs. WebServices|Server |JAX-WS|Receive: Average Method Invocation CPU Time (us) CPU time for a Web Services Server to perform basic receive processing of an incoming JAX-WS request. WebServices|Server |JAX-WS|Invoke:Average Method Invocation CPU Time (us) CPU time for a Web Services Server to process an incoming Invoke (two way exchange) call using JAX-WS. WebServices|Server |JAX-WS| InvokeOneWay:Average Method Invocation CPU Time (us) CPU time for a Web Services Server to process an incoming Invoke (one way exchange) call using JAX-WS. WebServices|Server |JAX-WS| InvokeAsync:Average Method Invocation CPU Time (us) CPU time for a Web Services Server to process an asynchronous Invoke call using JAX-WS. WebServices|Client |JAX-WS HTTP| ExecuteMethod:Average Method Invocation CPU Time (us) CPU time for a Web Services Client to process an ExecuteMethod call using JAX-WS HTTP support. WebServices|Client |JAX-WS HTTP| Send:Average Method Invocation CPU Time (us) CPU time for a Web Services Client to process a Send call using JAX-WS HTTP support. WebServices|Client|Execute:Average Method Invocation CPU Time (us) CPU time for a Web Services Client to process an Execute call.
50 for IBM z/OS Guide
CPU Usage Metrics - WebSphere MQ
CPU Usage Metrics - WebSphere MQ WebSphereMQ|Connector|Put|Queue Put: Average Method Invocation CPU Time (us) CPU time for an MQ Put call. WebSphereMQ|Connector|Put|Queue Put1: Average Method Invocation CPU Time (us) CPU time for an MQ Put2 call. WebSphereMQ|Connector|Get|Queue Get: Average Method Invocation CPU Time (us) CPU time for an MQ Get call. WebSphereMQ|Connector|Get|Queue Get1: Average Method Invocation CPU Time (us) CPU time for an MQ Get2 call. WebSphereMQ|JMS|Send|Producer Send: Average Method Invocation CPU Time (us) CPU time for a MQ/JMS Send call by a Producer application. WebSphereMQ|JMS|Send|Publish: Average Method Invocation CPU Time (us) CPU time for a MQ/JMS Publish call by a Publish/Subscribe application. WebSphereMQ|JMS|Receive|Consumer Receive: Average Method Invocation CPU Time (us) CPU time for a MQ/JMS Receive call by a Consumer application. WebSphereMQ|JMS|Receive|Get Topic: Average Method Invocation CPU Time (us) CPU time for a MQ/JMS Receive call by a Publish/Subscribe application. WebSphereMQ|JMS|Receive|Async Receive: Average Method CPU Time (us) CPU time for a MQ/JMS Asynchronous Receive by a Publish/Subscribe application. WebSphereMQ|JMS|Receive|Receive No Wait: Average Method Invocation CPU Time (us) CPU time for a MQ/JMS non blocking Receive by a Publish/Subscribe application.
Appendix A: CA APM for IBM z/OS Metrics 51
CPU Usage Metrics - CTG
CPU Usage Metrics - CTG Frontends|Client_to_CTG_Aggregates|CPU Time Aggregates|JCA_ECI| Execute: Average Method Invocation CPU Time (us) CPU time for a CTG Client Execute call. Frontends|Client_to_CTG_Aggregates|CPU Time Aggregates|JCA_ECI|Open Connection: Average Method Invocation CPU Time (us) CPU time for a CTG Client Open of a JCA connection call. Frontends|Client_to_CTG_Aggregates|CPU Time Aggregates|TCP Opens: Average Method Invocation CPU Time (us) CPU time for a CTG Client Open of a TCP connection call. Frontends|Client_to_CTG_Aggregates CPU Time Aggregates|SSL Opens: Average Method Invocation CPU Time (us) CPU time for a CTG Client Open of an SSL connection call. Frontends|Client_to_CTG_Aggregates|CPU Time Aggregates|Local EXCI Opens: Average Method Invocation CPU Time (us) CPU time for a CTG Client Open of a local EXCI connection to CICS. Frontends|Client_to_CTG_JSSE Session|CPU Time Aggregates|SSL Handshakes: Average Method Invocation CPU Time (us) CPU time for a CTG Client to process an SSL handshake sequence. Frontends|Client_to_CTG_Aggregates|TCP flow: Average Method Invocation CPU Time (us) CPU time for a CTG Client flow of a request to CICS over TCP. Frontends|Client_to_CTG_Aggregates|SSL flow: Average Method Invocation CPU Time (us) CPU time for a CTG Client flow of a request to CICS over SSL. Frontends|Client_to_CTG_Aggregates| Local EXCI flow: Average Method Invocation CPU Time (us) CPU time for a CTG Client flow of a request to CICS over EXCI. Backends|CTG_to_CICS_ECI_IPIC| CPU Time Aggregates|Execute: Average Method Invocation CPU Time (us) CPU time for a CTG Server execute of a request to CICS. Backends|CTG_to_CICS_ECI_IPIC| CPU Time Aggregates|Write Packet: Average Method Invocation CPU Time (us) CPU time for a CTG Server to setup and write a request packet to CICS.
52 for IBM z/OS Guide
CPU Usage Metrics - CTG
Backends|CTG_to_CICS_ECI_IPIC|CPU Time Aggregates|Read Packet: Average Method Invocation CPU Time (us) CPU time for a CTG Server to receive and process a reply packet from CICS. Backends|CTG_Global_Statistics|Worker Threads: Average Method Invocation CPU Time (us) CPU time for a CTG Server Worker thread to process a request. Backends|CTG_Global_Statistics|ConnectionManager Threads: Average Method Invocation CPU Time (us) CPU time for a CTG Server Connection thread to process a request. Backends|CTG_Global_Statistics|ConnectionManager: Average Method Invocation CPU Time (us) CPU time for a CTG Server to process a new request for either a connection or a data flow.
Appendix A: CA APM for IBM z/OS Metrics 53
Appendix B: Troubleshooting This appendix presents troubleshooting guidelines to find solutions for problems you can encounter when installing, customizing, or using CA APM for IBM z/OS. If your application does not use all the resources/components described in, those performance metrics will not appear in the dashboards. Consult CA Technologies Professional Services for customizing the dashboard and the extension for your application environment. This section contains the following topics: Management Module is Not Installed (see page 55) Dashboards Have Empty Panels (see page 55)
Management Module is Not Installed Symptom: The CA APM for IBM z/OS Management Module is not installed. Solution: The administrator does not have sufficient user privileges to move or copy the zOSExtensionManagementModule.jar file. Log on with a user ID that has the authority to move and copy directories.
Dashboards Have Empty Panels Symptom: Dashboards have empty panels. Solution: You are seeing empty panels because your application does not use those metrics. The CA APM for IBM z/OS dashboards display a general set of performance metrics by default.
Appendix B: Troubleshooting 55
