BMC Mainframe: z/OS Diagnostics & Debugging
The course is developed and delivered by © RSM Technology.
This course provides attendees with an effective and systematic approach to z/OS problem diagnosis. In the course the z/OS software environment is examined by looking at the Recovery Termination Manager (RTM) - the 'cleaning-up' function of MVS - and its ABEND-concept. All the different reports that come out of a z/OS system, in conjunction with failures (messages, dumps, traces, etc.) are also discussed. System ABENDs, and how you can analyze the information coming out of the system when they occur, is also covered.
Attendees will learn how to identify system problems promptly, in order to provide greater system availability. The course focuses on a debugging methodology using IPCS. Practical workshops provide an opportunity to learn to debug system problems in realistic situations.
Major release:
BMC Mainframe Infrastructure Platform Training
Good for:
System Programmers
Course Delivery:
Instructor-Led Training (ILT) | 40 hours
Course Modules
-
Recovery Basics
- Normal Program Termination
- EXIT (SVC 3)
- Abnormal program termination
- Program Checks
- System forced ABEND
- Program ABEND
- Why abnormal termination?
- Logical application error
- Program incomplete
- Application detected software error
- System detected software error
- Hardware detected software error
- PC FLIH and ABENDs
- Hardware detected software error example
- Program Checks in the Supervisor
- Hardware problems
- RTM actions
- Recovery
- Functional Recovery Routines (FRRs)
- Extended Specify Task Abnormal Exit (ESTAE)
- System breakdown
- Software problem types
- Review questions
-
z/OS Error Reporting & Dumps
- System error reporting
- MVS dumps
- Stand-Alone Dump (SADUMP)
- SVC dumps
- user ABEND dumps
- SYSUDUMP
- SYSABEND
- SYSMDUMP
- CEEDUMP
- Generating a user ABEND dump
- System generated ABEND dump
- Snap dumps
- Symptom dumps
- Review questions
-
ABEND Analysis
- What is ABEND?
- The MVS ABEND service
- Why ABEND?
- Allows for recovery routines
- Task termination
- Tasks in an Address Space
- How RTM is invoked
- Program checks
- ABEND
- How to trigger an ABEND
- ABEND macro and SVC 13
- CALLRTM macro
- Why not normal end?
- Application detected software errors
- System detected software errors
- All the system ABEND codes
- Where do you see the ABEND codes?
- The NOTIFY message
- The SYSLOG
- The job log
- The symptom dump
- ABEND dumps
- SVC dumps
- Stand-Alone dumps
- The symptom dump in the SYSLOG
- The symptom dump in the job log
- Explanations of ABEND and reason codes
- IBM z/OS manuals on the web
- Quickref and similar tools
- Analysis approach
- Examples of ABEND code explanation
- System messages - a good information source
- System message prefix
- Message level
- Standard message types
- Alternative message types
- Message identifier and MVS components
- Examples of system messages
- Explanation of system messages
- Common system ABEND codes
- System ABEND code numbers
- Common SVCs and their macros
- The x22 codes - caused by outside events
- The x13 codes - OPEN problems
- Other x13 codes
- Example of S013-18
- 806 - Program not found
- Sequence of events
- Example of S806-04
- 804, 80A, 878, 878 and DC2 - virtual storage problems
- The Virtual Address Space
- "above the bar"
- Traditional address space areas
- The need for managing virtual storage
- Storage for the program code
- Storage obtained outside the program
- Virtual Storage requests
- Limitations on Virtual Storage
- ABEND and reason codes
- Requestsforstorage below 2 GB (GETMAIN and STORAGE OBTAIN)
- Requestsforstorage above 2 GB (IAR64 GETSTOR)
- The REGION limit
- The effects of different REGION values
- Example of ABEND S822
- The MEMLIMIT parameter
- Example of ABEND SDC2
- The 0Cx codes
- The Program Check Interrupt
- Running RTM1
- PC FLIH and ABENDs
- The meaning of Program Checks
- Common ABENDs from Program Checks
- ABEND S0C4
- Storage Protect Keys
- Virtual address protection
- Reasonsfor translation exceptions
- Address truly invalid
- Address valid - new area
- Address valid - old area
- Other S0Cx ABENDs
- PIC 0001 Operation Exception (ABEND S0C1)
- PIC 0002 Privileged Operation Exception (ABEND S0C2)
- PIC 0007 Data Exception (ABEND S0C7)
- The S0E0 and 0Dx codes
- Miscellaneous problems
- Problems with translations
- Linkage Stack problems
- The Sx37 and SB14 codes
- Sx37
- EOV processing
- How disk data sets are allocated
- Physical Sequential (PS)data sets
- Problems when allocating a PS data set
- Initial allocation
- Primary allocation failure
- Data set full
- No secondary allocation (SD37-04)
- Secondary allocations(SB37-04)
- Example of unavailable primary allocation
- Example of SD37-04
- Message IEC031I
- Example of ABEND SB37-04
- Message IEC030I; Partitioned Data Sets (PDS)
- Problems when allocating a PDS
- Initial allocation
- Data set full
- No secondary allocation (SD37-04)
- Secondary allocations (SE37-04)
- Directory full (SB14-0C)
- Example of ABEND SE37-04
- Message IEC032I
- Example of ABEND SB14
- Message IEC217I
- Partitioned DataSets Extended (PDSE)
- Problems when allocating a PDSE
- Summary of common system ABEND codes
- Other ABEND codes
- MVS system codes (Sxxx)
- User ABEND codes (Uxxxx)
-
Interactive Problem Control System
- MVS's recovery management
- RMS
- What does RTM do?
- Interrupt types
- Anatomy of an Interrupt
- RTM - The Big Picture
- How is RTM invoked?
- Normal termination
- Abnormal termination - problem types
- Program check
- Software 'Abend'
- Abnormal termination - recovery
- Recovery routines
- RTM statusinformation
- ESPIE environment
- ESPIEprocessing
- ESTAE recovery routines
- ESTAE environment
- STAE Control Blocks (SCB)
- ESTAE processing
- Percolation
- Functional Recovery Routines
- FRR environment
- FRR stacks
- RTM2WA
- SDWA
- Variable Recording Area
- Interpreting the SDWA
- Interpreting the Variable Recording Area
- Logrec detail reports
-
Recovery & Termination
- MVS's recovery management
- RMS
- What does RTM do?
- Interrupt types
- Anatomy of an Interrupt
- RTM - The Big Picture
- How is RTM invoked?
- Normal termination
- Abnormal termination - problem types
- Program check
- Software 'Abend'
- Abnormal termination – recovery
- Recovery routines
- RTM statusinformation
- ESPIE environment
- ESPIEprocessing
- ESTAE recovery routines
- ESTAE environment
- STAE Control Blocks (SCB)
- ESTAE processing
- Percolation
- Functional Recovery Routines
- FRR environment
- FRR stacks
- RTM2WA
- SDWA
- Variable Recording Area
- Interpreting the SDWA
- Interpreting the Variable Recording Area
- Logrec detail reports
-
Request Block Analysis
- Address space structures
- RB loss of control
- Linkage stacks
- RBanalysis procedure
- Linkage Stack analysis
- General analysis
- RBanalysis
-
System Trace
- Starting the System Trace
- Formatting the Trace
- Sequence of events
- Interpreting Traces
- System Trace tips
-
SVC Dump Analysis Approach
- Generating SVC dumps
- Dump Analysis and Elimination
- Types of SVCDump
- Problem resolution overview
- Dump TITLE
- SDWA
- History
- RTM2WA
- Other dump types
-
Multi-Processor Environments
- Tightly coupled processing
- Prefixing
- Processor coexistence
- Processor STATUS
- Work In Progress
- Interrupt information
-
Locks
- The problem
- An example of what can go wrong
- Serialization via LOCKS
- Lock varieties
- Locking Hierarchy
- Locking Mechanics (SPIN)
- Spin Loop Identification
- Spin Lock Holder
- Local/CML Locks
- Locking Mechanics
- Global Suspend Locks ANALYZE
- Locks Held
- Locking Mechanics (CPU LOCK)
- SPIN lock summary
- SUSPEND lock summary
-
Dispatcher
- What does it mean to be dispatched?
- Where does the Dispatcher run?
- Dispatchable units of work
- Who calls the Dispatcher?
- Special exits
- Service Request Block routines
- Service Request Block (SRB)
- SRB example - IOS post
- Service Request Block (SRB)
- Suspended Service Request Block (SSRB)
- SRB priorities
- SRB scheduling with IEAMSCHD
- SRB enclaves
- Dispatcher queues
- Scheduling service requests
- Address spaces
- ASCB/ASXB contents
- Finding work within an address space – tasks
- TCB contents
- TCB chaining
- Address space task structure
- Serialization with Intersect
- Dispatcher indicators
- Global problem determination
- Global indicators - SRB queues
-
Address Space Control
- Cross Memory Services
- XMS instructions
- PC & PT/PR
- XMS authorization
- Primary, Secondary & Home modes
- Access Register mode
- SSAR
- Access Register Translation (ART)
- Accesslists
- ALETs
-
SAD Analysis Approach
- Big picture
- Dump environments
- When should A SADUMP be taken?
- Pre SADUMP considerations
- Taking a Standalone Dump
- Stand Alone Dump analysis path selection
- Disabled Wait analysis path
- Enabled WaitAnalysis path
- Enabled Running Analysis path
- Disabled Running Analysis path
-
Input/Output Supervisor
- IOS drivers
- Performing I/O
- I/O flow
- IOS analysis - high level
- Active I/O analysis
- IOS failure analysis
-
Real Storage Manager
- Types ofstorage
- Dynamic Address Translation
- Identifying The STD
- Managing real storage
- RSM high level check
- Detailed analysis- highfixed page utilisation
- Detailed analysis - other problems
- History - Component Trace
-
Auxiliary Storage Manager
- Paging a frame to a slot
- ASM high level check
- Detailed analysis: whatis the problem,
- Who is affected?