Scripts¶
There are some scripts in adapya.base.scripts that can be run on the command line. Usually they accept Unix style parameters. A help page is shown with the help option.
- ftpz.py - synchronize partitioned datasets with local folders
- getfilez.py - transfer single file and dump records
- smfreaderz.py - read and interpret SMF30 records
ftpz.py - synchronize PDS with local files in folders¶
ftpz.py synchronizes remote partitioned datasets on z/OS with local files in folders.
The syncronization depends on the modification time stamps in the file system and the PDS directory
Usage: ftpz.py [options]:
[-h] [-k] [-e [EXT]] [-n HOST] [-p PWD] [-s SITE] [-u USER]
[-r] [-t] [-v VERBOSE] [-d PREFIX] [-x [EXCLUDE [EXCLUDE ...]]]
[-i [INCLUDE [INCLUDE ...]]]
{upsync,upload,download,downsync,config} [pds]
Positional arguments:
{upsync,upload,download,downsync,config}:
- upsync: upload changed members to partioned dataset (PDS) and
- delete members not existing in source
- upload: upload changed files to partioned datasets (PDS)
- download: download changed files from partioned datasets (PDS)
- downsync: download changed members from partioned dataset (PDS)
- and delete members not existing in PDS
- config: display or set configuration pds -
- Partitioned Dataset name w/o quotes. With ending period: parameter is used as high-level qualifier
Optional arguments:
-h, --help show this help message and exit
-k, --keepsame On download/sync do not overwrite old file if contents
same as new file
-e [EXT], --ext [EXT]
extension to process (default is .s) e.g. -e .c -e
without argument: no extension
-n HOST, --host HOST ftp z/OS host name
-p PWD, --pwd PWD ftp z/OS user id
-s SITE, --site SITE Set ftp server options per quote site
-u USER, --user USER ftp z/OS user password
-r, --ignerr ignore ftp member access errors
-t, --test dry run - list actions only
-v VERBOSE, --verbose VERBOSE
verbose output (1 to 3)
-d PREFIX, --prefix PREFIX
dataset prefix
-x [EXCLUDE [EXCLUDE ...]], --exclude [EXCLUDE [EXCLUDE ...]]
list of libraries/members to exlude from processing
-i [INCLUDE [INCLUDE ...]], --include [INCLUDE [INCLUDE ...]]
list of libraries/members to include in processing
Examples:
sync-up PDS mm.test.adasrc from current directory and delete member that do not exist in source:
> ftpz upsync --user xyz --pwd secret --host zmax MM.TEST.ADASRC
update all changed members PDS mm.test.adasrc from current directory:
> ftpz upload --user xyz --pwd secret --host zmax MM.TEST.ADASRC
ftpz upsync -t mm.test.adasrc # dry run without updating
set configuration default values:
> ftpz config --user xyz --pwd secret --host zmax # set config > ftpz config # list config > ftpz upload MM.TEST.ADASRC # use config
When a configuration file is set up with the most common parameters in use - (user, host and password (pwd)) these do not have to be typed on each script execution. The password is encrypted so that it is not plainly readable on the command line. See example 2.
download
Download all missing or changed members of PDS mm.test.adasrc to current directory:
> ftpz download MM.TEST.ADASRC
downsync
Same as ‘download’ and in addition delete members in directory that are missing in PDS:
> ftpz downsync MM.TEST.ADASRC
download all changed members for all PDS starting with high-level quailfier (HLQ).
The current directory is used as the base matching its subdirectories name with partitioned datasets HLQ.name. Optionally exclude or include parameters can specifiy a list of libraries to exclude or include separated by space:
> ftpz download mm.test. --exclude unwanted abandond
will process pds mm.test.source and mm.test.macros but exclude mm.test.unwanted and mm.test.abandond
download specific missing or changed members ‘FOO’ and ‘BAR’ of a PDS as foo.s and bar.s in current directory:
> ftpz download mm.test.source --include foo bar
download to use extension .c and special translation table for ibm-1047 to ibm-819 with EBCDIC LF = 0x15
> ftpz download mm.c.source -e .c -s sbdataconn=MM.OEMVS31.TCPXLBIN
getfilez.py - transfer single file and optionally dump records¶
Read specific dataset or PDS member from z/OS per FTP converted to ASCII or binary.
Datasets may be variable blocked sequential dataset as binary with RDW record prefix.
Usage: getfilez [options]
The records of the local file can be dumped setting the –verbose switch 4 and a selected with –numrec and –skiprec parameters (example 3 below).
Options:
-a --ascii transfer with EBCDIC to ASCII conversion
-b --binary binary transfer (variable blocked) with RDW prefix
-d --dsn remote sequential dataset name
-e --ext extension (default .s) for member names if no
fname specified
-f --fname local file name (optional)
-c, --config set/show configuration
-n --numrec with verbose & 4: number of records to print
-p, --pwd <password> FTP ser1.0.4ogin password (*)
-u, --user <userid> (*)
-r, --recform specifies the record structure:
'RDW' variable records inlcude Record
Descriptor Word which is skipped
'RDW+' same as RDW but also return RDW
'BDW' data includes Block Descriptor Word
which is skipped (RECFM=U)
'BDW+' same as BDW, bu also return record with RDW
'EXCL4' 4 byte excl. length prefix
-s --skiprec with verbose & 4: number of records to skip
-v, --verbose 0: (default), 1: log ftp, 2: detailed ftp,
4: dump records
-x, --xlate full dataset name of the hlq.name.TCPXLBIN translate
table on mainframe for EBCDIC to ASCII conversion
using the "site SBDATACON=<xlate>
-h, --host <host name> of IBM FTP server (*)
-?, --help
Defaults marked with (*) are taken from configuration (-c) The configuration values are stored ciphered in file ~/.toolz
Examples:
set configuration user, password:
> getfilez --config --user hugo --pwd secret
read remote dataset with verbose FTP operations, user and password are taken from configuration. File is processed binary and RDW record headers are preserved:
> getfilez -bd mm.db8.uld1 -r RDW -h da3f -v2
dump VB records in local file limited by skiprec and numrec:
> getfilez -f mm.db8.uld1 -r RDW -v4 -n 1000 -s 1222000
copy member EPILOG from PDS to local file epilog.s and convert to ASCII:
> getfilez -ad mm.pds(epilog)
smfreaderz.py - read and print SMF30 records¶
Usage: smfreaderz [options]:
options:
-d --dsn <smf dataset name> remote SMF file
-f --file <file> local SMF file
-b --bfile <file> local SMF file VB blocked with BDW
-k, --skiprec <int> number of records to skip
-m, --maxrec <int> maximum number of records
-p, --pwd <password> FTP ser1.0.4ogin password (*)
-u, --user <userid> FTP ser1.0.4ogin userid (*)
-h, --host <host name> of IBM FTP server (*)
-s, --select <record selection criteria> see below (**)
-c, --config Set/show configuration
-v, --verbose level of printed information (default 2)
-?, --help
(**) record selection criteria kw1=val1[,kw2=val2]
enclose hole string with " if it contains blanks
valid keywords: job, id, user, group, prog
Criteria must be all fulfilled to select a record
Example: -s job=*MM*,group=RND,id=J*,prog=*ASM
(***) verbose level, composable: 1 - stats SMF records
2 - detailed print selected SMF records, 4 - dump records,
8 - debug
Defaults marked with (*) are taken from configuration.
The configuration for user specific parameters can be stored
with the --config option.
The reader can transfer the file (--dsn) per FTP from a remote z/OS
with the RDW option or can access the file locally if already
transfered (--file). On z/OS the --bfile option may be used.
Option -b/--bfile if file includes block descriptor word (BDW)
e.g. when running on z/OS with DCB=(RECFM=U) override on DD stmt
Examples:
1. set configuration user, password
smfreaderz --config --user hugo --pwd secret
2. read remote SMF dataset and print
smfreaderz -d cc.sysa.smf -h sysa
The following command (on Windows cmd) will select SMF30 records with program name ADARUN from a SMF system dataset:
>>> smfreaderz -v3 -s JOB=MM10026 -d ZMAX.SMFDAY.G3037V00
Record selected by condition [‘ JOB=MM10026’]:
--- Record 181197: SMF30 ---
Product or Subsystem Section
SMF30 sub type = Step total
Record version number = '05'
Subsystem product name = 'SMF'
MVS product level = 'SP7.2.1'
System name = 'ZMAX'
Sysplex name = 'MAXPLEX'
Job/Session Id Section
Job/session name = 'MM10026'
Program name = 'ADARUN'
Step name = 'ADANUC'
JES job id = 'S0207297'
Step number = 1
Device allocation start time = 14:35:54.95
Problem program start time = 14:35:55.23
Time initiator selected step = 14:35:54.95
Date initiator selected step = 2018.079
Time reader found job card = 14:35:54.79
Date reader found job card = 2018.079
Time reader found end of job = 14:35:54.82
Date reader found end of job = 2018.079
RACF group id = 'MFRAME'
RACF user id = 'RACFSTC'
Step name invoking procedure = 'STARTING'
Job class = 'STC'
Interval start time = 2018-03-20 14:35:54.955237.078
Interval end time = 2018-03-20 14:56:26.669574.079
Address space id = X'012E'
CPU accounting section
Timer Flag1 = X'80'
Step CPU time under TCB = 00:00:45.18
Step CPU time under SRB = 00:00:12.24
Initiator CPU time under TCB = 00:00:00.30
CPU time I/O Interrupts = 00:00:06.26
Step dependent enclave CPU time = 00:00:44.62
Time on zIIP = 00:00:42.66
Dependent enclave time on zIIP = 00:00:42.66
zIIP time on CP = 00:00:03.06
Dependent enclave zIIP time on CP = 00:00:03.06
Dependent enclave zIIP time on CP normalized = 00:07:51.38
CPU TCB time for step init = 00:00:00.24
Highest Task Program name = 'IEESB605'
Performance section
zIIP normalization factor = 1.98
To run this in z/OS batch the dataset must be referenced via DD name ‘SMF’:
//MMSMF30 JOB MM,CLASS=G,MSGCLASS=X,LINES=100
//*
//* smfreaderz.py reads SMF files from DD:SMF
//* -h option will print usage / description
//*
//BPX EXEC PGM=BPXBATSL
//SMF DD DISP=SHR,DSN=OPS.ZMAX.SMFDAY.G3037V00,DCB=(RECFM=U)
//STDPARM DD *
PGM /usr/mm/py27/bin/python
/usr/mm/apy/smfreaderz.py -b dd:SMF -v3
-s JOB=MM10026
/*
//STDOUT DD SYSOUT=*
//STDERR DD SYSOUT=*
//STDENV DD PATH='/usr/mm/apy/batsl.env',PATHOPTS=ORDONLY
//