scripts1.doc - Contents by Part#

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

scripts1.doc - Contents (continued)

 http://uvsoftware.ca/Freeescripts - direct access to directory of free scripts
        - may download individually or all from FreeScripts.zip

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_0 Vancouver Utility scripts for Conversion & General use

scripts calling unix/linux vs Vancouver Utilities

This scripts1.doc is intended to document 'general purpose' scripts that could be useful to any unix/linux user. Most of these scripts call standard unix/linux utilities (ls,mv,cp,cpio,diff,cmp,kill,etc) vs Vancouver Utilities (uvcopy,uvlist,uvsort,uvcp,uvhd,etc).

scripts1.doc lists about 40 of the most useful of the more than 500 scripts distributed in the Vancouver Utilities package. You can save this document, cut out the scripts that interest you, & transfer them to your unix/linux.

If you are a Vancouver Utilities customer, you have access to all scripts in the scripts directory as shown on the next page.

scripts1.doc vs CNVaids.doc

If you are more interested in scripts that could be useful during mainframe conversions (of JCL, COBOL,& EBCDIC DATA files), please see CNVaids.htm. Some of the scripts in CNVaids.htm call only unix/linux commands, but many of them do call Vancouver Utilities.

scripts1.doc vs ADMjobs.doc

Also see ADMjobs.htm for administrative type scripts such as 'backups' & scripts run by 'cron'. Again the intention of this scripts1.doc is to present the more general purpose scripts that could be useful to any unix/linux user.

Pre-Requisites to running test/demo scripts

1. Your site administrator has installed the Vancouver Utilities. Please see the directory diagrams on the next page, showing only the subdirs relevant to running these test/demo scripts.
   
   
2. Your site administrator has setup a login/homedir for you AND has setup the profiles to give you access to the Vancouver Utility programs, scripts, and uvcopy parameter files.
   
   
3. If you are the site administrator, please see ADMjobs.htm#Part_1 for listings of the supplied profiles & how best to set them up.
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

0A2. Vancouver Utility scripts for Conversion & General use

VU scripts - where stored, how accessed (PATH)

 /home/uvadm        <-- Vancouver Utilities homedir
 :     ---              --- showing relevant subdirs only ---
 :-----bin              - binaries (uvcopy,uvsort,etc)
 :-----doc              - documentation
 :-----env              - profiles distributed with VU
 :     :
 :-----sf              <-- Script Files (sub-directoried for uvadm only)
 :     :-----adm         - administrative scripts
 :     :-----demo        - demo scripts
 :     :-----IBM         - IBM mainframe conversion scripts
 :     :-----util        - utility scripts
 :     :

 /home/appsadm    <-- appsadm homedir
 :     ---              --- showing relevant subdirs only ---
 :-----bin              - binaries for site developed/modified programs
 :-----env              - profiles modified for your site
 :     :
 :-----sf             <-- shell scripts developed/modified by appsadm
 :     :

 /home/userxx     <-- user homedirs
 :     ---              --- showing relevant subdirs only ---
 :     :
 :-----sf             <-- shell scripts developed/modified by appsadm
 :     :

relevant lines from common_profile

 export UV=/home/uvadm                # <-- UV homedir symbol used below
 export APPSADM=/home/appsadm         # <-- site applications administrator

 export PATH=$PATH:$HOME/bin:$HOME/sf:$APPSADM/bin:$APPSADM/sf:$RUNLIBS/sf
 export PATH=$PATH:$UV/bin:$UV/sf/adm:$UV/sf/demo:$UV/sf/util:$UV/sf/IBM

The uvadm scripts directory is sub-directoried (sf/adm,sf/demo,sf/IBM,sf/util).

Note that the recommended profile (see ADMjobs.htm#1C3) searches the PATH of appsadm before uvadm, so any scripts that appsadm modifies will be found before any of the original scripts/jobs. Also note that the PATH searches $HOME/sf before /home/appsadm, so users could copy/modify a script in their homedir & it will be found before either appsadm/sf or uvadm/sf/...

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

0A3. Vancouver Utility scripts for Conversion & General use

uvadm, appsadm,& user homedirs

We recommend you setup a login/userid 'appsadm' to serve as the applications administrator for the unix/linux site. The appsadm home directory would hold various scripts, crontabs, log files, etc used in application administration.

One important purpose of appsadm is to hold the modified versions of any Vancouver Utility profiles, scripts & uvcopy jobs that you need to customize at your site.

Scripts that are useful to all programmers & operators should be stored in /home/appsadm/sf/... Scripts that are useful only to individual programmers should be stored in their homedirs /home/userxx/sf/...

This protects you from losing your customized versions when you install a future new version of Vancouver Utilities, which would overwrite your modified versions if stored in /home/uvadm/...

setup to run uvcopy test/demo jobs

 #1. Login to your homedir

 #2a. mkdir sf        <-- make subdir for your own scripts
      ========          - or to copy/modify scripts from /home/uvadm/sf/...

 #2b. mkdir dat1 tf   <-- make subdirs for the test/demo files
      =============

 #2c. mkdir tmp       <-- make subdir for test/demo outputs
      =========

 #3a. cp /home/uvadm/dat1/* dat1  <-- copy test/demo files to your homedir
      ==========================

 #3b. cp /home/uvadm/tf/* tf       <-- copy test/demo files to your homedir
      ======================

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_1 Vancouver Utility scripts for Conversion & General use

file 'rename' scripts - Contents

These scripts rename all files (or files matching a pattern) in a directory. Renaming dozens or hundreds of files is often required when you import files from outside sources & these scripts can save you hours of keyboard drudgery.

You can see a help screen for each script, by entering the script-name only, without its required arguments (omit the directory).

See usage example on the next page & all scripts listed on the following pages.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1A1. Vancouver Utility scripts for Conversion & General use

example - renaming Mainframe Data Files

Here is a great example of the extreme usefulness of these 'rename' scripts. When mainframe files are transferred to unix/linux, we like to remove any quotes, translate to lower case & remove any extraneous extensions. We will also convert any '$' Dollar-signs to '_' Underscores since '$' is not allowed in unix filenames.

Assume the files have been FTP'd to /p4/cnvdata/dat1/... & we wish to copy/ rename the files into /p4/cnvdata/dat2/... as shown below:

 /p4/cnvdata
 :---dat1         <-- directory of files FTP'd from Mainframe
 :   :
 :   :---'E2121653.M$TAXE.PISCINES.CLUS.VSEPRD' <-- filenames from MAINFRAME
 :   :---'E2121656.I$TAXE.FACTURAT.CLUS.VSEPRD'
 :   :---'E2122683.I$TAXE.DEGREVEM.CLUS.VSEPRD'

 :---dat2         <-- directory to rename files for unix/linux
 :   :
 :   :---e2121653.m_taxe.piscines          <-- filenames desired on Unix/Linux
 :   :---e2121656.i_taxe.facturat
 :   :---e2122683.i_taxe.degrevem

the HARD way to rename files

 #1. cp dat1/* dat2  <-- copy to 2nd directory (to preserve original filenames?)
 #2. cd dat2/
 #3. mv 'E2121653.M$TAXE.PISCINES.CLUS.VSEPRD' e2121653.m_taxe.piscines
 #4. mv 'E2121656.I$TAXE.FACTURAT.CLUS.VSEPRD' e2121656.i_taxe.facturat
 #5. mv 'E2122683.I$TAXE.DEGREVEM.CLUS.VSEPRD' e2122683.i_taxe.degrevem
     mv          --- etc, could be 100's or 1000's of files ---

the EASY way to rename files

 #1. cp dat1/* dat2  <-- copy files to dat2/... before renaming (if desired)
     ==============    - no need to copy, when you have satisfied yourself
                         that these scripts work (as advertised)

 #2. rename-QQ dat2  <-- remove the quotes (see 'rename-QQ' on page '1G1')
     ==============
 #3. renameD2U dat2  <-- change '$'s to '_'s (see 'renameD2U on page '1F2')
     ==============                     ('$' not allowed in unix filenames)
 #4. rename-X dat2 .CLUS.VSEPRD  <-- remove unwanted extensions
     ==========================    - see 'rename-X' on page '1D3')
 #5. renameL dat2    <-- translate to lower case (see 'renameL' on page '1B1')
     ============

Without these scripts, it would be a very long, boring, error-prone job to rename the scripts manually using 'mv' commands, OR you would have to develope similar scripts yourself.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1B1. Vancouver Utility scripts to Rename all files in directory

renameL - translate filenames to Lower case

 #!/bin/ksh
 # renameL - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameL - rename an entire directory of filenames to lower case
 #
 echo "rename all filenames in subdir to lower case"
 if [ -d "$1" ]; then :
    else echo "usage: renameL directory [y]"
         echo "       ====================="
         echo " - arg1 must be a directory"
         echo " - arg2 may be 'y' to bypass confirmation prompt"
         exit 1; fi
 #
 reply="$2"
 until [ "$reply" = "y" ]
 do echo "will rename all files in $1 to lower case OK ? y/n"
    read reply
 done
 #
 x=0; y=0
 for i in $1/*
 do let x=x+1
    f=${i##*/}
    typeset -l g=$f
    if [[ $g != $f ]]; then
       mv -i $1/$f $1/$g
       #================
       let y=y+1
       echo "file# $y (of $x) $1/$f - renamed to: $1/$g"
    fi
 done
 echo "total $y files in ${1}, $x renamed to lower case"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1B2. Vancouver Utility scripts to Rename all files in directory

renameU - translate filenames to UPPER case

 #!/bin/ksh
 # renameU - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameU - rename an entire directory of filenames to UPPER case
 #
 echo "rename all filenames in subdir - to UPPER case"
 if [ -d "$1" ]; then :
    else echo "usage: renameU directory [y]"
         echo "       ====================="
         echo " - arg1 must be a directory"
         echo " - arg2 may be 'y' to bypass confirmation prompt"
         exit 1; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "will rename all files in $1 to UPPER case OK ? y/n"
    read reply
 done
 #
 x=0; y=0
 for i in $1/*
 do let y=y+1
    f=${i##*/}
    typeset -u g=$f
    if [[ $g != $f ]]; then
       mv -i $1/$f $1/$g
       #================
       let x=x+1
       echo "file# $x (of $y) $1/$f  - renamed to:  $1/$g"
    fi
 done
 echo "total $y files in ${1}, $x renamed to UPPER case"
 exit 0

renameU same as renameL except for:

'renameU' is the same as 'renameL' except line 21 is changed:

      typeset -l g=$f   <-- from this
      ===============
      typeset -u g=$f   <-- to this
      ===============

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1B3. Vancouver Utility scripts to Rename all files in directory

renameLsd - Lower case all files in all subdirs of superdir

 #!/bin/ksh
 # renameLsd - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameLsd - lower case rename all files in all subdirs of a superdir
 #           - by Owen Townsend, UV Software, May 2015
 #
 echo "lower case rename all files in all subdirs of a superdir"
 d1="$1"
 if [ -d "$d1" ]; then :
    else echo "usage: renameLsd directory [y]"
         echo "       ======================="
         echo " - arg1 must be a directory"
         echo " - arg2 may be 'y' to bypass confirmation prompt"
         exit 1; fi
 #
 reply="$2"
 until [ "$reply" = "y" ]
 do echo "will rename all files in all subdirs of $d1 to lower case OK ? y/n"
    read reply
 done
 #
 ds=0; fs=0
 for d1d2 in $d1/*
   { d2=$(basename $d1d2)
     if [[ -d $d1/$d2 ]]; then
        ((ds+=1))
        for d1d2f in $d1/$d2/*
          { f1=$(basename $d1d2f)
            typeset -l f2=$f1
            if [[ $f2 != $f1 ]]; then
               mv -i $d1/$d2/$f1 $d1/$d2/$f2
               #============================
               ((fs+=1))
               echo "file# $fs $d1/$d2/$f1 - renamed to: $d1/$d2/$f2"
            fi
          }
       fi
   }
 echo "total $fs files in $ds subdirs renamed to lower case"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1C1. Vancouver Utility scripts to Rename all files in directory

renameP - replace Prefix with alternate

 #!/bin/ksh
 # renameP - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameP - rename an entire directory of filenames
 #         - replacing an existing prefix with a new prefix
 #         - may use pattern characters in prefix
 #
 echo "rename all filenames in subdir - replace prefix with alternate"
 if [[ -d "$1" && -n "$2" && -n "$3" ]]; then :
    else echo "usage: renameP directory oldprefix newprefix"
         echo "       ====================================="
         exit 1; fi
 reply="n"
 until [[ "$reply" = "y" ]]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for i in $1/*
 do f=${i##*/}                  # remove any preceding directories
    g=${f#$2}                   # remove smallest matching old prefix
    h=${3}${g}                  # insert new prefix
    let x=x+1
    if [[ ($f == *${2}*) && ($h != $f) ]]; then
       mv -i $1/$f $1/$h
       #================
       let y=y+1
       echo "file# $x - $1/$f  - renamed to:  $1/$h"
    fi
 done
 echo "total $x files, $y renamed (replacing prefix $2 with $3)"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1C2. Vancouver Utility scripts to Rename all files in directory

rename-P - remove any specified prefix

 #!/bin/ksh
 # rename-P - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-P - rename an entire directory of filenames
 #          - removing a specified prefix
 #          - may use pattern characters in prefix
 #
 echo "rename all filenames in subdir - removing a specified prefix"
 if [ -d "$1" -a -n "$2" ]; then :
    else echo "usage: rename-P directory prefix"
         echo "       ========================="
         exit 1; fi
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for i in $1/*
 do f=${i##*/}                  # remove any preceding directories
    g=${f#$2}                   # remove smallest matching prefix
    let x=x+1
    if [[ $g != $f ]]; then
       mv -i $1/$f $1/$g
       #================
       let y=y+1
       echo "file# $x - $1/$f  - renamed to:  $1/$g"
    fi
 done
 echo "total $x files, $y renamed (removing $2 prefix)"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1C3. Vancouver Utility scripts to Rename all files in directory

rename-P. - remove prefix up to 1st '.'

 #!/bin/ksh
 # rename-P. - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-P. - rename an entire directory of filenames
 #           - removing Prefix until &including 1st '.' Dot
 #
 echo "rename all filenames in subdir - removing prefix until 1st '.'"
 if [ -d "$1" ]; then :
    else echo "usage: rename-P. directory "
         echo "       ===================="
         exit 1; fi
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for i in $1/*
 do f=${i##*/}                  # remove any preceding directories
    g=${f#*.}                   # remove smallest match to 1st Dot
    let x=x+1
    if [[ $g != $f ]]; then
       mv -i $1/$f $1/$g
       #================
       let y=y+1
       echo "file# $x - $1/$f  - renamed to:  $1/$g"
    fi
 done
 echo "total $x files, $y renamed, removing prefix to 1st '.'"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1C4. Vancouver Utility scripts to Rename all files in directory

rename+P - add any specified prefix

 #!/bin/ksh
 # rename+P - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename+P - rename an entire directory of filenames
 #          - adding a specified prefix
 #          - inhibited if a specified pattern already present
 #
 echo "rename all filenames in subdir - adding a specified prefix"
 if [[ -d "$1" && -n "$2" && -n "$3" ]]; then :
    else echo "USAGE: rename+P directory prefix inhibitor/all"
         echo "       ======================================="
         echo "example: rename+P datadir raw. raw.  #inhibit if already present"
         echo "         =========================="
    exit 1; fi
 q=$3; if [[ "$3" = "all" ]]; then q="allALL"; fi
 x=0;y=0
 for i in $1/*
 do f=${i##*/}
    let x=x+1
    if [[ "$f" = *"$q"* ]]; then  # if filename contains inhibitor
         echo "file# $x - $1/$f"
    else g=${2}${f}               # insert prefix
         mv -i $1/$f $1/$g        # change fielname
         #================
         let y=y+1
         echo "file# $x - $1/$f  - renamed to:  $1/$g"
    fi
 done
 echo "EOD - $y files renamed of $x total in $1"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1D1. Vancouver Utility scripts to Rename all files in directory

renameX - change any extension to any alternate

 #!/bin/ksh
 # renameX - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameX - change a specified extension to a specified alternative
 #           for all files within a directory
 #
 echo "rename all filenames in subdir - replace extension with alternate"
 if [[ -d "$1" && -n "$2" && -n "$3" ]]; then :
    else echo "USAGE:   renameX directory ext1 ext2"
         echo "         ==========================="
         echo "example: renameX cobs .cbl .bat"
         echo "         ======================"
    exit 1; fi
 #
 reply="n"
 until [[ "$reply" = "y" ]]
 do echo "will rename all files in $1 changing ext $2 too $3 OK ? y/n"
    read reply
 done
 #
 x=0;y=0
 for i in $1/*
 do f=${i##*/}
    let x=x+1
    if [[ "$f" == *"$2" ]]; then    # if ext1 present ?
       let y=y+1
       g=${f%$2}                  # remove ext1
       h=$g$3                     # add ext2
       mv -i  $1/$f $1/$h         # rename
       #=================
       echo "file# $x - $1/$f  renamed to:  $1/$h"
    fi
 done
 echo "EOD - $y files renamed of $x total in $1"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1D2. Vancouver Utility scripts to Rename all files in directory

rename.X - change any .extension to any .alternate

 #!/bin/ksh
 # rename.X - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename.X - change a specified .extension to a specified .alternative
 #            for all files within a directory
 #          - '.' must be present (unlike 'renameX')
 # renameX  - does not insist on '.' in filename
 #
 echo "rename all filenames in subdir - replace .extension with .alternate"
 if [[ -d "$1" && -n "$2" && -n "$3" ]]; then :
    else echo "USAGE:   rename.X directory ext1 ext2"
         echo "         ============================"
         echo "example: rename.X cobprogs .cob .cbl  <--change .cob to .cbl"
         echo "         ============================"
    exit 1; fi
 #
 reply="n"
 until [[ "$reply" = "y" ]]
 do echo "will rename all files in $1 changing ext $2 too $3 OK ? y/n"
    read reply
 done
 #
 x=0;y=0
 for i in $1/*
 do f=${i##*/}
    let x=x+1
    if [[ ($f == *.*) && ($f == *$2) ]]; then # if .ext1 present ?
       let y=y+1
       g=${f%$2}                # remove ext1
       h=$g$3                   # add ext2
       mv -i  $1/$f $1/$h       # rename
       #=================
       echo "file# $x - $1/$f  renamed to:  $1/$h"
    fi
 done
 echo "EOD - $y files renamed of $x total in $1"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1D3. Vancouver Utility scripts to Rename all files in directory

rename-X - remove any extension

 #!/bin/ksh
 # rename-X - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-X - rename an entire directory of filenames
 #          - removing the specified extension (no '.' required)
 #          - may use pattern characters in extension
 #
 echo "rename all files in subdir - removing specified extension (no '.' reqd)"
 if [ -d "$1" -a -n "$2" ]; then :
    else echo "usage:  rename-X directory extension  <-- arg1 subdir, arg2 ext"
         echo "        ============================"
         echo "example: rename-X jcldir .jcl  <--remove .jcl from all jcldir"
         echo "         ===================="
    exit 1; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for i in $1/*
 do f=${i##*/}
    g=${f%$2}
    let x=x+1
    if [[ $g != $f ]]; then
       mv -i $1/$f $1/$g
       #================
       let y=y+1
       echo "file# $x - $1/$f  - renamed to:  $1/$g"
    fi
 done
 echo "total $x files, $y renamed (removing $2)"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1D4. Vancouver Utility scripts to Rename all files in directory

rename-.X - remove any .extension

 #!/bin/ksh
 # rename-.X - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-.X - rename an entire directory of filenames
 #           - removing extension back to specified character (default '.')
 #
 dir="$1"; xchar="$2";
 echo "rename all files in subdir - removing extension back to spcfd character (default '.')"
 if [ -d "$dir" ]; then :
    else echo "usage:  rename-.X directory [xchar]"
         echo "        ==========================="
         echo " - arg1 must be a directory"
         echo " - arg2 may specify a separator '.' default"
         echo " "
         echo "example1: rename-.X scripts   <-- remove any .extension present"
         echo "          =================     - migt be .jcl, .ksh, etc"
         echo "example2: rename-.X parms _   <-- remove any _extension present"
         echo "          ==================    - specifying '_' vs default '.'"
    exit 1; fi
 #
 if [[ -z "$xchar" ]]; then xchar='.'; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for df1 in $dir/*
 do f1=$(basename $df1)
    f2=${f1%${xchar}*}
    let x=x+1
    if [[ $f2 != $f1 ]]; then
       mv -i $dir/$f1 $dir/$f2
       #======================
       let y=y+1
       echo "file# $x - $dir/$f1 - renamed to: $dir/$f2"
    fi
 done
 echo "total $x files, $y renamed (removing ext back to "$xchar")"
 exit 0
 #

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1D5. Vancouver Utility scripts to Rename all files in directory

rename+X - add any specified extension

 #!/bin/ksh
 # rename+X - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename+X - rename an entire directory of filenames
 #          - adding any specified extension
 #          - inhibited if extension already present
 #
 echo "rename all filenames in subdir - adding a specified extension"
 if [[ -d "$1" && -n "$2" ]]; then :
    else echo "USAGE:   rename+X directory extension"
         echo "         ============================"
         echo "example: rename+X jcls .jcl"
         echo "         =================="
         echo " - add extension '.jcl' (if not already present)"
    exit 1; fi
 x=0;y=0
 for i in $1/*
 do f=${i##*/}
    let x=x+1
    if [[ "$f" = *$2 ]]; then  # if extnsn allready present ?
         echo "file# $x - $1/$f"
    else g=${f}$2
         mv -i $1/$f $1/$g
         #================
         let y=y+1
         echo "file# $x - $1/$f  - renamed to:  $1/$g"
    fi
 done
 echo "EOD - $y files renamed of $x total in $1"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1E1. Vancouver Utility scripts to Rename all files in directory

rename-AA - remove Any pattern Anywhere in filename

 #!/bin/ksh
 # rename-AA - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-AA - rename an entire directory of filenames
 #           - removing Any pattern occuring Anywhere in filename
 #
 echo "rename files in subdir - removing a pattern that can occur Anywhere"
 if [ -d "$1" -a -n "$2" ]; then :
    else echo "usage: rename-AA directory pattern"
         echo "       ==========================="
         exit 1; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for dfn in $1/*
 do fn=${dfn##*/}           # remove any preceding directories
    fn1=${fn%${2}*}         # remove smallest matching suffix
    fn2=${fn#*${2}}         # remove smallest matching prefix
    fn3=${fn1}${fn2}        # concatenate the left & right remainders
    let x=x+1
    if [[ $fn3 != $fn ]]; then
       mv -i $1/$fn $1/$fn3
       #==================
       let y=y+1
       echo "file# $x $1/$fn - renamed to: $1/$fn3"
    fi
 done
 echo "total $x files, $y renamed (removing $2 pattern)"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1E2. Vancouver Utility scripts to Rename all files in directory

renameAA - replace Any pattern with any Alternate

 #!/bin/ksh
 # renameAA - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameAA - rename an entire directory of filenames
 #          - replacing Any pattern Anywhere in filename with an Alternative
 #
 echo "rename files in subdir - replacing a pattern that can occur Anywhere"
 if [[ (-d "$1") && (-n "$2") && (-n "$3") ]]; then :
    else echo "usage: renameAA directory pattern altpattern"
         echo "       ====================================="
         exit 1; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for dfn in $1/*
 do fn=${dfn##*/}           # remove any preceding directories
    if [[ $fn != *${2}* ]] then continue; fi
    fn1=${fn%${2}*}         # remove smallest matching suffix
    fn2=${fn#*${2}}         # remove smallest matching prefix
    fn3=${fn1}${3}${fn2}    # concatenate left-part, altpattern,& right-part
    let x=x+1
    if [[ $fn3 != $fn ]]; then
       mv -i $1/$fn $1/$fn3
       #===================
       let y=y+1
       echo "file# $x $1/$fn - renamed to: $1/$fn3"
    fi
 done
 echo "total $x files, $y renamed (replacing $2 with $3)"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1F1. Vancouver Utility scripts to Rename all files in directory

renameB2U - change any embedded blanks to underscores

 #!/bin/ksh
 # renameB2U - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameB2U - rename all files in a directory, changing any Blanks to _underscores
 #
 echo "rename all files in directory, changing any Blanks to _underscores"
 if [ -d "$1" ]; then :
    else echo "usage: renameB2U directory [y]"
         echo "       ======================="
         echo " - arg1 must be a directory"
         echo " - arg2 may be 'y' to bypass confirmation prompt"
         exit 1; fi
 #
 reply="$2"
 until [ "$reply" = "y" ]
 do echo "enter 'y' to rename files in $1 changing any Blanks to _underscores"
    read reply
 done
 #
 x=0; y=0
 for i in $1/*
 do let x=x+1
    f=${i##*/}
 ## typeset -l g=$f
    g=$(echo $f | tr " " "_")
    if [[ $g != $f ]]; then
       mv -i "$1/$f" $1/$g
       #==================
       let y=y+1
       echo "file# $y (of $x) $1/$f - renamed to: $1/$g"
    fi
 done
 echo "total $y files in ${1}, $x renamed changing Blanks to _underscores"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1F2. Vancouver Utility scripts to Rename all files in directory

renameD2U - change any '$' dollar signs to '_' underscores

 #!/bin/ksh
 # renameD2U - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameD2U - rename changing Dollar signs to _underscores
 #
 echo "rename changing Dollar signs to _underscores."
 if [ -d "$1" ]; then :
    else echo "usage: renameD2U directory  <-- arg1 must be a directory"
         echo "       ================="
         exit 1; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "rename changing Dollar signs to _underscores? y/n"
    read reply
 done
 #
 x=0; y=0
 for i in $1/*
 do let y=y+1
    f=${i##*/}
    g=$(echo $f | tr '$' '_')
    if [[ $g != $f ]]; then
       mv -i $1/$f $1/$g
       #================
       let x=x+1
       echo "file# $x (of $y) $1/$f  - renamed to:  $1/$g"
    fi
 done
 echo "total $y files in ${1}, $x renamed changing Dollar signs to _underscores."
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1G1. Vancouver Utility scripts to Rename all files in directory

rename-QQ - remove any 'Quotes'

 #!/bin/ksh
 # rename-QQ - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-QQ - rename an entire directory of filenames - removing 'Quotes'
 #
 echo "rename all filenames in subdir - removing 'Quotes'"
 if [ ! -d "$1" ]
    then echo "usage: rename-QQ directory  <-- arg1 must be a directory"
         echo "       ==================="
         exit 1; fi
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for i in $1/*
 do
    f=$(basename $i)
    echo "f=$f"
    if [[ $f == \'* && $f == *\' ]]; then
       g=${f#\'}              # remove left side 'quote
       h=${g%\'}              # remove right side quote'
       let x=x+1
       mv $1/$f $1/$h
       #=============
       echo "file# $x - $f renamed to: $h"
    fi
 done
 echo "total $x files in $1 renamed (removed quotes)"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1G2. Vancouver Utility scripts to Rename all files in directory

rename-PP - remove any (Parenthesis)

 #!/bin/ksh
 # rename-PP - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-PP - remove any (parenthesis) from all filenames in a subdir
 #           - also see renamexPP to replace the parens with anotehr character
 #             (default underscore '_')
 #
 #    sample input-names    output-names
 #  ------------------------------------------
 #  wcs41.parm(ed06srt1)    wcs.parmed06srt1
 #  wcs41.parm(lc28srt2)    wcs.parmlc28srt2
 #
 echo "rename all filenames in subdir - removing any '(' & ')' parens"
 if [ ! -d "$1" ]
    then echo "usage: rename-PP directory  <-- arg1 must be a directory"
         echo "       ==================="
         exit 1; fi
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for df in $1/*
 do
    f=$(basename $df)
    echo "f=$f"
    if [[ $f == *\(* && $f == *\)* ]]; then
       g=$(echo $f | tr -d "()")
       let x=x+1
       mv $1/$f $1/$g
       #=============
       echo "file# $x - $1/$f - renamed to:  $1/$g"
    fi
 done
 echo "total $x files in $1 renamed removing any '(' & ')' parens"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1G3. Vancouver Utility scripts to Rename all files in directory

rename2PP - rename to whatever is in (Parenthesis)

 #!/bin/ksh
 # rename2PP - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename2PP - rename an entire directory of filenames to the name in parenthesis
 #          - used in mainframe JCL conversion of control card member libraries
 #
 #       sample input-names      output-names
 #  ------------------------------------------
 #  wcs41.parmlib(ed06srt1).rtf    ed06srt1
 #  wcs41.parmlib(lc28srt2).rtf    lc28srt2
 #
 echo "rename all filenames in subdir - to the name in (parenthesis)"
 if [ ! -d "$1" ]
    then echo "usage: rename2PP directory  <-- arg1 must be a directory"
         echo "       ==================="
         exit 1; fi
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for df in $1/*
 do
    f=$(basename $df)
    echo "f=$f"
    if [[ $f == *\(* && $f == *\)* ]]; then
       g=${f#*\(}                  # remove left side to (
       h=${g%\)*}                  # remove right side to (
       let x=x+1
       mv $1/$f $1/$h
       #=============
       echo "file# $x - $1/$f - renamed to:  $1/$h"
    fi
 done
 echo "total $x files in $1 renamed to (name) in parens"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1H1. Vancouver Utility scripts to Rename all files in directory

rename-TS - remove timestamp _HHMMSS

 #!/bin/ksh
 # rename-TS - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-TS - rename an entire directory of filenames
 #           - removing a timestamp format '_HHMMSS' underscore + 6 digits
 # filenamex_175500.txt <-- sample input
 # filenamex.txt        <-- sample output
 #
 echo "rename files in subdir - removing timestamp '_HHMMSS' ('_' + 6 digits)"
 if [[ -d "$1" ]]; then :
    else echo "usage: rename-TS directory"
         echo "       ==================="
         exit 1; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for dfn in $1/*
 do fn=${dfn##*/}           # remove any preceding directories
    fn1=${fn%_[0-9][0-9][0-9][0-9][0-9][0-9]*} # remove timestamp & suffix
    fn2=${fn#*_[0-9][0-9][0-9][0-9][0-9][0-9]} # remove prefix & timestamp
    fn3=${fn1}${fn2}        # concatenate the left & right remainders
    let x=x+1
    if [[ $fn3 != $fn ]]; then
       mv -i $1/$fn $1/$fn3
       #===================
       let y=y+1
       echo "file# $x $1/$fn - renamed to: $1/$fn3"
    fi
 done
 echo "total $x files, $y renamed (removing _timestamp)"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1I1. Vancouver Utility scripts to Rename all files in directory

renameGDG - change mainframe GDG suffix .G####V00 to _000001

 #!/bin/ksh
 # renameGDG - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameGDG - change file suffix .G####VXX to _00#### for all files in directory
 #           - for mainframe GDG file conversions to unix/linux/windows
 #           - for exportgen functions called by JCL/scripts converted by VU
 #           - see www.uvsoftware.ca/mvsjcl.htm#Part_5
 #
 d="$1";
 if [[ ! -d "$1" ]]; then
    echo "usage:   renameGDG directory (arg1 must be a directory)"
    echo "         ================="
    echo "example: renameGDG d2asc"
    echo "         ==============="
    exit 1;
 fi
 echo "change all suffixes in $d from G####VXX to _00#### "
 echo " - enter to proceed (or kill)"; read reply
 x=0; y=0;
 for df in $d/*
   do let x=x+1
      if [[ $df == *G[0-9][0-9][0-9][0-9]V[0-9][0-9] ]]; then
         f1=$(basename $df)
         f2=${f1%V[0-9][0-9]}
         f3=${f2%.G[0-9][0-9][0-9][0-9]}
         f4=${f2##*G}
         f5=${f3}_00$f4
         mv -f $d/$f1 $d/$f5
         #==================
         echo "file# $x - $d/$f1 renamed to: $d/$f5"
         let y=y+1
      fi
   done
 echo "EOD - $y files renamed of $x total in $d"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1I2. Vancouver Utility scripts to Rename all files in directory

renameGDG1 - change '.G####V00' to '_000001' (keep latest)

 #!/bin/ksh
 # renameGDG1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameGDG1 - change file suffix G####VXX to _000001 for all files in a directory
 #         - for mainframe GDG file conversions to unix/linux/windows
 #         - for exportgen functions called by JCL/scripts converted by VU
 #         - see www.uvsoftware.ca/mvsjcl.htm#Part_5
 #
 d="$1";
 if [[ ! -d "$1" ]]; then
    echo "usage:   renameGDG1 directory (arg1 must be a directory)"
    echo "         ===================="
    echo "example: renameGDG1 mstr"
    echo "         ==============="
    exit 1;
 fi
 echo "change all suffixes in $d from G####VXX to _00#### "
 echo " - enter to proceed (or kill)"; read reply
 x=0; y=0;
 for df in $d/*
   do let x=x+1
      if [[ $df == *G[0-9][0-9][0-9][0-9]V[0-9][0-9] ]]; then
         f1=$(basename $df)
         f2=${f1%.G[0-9][0-9][0-9][0-9]V[0-9][0-9]}
         f3=${f2}_000001
         mv -i $d/$f1 $d/$f3
         #==================
         echo "file# $x - $d/$f1 renamed to: $d/$f3"
         let y=y+1
      fi
   done
 echo "EOD - $y files renamed of $x total in $d"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1J1. Vancouver Utility scripts to Rename all files in directory

renameCC - convert any 1 character to a 2nd character

 #!/bin/ksh
 # renameCC - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameCC - rename an entire directory of filenames
 #          - replacing a specified character in filenames with an alternate
 #          - such as replacing all '.'s with '_'s
 #
 echo "rename files in subdir - replacing a pattern that can occur Anywhere"
 if [[ (-d "$1") && (-n "$2") && (-n "$3") ]]; then :
    else echo "usage: renameCC directory 'character' 'altcharacter'"
         echo "       ============================================="
         echo " - must enclose characters in single quotes"
         exit 1; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for dfn in $1/*
 do fn1=${dfn##*/}           # remove any preceding directories
    fn2=$(echo $fn1 | tr $2 $3)
    let x=x+1
    if [[ $fn1 == $fn2 ]] then continue; fi
    mv -i $1/$fn1 $1/$fn2
    #====================
    let y=y+1
    echo "file# $x $1/$fn1 - renamed to: $1/$fn2"
 done
 echo "total $x files, $y renamed (replacing $2 with $3)"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_2 scripts1.doc - Vancouver Utility scripts for Conversion & General use

2A1. Vancouver Utility 'aliases'

'aliases save keystrokes for often repeated commands (alias l='ls -l' is a good example).

Here are the aliases defined in the profiles provided with Vancouver Utilities. See the common_profile listed at ADMjobs.htm#1C3 starting about line 200.

 alias rm='rm -i'              # confirm removes
 alias mv='mv -i'              # confirm renames
 alias cp='cp -i'              # confirm copy overwrites
 alias rmf='rm -f'             # force removes (no prompts)
 alias mvf='mv -f'             # force renames (no prompts)
 alias cpf='cp -f'             # force copies (no prompts)
 alias l='ls -l'               # save keystrokes
 alias more='less'             # less is way better than more
 alias grep='grep -n'          # ensure -n option used on grep
 alias cdd='cd $RUNDATA'       # quick access to data superdir
 alias cdl='cd $RUNLIBS'       # quick access to libs superdir
 alias cdc='cd $CNVDATA'       # quick access to data conversion superdir
 alias cdk='cd $CMPDATA'       # quick access to data conversion superdir
 alias cdb='cd $BACKUP'        # quick access to backup superdir
 alias cdr='cd $RESTORE'       # quick access to restore superdir

notes

We recommend the aliases to prompt for overwrite, such as alias rm='rm -i' Use option '-f' when you have many files (rm -f tmp/*, etc)

Aliases such as cdd='cd $RUNDATA' depend on the prior definitions (in the profiles) of RUNDATA, RUNLIBS, etc in the profiles. This is described in ADMjobs.htm#Part_1.

The result is that programmers will be changed to their testdata,testlibs,etc. But operators would be changed to their proddata, prodlibs, etc. For example the definitions of RUNDATA & RUNLIBS for a programmer might be:

 export RUNDATA=/home/userxx/testdata     <-- private testdata/testlibs
 export RUNLIBS=/home/userxx/testlibs

 export RUNDATA=/p1/apps/testdata         <-- common testdata/testlibs
 export RUNLIBS=/p1/apps/testlibs           - shared by all programmers

Note that some sites may define RUNDATA & RUNLIBS with rather long pathnames & the aliases can save more keystrokes than indicated here.

A major advantage is that administrators can change the locations of data & libraries. The use of environmental variables RUNDATA & RUNLIBS means that programmers & operators do not have to change their usual commands.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_3 Vancouver Utility scripts for Conversion & General use

listing file & directory names

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3A1. Vancouver Utility scripts for Conversion & General use

llc - list directory with File & Line counts

 #!/bin/ksh
 # llc - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # llc - list filenames in directory with file & line counts
 echo "llc - list filenames in directory with file & line counts"
 d1="$1";
 if [[ -z "$d1" ]]; then d1=$PWD; fi
 if [[ ! -d $d1 ]]; then
    echo "usage: llc directory    <-- arg1 must be a directory"
    echo "       ============="
    echo "example: llc demo/dat1  <-- see report at end script"
    echo "         ============="
    exit 1; fi
 typeset -R5 cntf=00000
 typeset -R7 cntl=0000000 cntt=0000000
 integer page=0 cntfi=0 cntd=0
 test -d tmp || mkdir tmp
 d2=$(echo $d1 | tr '/' '_')
 echo "File#   Lines Directory/Filename" >tmp/${d2}_llc
 for d1f1 in $d1/*
   { if [[ ! -f $d1f1 ]]; then continue; fi
     ((cntf+=1))
     d1f2=${d1f1}
     lsl=$(ls -l $d1f1)
     cntx=$(wc -l $d1f1)
     cntl=${cntx%% $d1f1}    # drop right hand filename
     ((cntt+=cntl))
     echo "$cntf $cntl $d1f2" >>tmp/${d2}_llc
     ((cntfi+=1));
   }
 echo "$cntf files, $cntt total lines in $cnte files in directory $d1" >>tmp/${d2}_llc
 more tmp/${d2}_llc
 exit 0
 # llc demo/dat1  <-- example, report shown below
 # =============
 # File#   Lines Directory/Filename
 #     1     335 demo/dat1/CanadaMPs
 #     2     336 demo/dat1/CanadaMPs.csv
 #     3     539 demo/dat1/UScongress
 #     4     540 demo/dat1/UScongress.csv
 #          1750 total lines in 4 files in directory demo/dat1

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3A2. Vancouver Utility scripts for Conversion & General use

sample output for 'llc'

 llc /home/uvadm/demo/dat2   <-- list Vancouver Utility demo data files
 =========================     - with File & Line counts

 File#   Lines Directory/Filename
     1     335 dat2/CanadaMPs
     2      13 dat2/CanadaProvinces
     3       8 dat2/nameadrs2
     4     305 dat2/UScities
     5     539 dat2/UScongress
     6      50 dat2/USstates
          1250 total lines in  files in directory

sample output for 'llc2'

 llc /home/uvadm/demo/dat2   <-- list Vancouver Utility demo data files
 =========================     - with File & Line counts

  File#  Lines MinRsiz MaxRsiz  MinRnum MaxRnum  Directory/FileName
      1    335      94      94        1     335  dat2/CanadaMPs
      2     13      13      33       13       5  dat2/CanadaProvinces
      3      8      69      74        6       7  dat2/nameadrs2
      4    305      74      81        2       1  dat2/UScities
      5    539      70      73        2     422  dat2/UScongress
      6     50      12      22       15      40  dat2/USstates

See the 'llc2' script listed on the next page:

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3A3. Vancouver Utility scripts for Conversion & General use

List directory showing Min & Max record sizes & Line#s where found

 #!/bin/ksh
 # llc2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/...
 # llc2 - list directory stats: file#,Lines,Minsize,Minrec#,Maxsize,Maxrec#, dir/filename
 echo "llc2 - list directory: File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#, Dir/Filename"
 d1="$1";
 if [[ ! -d "$d1" ]]; then
    echo "usage: llc2 directory    <-- ERROR - arg1 must be a directory"
    echo "       =============="
    echo "example: llc2 demo/dat1  <-- Correct Format & sample report"
    echo "         =============="
    echo " File#  Lines MinRsiz MaxRsiz  MinRnum MaxRnum  Directory/FileName"
    echo "     1    335      94      94        1     335  dat1/CanadaMPs"
    echo "     2     13      13      33       13       5  dat1/CanadaProvinces"
    echo "     3      8      69      74        6       7  dat1/nameadrs1"
    echo "     4    305      74      81        2       1  dat1/UScities"
    echo "     5    539      70      73        2     422  dat1/UScongress"
    echo "     6     50      12      22       15      40  dat1/USstates"
    exit 1; fi
 #
 d2=$(echo $d1 | tr '/' '_')
 test -d rpts || mkdir rpts
 echo " File#   Lines      KB MinRsiz MaxRsiz MinRnum MaxRnum  Directory/FileName" >rpts/${d2}_llc2
 typeset -R6 files=0 tfiles=0;
 typeset -R7 tlines=0 fkb=0 tfkb=0 tMinRsz=9999999 tMaxRsz=0;
 for d1f in $d1/*
   { if [[ ! -f $d1f ]]; then continue; fi
     typeset -R7 lines=0 l1cn=0 MinRsiz=9999999 MinRnum=0 MaxRsiz=0 MaxRnum=0;
     fdus=$(\du -s $d1f | tr '\t' ' '); fkb=${fdus%% *};
     ((files+=1))                 # count files
     exec 3< $d1f                 # open file for reading
     while read -u3 line          # read next filename
       do ((lines+=1))            # increment line# in current file
          l1c=${#line}            # get line length
          l1cn=${l1c%% *}         # drop any trailing blanks
          ((l1cn+=1))             # +1 for LineFeed
          if ((l1cn <  MinRsiz)); then MinRsiz=$l1cn; MinRnum=$lines; fi
          if ((l1cn >= MaxRsiz)); then MaxRsiz=$l1cn; MaxRnum=$lines; fi
       done
     exec 3<&-               # close file of filenames
        echo "$files $lines $fkb $MinRsiz $MaxRsiz $MinRnum $MaxRnum  $d1f" >>rpts/${d2}_llc2
        ((tlines+=lines))
        ((tfkb+=fkb))
        if ((MinRsiz <  tMinRsz)); then tMinRsz=$MinRsiz; fi
        if ((MaxRsiz >= tMaxRsz)); then tMaxRsz=$MaxRsiz; fi
   }
 echo "$files $tlines $tfkb $tMinRsz $tMaxRsz                  *Totals* in $d1" >>rpts/${d2}_llc2
 echo "     - report left in --> $PWD/rpts/${d2}_llc2"
 more "rpts/${d2}_llc2"
 cfhelp  # display help screen summary of 8 scripts to count Files,Lines,KB in directories"
 #-------# export UVEXPERT=n to inhibit
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3B1. Vancouver Utility scripts for Conversion & General use

llm - list long & pipe to more

 #!/bin/ksh
 # llm - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # llm  - script to list any directory with long option & pipe into more
 echo "llm - list directory with long option & pipe to more"
 if (($# > 1));then
    echo "usage: llm directory"
    echo "       ============="
    echo "example: llm 'jcls/map*'"
    echo "         ==============="
    echo "- enclose any *wildcard patterns in single quotes"
    exit 1; fi
 #
 if  [[ -z "$1" ]] ; then d="."; else d=$1; fi
 ls -l $d | more
 #==============
 cntf=$(ls $d | wc -l)
 echo "$cntf files in $d"
 echo "use script 'llc' for file & line counts within each file"
 echo "lls - biggest files 1st, llt - newest files 1st, lltr - oldest files 1st"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3B2. Vancouver Utility scripts for Conversion & General use

lla - lists All files including hidden .files

 #!/bin/ksh
 # lla - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lla - script to list directory with ALL & LONG option & pipe into more
 #     - same as llm, but with '-a' option to list hidden .files
 #
 #usage with '*' wildcard patterns
 # lla dir/abc*      <-- does not work (will match only 1st occurrence)
 # lla 'dir/abc*'    <-- does work (enclose patterns in single quotes)
 #
 echo "lla - list All files (including .hidden) & pipe to more"
 if (($# > 1)); then
    echo "usage: lla directory"
    echo "       ============="
    echo "example: lla 'jcls/map*'"
    echo "         ==============="
    echo "- enclose any *wildcard patterns in single quotes"
    exit 1; fi
 #
 if  [[ -z "$1" ]] ; then d="."; else d=$1; fi
 ls -al $d | more
 #===============
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3C1. Vancouver Utility scripts for Conversion & General use

llt - list files, latest First

 #!/bin/ksh
 # llt - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # llt  - list directory long in date sequence with Newest first
 #      - pipe to more (show 1 screen & wait for null entry)
 #
 #usage with '*' wildcard patterns
 # llt dir/abc*      <-- does not work (will match only 1st occurrence)
 # llt 'dir/abc*'    <-- does work (enclose patterns in single quotes)
 #
 echo "llt - list directory in date sequence with Newest files 1ST"
 if (($# > 1));then
    echo "usage: llt directory"
    echo "       ============="
    echo "example: llt 'jcls/map*'"
    echo "         ==============="
    echo "- enclose any *wildcard patterns in single quotes"
    exit 1; fi
 #
 if [[ -z "$1" ]] ; then  d="."; else d=$1; fi
 ls -lt  $d | more
 #================
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3C2. Vancouver Utility scripts for Conversion & General use

lls - list files in Size order

 #!/bin/ksh
 # lls - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lls - List directory with options Long sorted by file Size & pipe to more
 #     - sorts biggest files first
 #
 #usage with '*' wildcard patterns
 # lls dir/abc*      <-- does not work (will match only 1st occurrence)
 # lls 'dir/abc*'    <-- does work (enclose patterns in single quotes)
 #
 echo "lls - list directory sorted by file-size (biggest files 1st)"
 if (($# > 1));then
    echo "usage: lls directory"
    echo "       ============="
    echo "example: lls 'cbls/car*'"
    echo "         ==============="
    echo "- enclose any *wildcard patterns in single quotes"
    exit 1; fi
 #
 if  [[ -z "$1" ]] ; then d="."; else d=$1; fi
 ls -lS $d | more
 #===============
 exit 0
 #

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3C3. Vancouver Utility scripts for Conversion & General use

llu - list files Unsorted

 #!/bin/ksh
 # llu - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # llu - list directory with options Long & Unsorted & pipe into More
 #
 #usage with '*' wildcard patterns
 # llu dir/abc*      <-- does not work (will match only 1st occurrence)
 # llu 'dir/abc*'    <-- does work (enclose patterns in single quotes)
 #
 echo "llu - list directory with options Long & Unsorted & pipe to More"
 if (($# > 1));then
    echo "usage: llu directory"
    echo "       ============="
    echo "example: llu 'jcls/car*'"
    echo "         ==============="
    echo "- enclose any *wildcard patterns in single quotes"
    exit 1; fi
 #
 if  [[ -z "$1" ]] ; then d="."; else d=$1; fi
 ls -lU $d | more
 #===============
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3D1. Vancouver Utility scripts for Conversion & General use

lld - list directories only (omit files)

 #!/bin/ksh
 # lld - Korn shell script from UVSI stored in: /home/uvadm/sf/util/...
 # lld - list directories ONLY with long option,
 #     - by Owen Townsend, UV Software, Aug13/2019
 #
 d1="$1";
 if [[ -z "$d1" ]]; then d1=$PWD; fi
 if [[ ! -d $d1 ]]; then
    echo "usage: lld directory    <-- arg1 must be a directory"
    echo "       ============="
    echo "example: lld $UV/ctl    <-- will get 7 dirs in 67 entries"
    echo "         ==========="
    exit 1; fi
 typeset -R5 lines=00000 dirs=00000
 test -d tmp || mkdir tmp
 d2=$(echo $d1 | tr '/' '_')
 echo "lld - list directories only in $d1" >tmp/${d2}_lld
 #
 ls -l $d1 >tmp/$d2   # list dir entries into a tmp/file
 exec 3< tmp/$d2      # open tmp/file
 while read -u3 line  # read lines from tmp/file
   do
      ((lines+=1))
      if [[ "$line" == d* ]]; then
         echo $line >>tmp/${d2}_lld
         ((dirs+=1))
      fi
   done
 exec 3<&-           # close tmp/file
 echo "$dirs directories in $lines entries of $d1" >>tmp/${d2}_lld
 more tmp/${d2}_lld
 exit 0
 #     ----- sample command & output -----
 # lld $UV/ctl - list directories only in $UV/ctl
 # ===========
 # drwxrwxr-x 3 uvadm apps 4096 Jan 28 2019 add
 # drwxrwxr-x 2 uvadm apps 4096 Sep 24 2018 addold
 # drwxrwxr-x 2 uvadm apps 4096 May 7 2018 rexxcfgs
 # drwxrwxr-x 2 uvadm apps 4096 Sep 24 2018 tmp
 #     4 directories in    68 entries of ctl

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3E1. Vancouver Utility scripts for Conversion & General use

llr - list files & subdirs Recursively

 #!/bin/ksh
 # llr - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # llr  - list a directory with long & Recursive option & pipe into more
 #      - Recursive option shows contents of all sub-directories
 #
 #usage with '*' wildcard patterns
 # llr dir/abc*      <-- does not work (will match only 1st occurrence)
 # llr 'dir/abc*'    <-- does work (enclose patterns in single quotes)
 #
 echo "llr - list directory with long & Recursive options & pipe to more"
 if (($# > 1));then
    echo "usage: llr directory"
    echo "       ============="
    echo "example: llr 'jcls/map*'"
    echo "         ==============="
    echo "- enclose any *wildcard patterns in single quotes"
    exit 1; fi
 #
 if [[ -z "$1" ]] ; then d="."; else d=$1; fi
 ls -lR $d | more
 #===============
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3F1. Vancouver Utility scripts for Conversion & General use

lslp - list filenames in directory & print immediately

 #!/bin/ksh
 # lslp - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lslp - list directory (long) & print via uvlp12
 #
 echo "lslp - list directory (long) & print via uvlp12"
 d1="$1";
 if [[ "$d1" == "." ]]; then d1=$(pwd); fi
 if [[ ! -d "$d1" ]]; then
    echo "usage: lslp directory"
    echo "       =============="
    exit 9; fi
 #
 if [[ ! -d tmp ]]; then mkdir tmp; fi
 dir=$(basename $d1)
 ls -l $1 > tmp/$dir
 files=$(ls $1 | wc -l)
 echo "directory: $dir, files = $files" >>tmp/$dir
 #================================================
 uvlp12 tmp/$dir
 #==============
 exit 0

Note re 'uvlp12'

This is 1 of the few scripts in this section (scripts1.doc) that call a Vancouver Utility vs a unix/linux command. You could change 'uvlp12' to 'lp'.

'uvlp12' is a script that calls the 'uvlist' utility, which is a Vancouver Utility C program that inserts page headings with filename, userid, page#, etc.

These scripts also insert PCL5 commands for laser printing with margins for 3 hole punching & for various cpi (in this case 12 cpi to get 90 chars across).

You can see many more 'uvlp__' scripts at uvlist.htm.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3G1. Vancouver Utility scripts for Conversion & General use

spreadA - list filenames 4-up across the page

 #!/bin/ksh
 # spreadA - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # spreadA - list directory in multi-columns with options for columns & width
 #
 # - spreadA uses 'ls' to generate a tmp file, input to uvcopy job spread1
 # - uvcopy/spread1 prompts with options for no of columns & column-width
 # - report is created in 5up subdir with headings for dirname & date/time
 # - prompt for report disposition: vi, more, cat, lp, uvlp, etc (or null)
 # - report is left in subdir 5up named the same as the specified directory
 #
 echo "spreadA - list directory in multi-columns, options for columns & width"
 echo "        - will leave report in subdir 5up/dirname.5up"
 dir="$1"
 if [[ ! -d "$dir" ]]; then
    echo "usage: spreadA directory"
    echo "       ================="
    exit 99; fi
 #
 if [[ ! -d tmp ]]; then mkdir tmp; fi
 if [[ ! -d 5up ]]; then mkdir 5up; fi
 ls $dir >tmp/$dir         # create file of filenames in specified directory
 #
 # create full path name of directory, to show on report (if not already)
 if [[ $dir = /* ]]; then DIR=$dir; else DIR=${PWD}/$dir; fi
 export DIR=$DIR           # export full path dirname for uvcopy job hdngs
 export JOBID=spreadA      # export JOBID (spreadA) for uvcopy job hdngs
 # use basename of directory as uvcopy job output filename in 5up
 dirfile=$(basename $dir)
 #
 uvcopy spread1,fili1=tmp/$dir,filo1=5up/${dirfile}.5up
 #=====================================================
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3F2. Vancouver Utility scripts for Conversion & General use

script spreadA calls 'uvcopy job' spread1

'uvcopy' is Vancouver Utility C program for general purpose data manipulation. It is an interpreter for the instructions in a separate instruction file. 'spread1' is found via 'PFPATH', defined in the profile ('ADMjobs.doc#1C3').

The 'spreadA' script (above) uses 'ls' to capture all filenames in a tmp/file, and calls uvcopy job 'spread1' to format the filenames across the page.

Note the options 'uop=c18n4' at the end of the uvcopy command line above. 'c18' sets Column width to 18 & 'n4' sets Number of columns to 4.

See the uvcopy job 'spread1' listed at UVjobs1.htm#U6.

Also see sample operating instructions & a sample report at UVjobs1.htm#L1.

uvcopy can perform complex data manipulations with very few instructions. It is an essential part of the mainframe conversions because it can do many things not possible with the standard unix/linux utilities (such as processing packed decimal fields & handling Indexed files compatible with Micro Focus COBOL). See more about uvcopy at uvcopy1.htm.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3H1. Vancouver Utility scripts for Conversion & General use

longest1 - find longest line in a text file

 #!/bin/ksh
 # longest1 - find longest line in a file
 #
 echo "longest1 - find longest line in a file"
 file="$1";
 if [[ ! -f "$file" ]]; then
    echo "usage: longest1 filename"
    echo "       ================="
    echo " - arg1 must be a file"
    echo " "
    echo "example: longest1 sf/util/longest1"
    echo "         ========================="
    exit 1; fi
 #
 integer lines=0;        # init line counter
 integer longest=0;      # init longest line char count
 longestline="";         # init longest line data
 exec 3< $file           # open file for reading
 while read -u3 line     # read next filename
   do ((lines+=1))
      l1c=${#line}
      l1cn=${l1c%% *}
      if ((l1cn > longest)); then
         longest=l1cn; longestline=$line; fi
   done
 exec 3<&-               # close file of filenames
 echo "longest line: $longest characters in $file, contents:"
 echo "$longestline"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

3I1. Vancouver Utility scripts for Conversion & General use

dtree - create directory tree illustration

 #!/bin/ksh
 # dtree - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # dtree - list a directory tree
 #       - contributed by Howard Lobsinger (Peacock Engineering, Montreal)
 #usage: dtree directory
 #       ===============
 D=${1:-`pwd`}
 (cd $D; pwd)
 find $D -type d -print | sort |
 sed -e "s,^$D,,"\
     -e "/^$/d"\
     -e "s,[^/]*/\([^/]*\)$,\:-----\1,"\
     -e "s,[^/]*/,:     ,g" | more
 exit 0

dtree example

 #1. Login as yourself (or anybody)

 #2. dtree /home/uvadm/demo
     ======================

 /home/uvadm/demo
 :-----archive
 :     :-----bakup_dat1
 :     :-----bakup_dat2
 :-----dat1
 :-----dat2
 :-----ftps
 :-----jcl0
 :-----jcl2
 :-----jcl3
 :-----pf
 :-----rpts
 :-----selects
 :-----sf
 :-----sql0
 :-----sqls
 :-----stats
 :-----tf
 :-----tmp

We illustrate using the demo directory - a small directory in /home/uvadm.

 #3. dtree /home/uvadm   <-- would create about 600 lines
     =================     - not shown here, but you can run & observe

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_4 Vancouver Utility scripts for Conversion & General use

scripts to process all files in directory

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part 4 Contents continued

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4A1. scripts to process all files in directory

alldiff2 - compare all text files in 2 directories

 #!/bin/ksh
 # alldiff2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # alldiff2 - script to compare all text files in 1 directory to a 2nd directory
 #          - redirects output to tmp/dir2.dif & prompts to view/print/etc
 # alldiff  - original version, dsiplays output on screen only
 #          - alldiff2 better, allows subsequent --> vi tmp/dir2.dif <--
 # alldiff2sub - compare all files in all subdirs in a pair of superdirs
 d1="$1"; d2="$2";
 echo "alldiff2 - compare all text files in 2 subdirs & write dif report to tmp"
 if [[ -d "$1" && -d $2 ]]; then :
 else echo "usage: alldiff dir1 dir2"
      echo "       ================="
      echo "example: alldiff cbls.old cbls"
      echo "         ====================="
      exit 1; fi
 d2b=$(basename $d2)     # get basename of dir2 (drop any preceding /path/...)
 log=tmp/$d2b.dif        # make name for output log file
 >$log                   #init logfile in tmp subdir w same name as dir2 + .dif
 integer fils=0; difs=0; # fils=Total Files, difs=Total Diffs
 integer nfs=0;          # nfs=mismatch in 2nd dir or not a file
 typeset -L9 flns;       # flns=Lines in current file
 for i in $d1/*
 do
   ((fils+=1))
   f=${i##*/}
   if [[ -f $d2/$f ]]; then
      diff -a -bBw $d1/$f $d2/$f >>$log
      #=================================
      if [[ $? -gt 0 ]]; then
         fln1=$(wc -l $d2/$f)
         flns=${fln1%% $d2/$f}
         echo "diff file# $fils - $d1/... vs $d2/$f lines=$flns" >>$log
         echo " " >>$log
         ((difs+=1))
      fi
   else
      ((nfs+=1))
      echo "     file# $fils $d1/$f - nomatch in 2nd dir or not a file" >>$log
   fi
 done
 tlns=$(wc -l $log)      # capture line count
 echo "$nfs missmatches in 2nd dir or not files" >>$log
 echo "$difs different of $fils files compared $d1 to $d2" >>$log
 echo "$nfs missmatches in 2nd dir or not files"
 ((difs+=nfs))
 echo "$difs diff of $fils files in $d1 & $d2, report is: $tlns"
 echo "--> use uvlp12,uvlp14,uvlp16 to laser print at 12,14,16 cpi"
 echo "--> enter command (vi,cat,more,uvlp12,etc, or null)"
 read ans
 if [[ ! "$ans" = "" ]]; then $ans $log; fi
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4A2. scripts to process all files in directory

alldiff3 - same as alldiff2 + drop insignificant diffs

 #!/bin/ksh
 # alldiff3 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # alldiff3 - script to compare all text files in 1 directory to a 2nd directory
 #          - redirects output to tmp/dir2.dif & prompts to view/print/etc
 #Jul11/17 - add diffdrop2 script to drop insignificant lines on JCL/script diffs
 #         - ver:, UVSW, diff file# line, empty diffs after pattern drops
 #
 d1="$1"; d2="$2";
 echo "alldiff3 - compare all files in 2 subdirs & write all diffs to tmp/dir2.dif"
 echo " - then drop diffs due to time stamps & table of patterns (prompt to add)"
 if [[ -d "$1" && -d $2 ]]; then :
 else echo "usage: alldiff3 dir1 dir2"
      echo "       =================="
      echo "example: alldiff3 cbls.old cbls"
      echo "         ======================"
      exit 1; fi
 #
 d2b=$(basename $d2)   # get basename of dir2 (drop any preceding /path/...)
 log=tmp/$d2b.dif      # make name for output log file
 log2=tmp1/$d2b.dif    # make name for diffdrop2 outputfile
 >$log                 #init logfile in tmp subdir w same name as dir2 + .dif
 x=0; y=0;
 for i in $d1/*
 do
   let x=x+1
   typ=$(file $i)
   if [[  $typ == *text* || $typ == *script* || $typ == *data* ]]
      then f=${i##*/}
           diff -a -bBw $d1/$f $d2/$f >>$log
           #=================================
           if [[ $? -gt 0 ]]; then
              echo "diff file# $x - $d1/... vs $d2/$f" >>$log
              echo " " >>$log
              let y=y+1
           fi
      else echo " file# $x $typ - NOT text/script file" >>$log
   fi
 done
 lines=$(wc -l $log)      # capture line count
 echo "$y different of $x files compared $d1 to $d2" >>$log
 echo "$y diff $x files in $d1 & $d2, report: $log $lines lines"
 echo "--> enter additional drop patterns 'arg1=xxx,arg2=yyy' or null"
 read args
 uvcopy diffdrop2,fili1=$log,filo1=$log2,$args
 #============================================
 echo "--> enter command to view (vi,cat,more,uvlp12,etc,or null)"
 read cmd
 if [[ ! "$cmd" = "" ]]; then $cmd $log2; fi
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4A3. scripts to process all files in directory

alldiff2sub - compare all files in all subdirs in 2 superdirs

 #!/bin/ksh
 # alldiff2sub - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # alldiff2sub - run alldiff2 for all subdirs in a pair of superdirs
 #             - by Owen Townsend, UV Software, Oct22/2013
 #
 # Writes diff reports to tmp/... named as subdir1.dif, subdir2.dif, etc
 #
 # Calls 'alldiff2' for each pair of subdirs
 # - alldiff2 prompts for (vi,more,etc) at end each pair)
 # - so best to enter null until end all pairs, then vi tmp/*
 #
 echo "alldiff2sub - run alldiff2 for all subdirs in a pair of superdirs"
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
 else echo "usage: alldiff2sub superdir1 superdir2"
      echo "       ==============================="
      echo "- arg1 & arg2 must be directories with multiple subdirs"
      exit 1; fi
 x=0;
 for d1sd in $d1/*
  { sd=$(basename $d1sd)
    if [[ -d $d1/$sd ]]; then
       let x=x+1
       echo "$x alldiff2 $d1/$sd to $d2/$sd"
       alldiff2 $d1/$sd $d2/$sd
       #=======================
    fi
  }
 echo "$x alldiff2 reports created in tmp/... for superdirs $d1 & $d2"
 exit

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4B1. scripts to process all files in directory

allcmp - compare all packed/binary files in 2 directories

 #!/bin/ksh
 # allcmp - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # allcmp - script to execute the UNIX 'cmp' command to compare
 #          corresponding files in 2 directories
 #
 #usage:  allcmp dir1 dir2 ['pattern']     (pattern defaults to *)
 #        ============================
 #
 #example#1: allcmp tmp1 tmp2         - compare all files in tmp1 & tmp2
 #           ================
 #
 #example#1: allcmp tmp1 tmp2 'a*'    - compare all files in tmp1 & tmp2
 #           =====================      whose names begin with 'a'
 #
 echo "allcmp - compare files in 2 directories, using unix 'cmp'"
 echo "- for files with packed/binary & no LineFeeds (use diff for text files)"
 #
 if [[ -d "$1" && -d "$2" ]]; then :
    else echo "usage: allcmp dir1 dir2"
         echo "       ================"
         echo "example1: allcmp gl.old gl"
         echo "          ================"
         echo "example2: allcmp mstr.old mstr 'gl*'"
         echo "          =========================="
         echo " - arg3 optional prefix* must be in single quotes"
         exit 99; fi
 #
 pat='*'            # default pattern to '*'
 if [[ -n "$3" ]]; then pat="$3"; fi
 x=0;y=0;
 for i in $1/$pat
   { let x=x+1
     f=${i##*/}
     cmp $1/$f $2/$f
     #==============
     if (($? != 0)); then
        echo "file# $x cmp $1/$f $2/$f"
        echo " ";       # generate blank line to highlight diffs
        let y=y+1
     fi
   }
 echo "$y mismatches of $x total files compared in $1 & $2"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4C1. scripts to process all files in directory

all2lower - translate to lower copying all files to 2nd dir

 #!/bin/ksh
 # all2lower - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # all2lower - script to translate all files to lower case
 #             while copying from 1 directory to a 2nd directory
 #
 echo "all2lower - translate all files in directory to lower case in 2nd dir"
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
 else echo "usage: all2lower dir1 dir2"
      echo "       ==================="
      echo " - arg1 & arg2 must be directories"
 exit 1; fi
 integer fn=0
 for df in $d1/*
  {
   f=$(basename $df)
   ((fn+=1))
   echo "file# $fn - $d1/$f"
    tr [:upper:] [:lower:] <$d1/$f >$d2/$f
  }
 echo "$fn total files translated to lower case from $d1 to $d2"
 exit

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4C2. scripts to process all files in directory

all2upper - translate to UPPER copying all files to 2nd dir

 #!/bin/ksh
 # all2upper - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # all2upper - script to translate all files to UPPER case
 #             while copying from 1 directory to a 2nd directory
 #
 echo "all2upper - translate all files in directory to UPPER case in 2nd dir"
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
 else echo "usage: all2upper dir1 dir2"
      echo "       ==================="
      echo " - arg1 & arg2 must be directories"
 exit 1; fi
 integer fn=0
 for df in $d1/*
  {
   f=$(basename $df)
   ((fn+=1))
   echo "file# $fn - $d1/$f"
    tr [:lower:] [:upper:] <$d1/$f >$d2/$f
  }
 echo "$fn total files translated to UPPER case from $d1 to $d2"
 exit

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4D1. scripts to process all files in directory

rmzf - remove zero-length (empty) files from a directory

 #!/bin/ksh
 # rmzf - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rmzf - script to remove zero-length files from a directory
 #
 # use after jobs that process 100's of files
 # but produce output for only a small no of the input files
 # this will remove the zero-length files from the directory
 # many other methods wont work because you get arg list too long
 #
 echo "rmzf - remove all zero-length (empty) files in directory"
 if [[ ! -d "$1" ]]; then
    echo "usage: rmzf directory"
    echo "       =============="
    exit 99; fi
 #
 x=0; y=0
 for i in $1/*
 do
   if [[ ! -s $i ]]; then
      rm -f $i
      #=======
      let x=x+1
   fi
   let y=y+1
 done
 echo "removed $x files of $y files in directory $1"
 exit

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4E1. scripts to process all files in directory

allrm - remove all files in a directory

 #!/bin/ksh
 # allrm - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # allrm - script to remove all files from a directory
 #       - use if no of files greater than allowed by UNIX shell (512,1024,?)
 #         (when you get errmsg "arg list too long")
 #       - also provides audit trail with file-counts
 #
 #usage: allrm directory
 #       ===============
 #
 reply="n"
 until [ "$reply" = "y" ]
   do echo "Are you sure you want to remove files from $1 ? y/n"
      read reply
   done
 #
 if [ -d "$1" ]; then :
 else echo "USAGE: allrm dir1 "
 exit 1; fi
 x=0;
 for i in $1/*
   { let x=x+1
     echo "removing file# $x - $i "
     rm -f $i
     #=======
   }
 echo "total $x files removed from $1"
 exit

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4F1. scripts to process all files in directory

allcancel - cancel all printer requests for current user

 #!/bin/ksh
 # allcancel - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # allcancel - cancel all printer requests for the current user
 #           - no arguments required
 #           - very handy when you need to cancel all your print requests
 #             (because cancel does not provide an option for all requests)
 #
 if [[ ! -d tmp ]]; then mkdir tmp; fi  # if tmpdir not present, make it
 lpstat >tmp/allcan1                    # request-id is 1st word on each line
 cut -f1 -d' ' tmp/allcan1 >tmp/allcan2 # cut 1st word to separate file
 # now open the file & read line contents into variable $reqid
 exec 3< tmp/allcan2                    # open file for reading
 while read -u3 reqid                   # read current request id
   do
     cancel $reqid                      # cancel request id
     echo "$reqid cancelled"
     #======================
   done
 exec 3<&-                              # close the file
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4G1. CNVaids: Unix/Linux Aids

scripts to list/copy/move/remove old/new files

These scripts are recursive, they process an entire directory tree, descending all levels, looking for files older/newer than a specified no of days.

help screens

You can see a help screen for each script, by entering the script-name only, without its required arguments (omit the directory & no of days).

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4G2. CNVaids: Unix/Linux Aids

demo lsold1 & rmold1

 #1 l data    <-- list data directory to see file creation dates
 =========

 -rw-rw-r-- 1 berks apps 186208 Mar 22  2013 abilitydeletesfixa
 -rw-rw-r-- 1 berks apps   2081 Dec 26  2012 berkscodes
 -rw-rw-r-- 1 berks apps    980 Sep  8  2010 empnomagmed100_100908
 -rw-rw-r-- 1 berks apps    193 Sep 14  2010 empnotice_091310
 -rw-rw-r-- 1 berks apps    205 Sep 26  2011 networkgetin
 -rw-rw-r-- 1 berks apps    180 Nov 11 09:53 othperscomptime
 -rw-r--r-- 1 berks apps  26584 Nov 24  2010 securedsite.PDF
 -rw-rw-r-- 1 berks apps    139 Sep 26  2011 sharepointnotes
 -rw-rw-r-- 1 berks apps     84 Jun 19  2013 todo

 #2. lsold1 data 730   <-- see which files will be removed
     ===============

 -rw-rw-r-- 1 berks apps 193 Sep 14  2010 data/empnotice_091310
 -rw-rw-r-- 1 berks apps 139 Sep 26  2011 data/sharepointnotes
 -rw-r--r-- 1 berks apps 26584 Nov 24  2010 data/securedsite.PDF
 -rw-rw-r-- 1 berks apps 980 Sep  8  2010 data/empnomagmed100_100908
 -rw-rw-r-- 1 berks apps 205 Sep 26  2011 data/networkgetin

 #3. rmold1 data +730 -f  <-- remove files older than 730 days (2 years)
     ===================      '-f' option Forces removal with no prompt
                              '-i' option would prompt to remove each file

 #4. l data      <-- list to see files remaining
     ======

 -rw-rw-r-- 1 berks apps 186208 Mar 22  2013 abilitydeletesfixa
 -rw-rw-r-- 1 berks apps   2081 Dec 26  2012 berkscodes
 -rw-rw-r-- 1 berks apps    180 Nov 11 09:53 othperscomptime
 -rw-rw-r-- 1 berks apps     84 Jun 19  2013 todo

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4H1. CNVaids: Unix/Linux Aids

lsold1 - list files older than x days

 #!/bin/ksh
 # lsold1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lsold1 - list files older than a specified number of days
 #
 if [[ -d "$1" && -n "$2" ]]; then :
    else echo "usage: lsold1 directory days"
         echo "       ====================="
         echo "example: lsold1 /u2/apps/data 30"
         echo "         ======================="
         echo "- list files in directory tree older than 30 days"
         echo " "
         echo "example2: lsold1 . 30  (use '.' if in directory)"
         echo "          ==========="
         echo "- Also see 'lsold1p' that allows a filename matching pattern"
         exit 1; fi
 days="+$2"
 #
 find $1 -mtime $days -type f -exec ls -l {} \;
 #=============================================
 # - list files older than a specified number of days
 exit 0

lsold2 - list files created/modified between 2 days

 #!/bin/ksh
 # lsold2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lsold2 - list files modified/created between 2 days
 #
 if [[ -d "$1" && -n "$2" && -n "$3" ]]; then :
    else echo "usage: lsold2 directory days1 days2"
         echo "       ============================"
         echo "example: lsold2 /u2/apps/data 30 60"
         echo "         =========================="
         echo "- list files in directory older than 30 days & newer than 60 days"
         echo " "
         echo "example2: lsold2 . 30 60 "
         echo "         ======================"
         echo "- list files in current dir 30 to 60 days old"
         echo "Note - also see 'lsold2p' allows filename matching pattern"
         exit 1; fi
 #
 from=+$2; to=-$3
 #
 find $1 -mtime $from -mtime $to -exec ls -l {} \;
 #================================================
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4H2. CNVaids: Unix/Linux Aids

lsnew1 - list files newer than x days

 #!/bin/ksh
 # lsnew1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lsnew1 - list files newer than or modified within specified number of days
 #
 if [[ -d "$1" && -n "$2" ]]; then :
    else echo "usage: lsnew1 directory days"
         echo "       ====================="
         echo "example: lsnew1 /u2/apps/data 30"
         echo "         ======================="
         echo "- list files in directory tree modified in last 30 days"
         echo " "
         echo "example2: lsnew1 . 30  (use '.' if in directory)"
         echo "          ==========="
         echo "- Also see 'lsnew1p' that allows a filename matching pattern"
         exit 1; fi
 days="-$2"
 #
 find $1 -mtime $days -type f -exec ls -l {} \;
 #=============================================
 # - list files newer than or modified within specified number of days
 exit 0

lsnew2 - list files created/modified between 2 days

 #!/bin/ksh
 # lsnew2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lsnew2 - list files in a subdir newer than ctl/lsnew2.ctl
 #        - first create a control file with current time
 #        - then lsnew2 lists files in a subdir newer than the control file
 #
 if [[ ! -d "$1" ]]; then
    echo "prep:  touch ctl/lsnew2.ctl  <-- set current time on control file"
    echo "       ===================="
    echo "prep:  touch -d 10:00:00 ctl/lsnew2.ctl <-- set desired time"
    echo "       ================================     10AM for example"
    echo "usage: lsnew2 directory      <-- list files newer than ctl file"
    echo "       ================"
    echo "example: lsnew2 data"
    echo "         ==========="
    echo " - list files in data subdir newer than ctl/lsnew2.ctl"
    exit 1; fi
 #
 find $1 -newer ctl/lsnew2.ctl -type f -exec ls -l {} \;
 #======================================================
 # - list files in arg1 subdir newer than control file ctl/lsnew2.ctl
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4H3. CNVaids: Unix/Linux Aids

rmold1 - remove files older than x days

 #!/bin/ksh
 # rmold1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rmold1 - remove files from a specified directory (& all subdirs),
 #          that are older(+) than a specified number of days
 #        - option to prompt y/n for each file to be removed
 #
 # 3 variations of this script as follows:
 #*rmold1 (this) - remove old files, option to prompt for each file to be removed
 # rmold2 - lists all old files, then prompts for each old file to be removed
 # rmold3 - lists all old files, then prompts once to remove all old files
 #
 # Also see:
 # lsold1 - list files older than specified number of days
 #        - might run this script before 'rmold1' (removes files)
 #          to prove files to be removed are as intended
 # mvold1 - might run this script instead of 'rmold1' (removes files)
 #          so you could recover files from the move to dir if required
 #
 if [[ -d "$1" && "$2" == +* && ("$3" == "-i" || "$3" == "-f") ]]; then :
    else echo "usage: rmold1 directory +days option"
         echo "       ============================="
         echo "example rmold1 history +730 -i"
         echo "        ======================"
         echo " - remove files over 2 yrs old, prompting for confirmation"
         echo " - arg1 must be a directory"
         echo " - arg2 must start with + (older than)"
         echo " - arg3 must be '-i' for confirmation, '-f' for none"
         exit 1; fi
 #
 find $1 -mtime $2 -exec rm $3 {} \;
 #==================================
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4H4. CNVaids: Unix/Linux Aids

rmold2 - lists old files, then prompt to remove EACH

 #!/bin/ksh
 # rmold2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rmold2 - remove files from a specified directory (& all subdirs),
 #          that are older(+) than a specified number of days
 #        - lists all old files & prompts y/n for each file to be removed
 #        - prompts y/n for each file to be removed
 #        - 3 variations of this script as follows:
 #
 # rmold1 - prompts y/n for each old file to be removed
 #*rmold2 (this script) - lists all old files & prompts for each file to remove
 # rmold3 - lists all old files & prompts once to remove all old files
 #
 if [[ -d "$1" && "$2" == +* ]]; then :
    else echo "usage: rmold2 directory +days"
         echo "       ======================"
         echo "example: rmold2 history +730  <-- remove files over 2 yrs old"
         echo "         ==================="
         echo " - arg1 must be a directory"
         echo " - arg2 must start with + (older than)"
         exit 1; fi
 #
 find $1 -mtime $2 -exec ls -l {} \;  #<-- list all old files
 #==================================
 echo "files older than $2 days listed above, enter to remove (prompt for each)"
 read reply
 #
 find $1 -mtime $2 -exec rm -i {} \;  #<-- remove old files with prompt for each
 #==================================
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4H5. CNVaids: Unix/Linux Aids

rmold3 - lists old files, then prompt to remove ALL

 #!/bin/ksh
 # rmold3 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rmold3 - remove files from a specified directory (& all subdirs),
 #          that are older(+) than a specified number of days
 #        - does not prompt y/n for each file to be removed (rmold1 for prompt)
 #          but lists files to be removed & prompts once for all to be removed
 #        - 3 variations of this script as follows:
 #
 # rmold1 - prompts y/n for each old file to be removed
 # rmold2 - lists all old files & prompts for each file to remove
 #*rmold3 (this script) - lists all old files & prompts once to remove all old
 #
 if [[ -d "$1" && "$2" == +* ]]; then :
    else echo "usage: rmold3 directory +days"
         echo "       ======================"
         echo "example: rmold3 history +730  <-- remove files over 2 yrs old"
         echo "         ==================="
         echo " - arg1 must be a directory"
         echo " - arg2 must start with + (older than)"
         exit 1; fi
 #
 # list files to be removed, prompt once y/n, then execute remove all
 #
 find $1 -mtime $2 -exec ls -l {} \;  #<-- list all old files
 #==================================
 #
 reply="x"
 until [[ "$reply" = "y" || "$reply" = "n" ]]
   do echo "remove files (listed above) from $1 ? y/n"
      read reply
   done
 #
 if [[ "$reply" = "y" ]]; then
    find $1 -mtime $2 -exec rm -f {} \;  #<-- remove all old files (no prompts)
    #==================================
 fi
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

4I1. scripts to process all files in directory

editmfp1 - edit multi files in directory in Size order

 #!/bin/ksh
 # editmfp1 - edit Multi-files containing a specified Pattern
 #          - by Owen Townsend, UV Software, feb03/2013
 #
 echo "editmfp1 - edit Multi-files containing a specified Pattern"
 d="$1"; p="$2";
 if [[ -d "$d" && -n "$p" ]]; then :
    else echo "usage: editmfp1 directory pattern"
         echo "       =========================="
         echo "examfple: editmfp1 in1fix 'E#'"
         echo "          ===================="
         echo "- arg1 must be a directory, arg2 must be specified"
         echo "Enter :q for next file (not :n)"
         echo "- prompt every 5th file to allow quit before all files edited"
         exit 99; fi
 integer tf=0 ef=0 ef1=0;
 #
 for df in $d/*
   { ((tf++))            # count Total Files
     grep -q $p $df      # grep -q no output, set status 0 if pattern found
     if [[ $? -eq 0 ]];
        then vi $df      # edit files with match to pattern
             ((ef++)); ef1=$((ef%5))
             if [[ $ef1 == 0 ]]; then
                print "\nenter null to edit file# $ef OR 'Q' to Quit\n"
                read reply
                if [[ "$reply" == "Q" ]]; then exit 9; fi
             fi
     fi
   }
 echo "End editmfp1 - $ef files selected of $tf total files in directory"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_5 Vancouver Utility scripts for Conversion & General use

Counting Files,Lines,& KB in directories

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5A1. Counting Files,Lines,& KB in directories

v12 - help menu for counting scripts

'help' screens are stored at $UV/help/... & it is in the profile PATH, so you can display the help/menu screen anytime anywhere, for example:

 uvhelp   <-- display main menu of help screens
 ======     - determine desired sub-menu code 'v12' in this case,
            - then quit the uvhelp main menu, & enter the sub-menu code 'v12'

 v12      <-- display sub-menu of 'counting lines & files' (shown below)
 ====

v12 - counting Lines,Files,& KB in Directories

1. cfl - Count Lines in 1 File
2. cfd - Count Files & kiloBytes in a Directory
3. cfdt - Totals-Only version of Count Files,Lines,& KiloBytes in a Directory
4. cfdpf - Count Files in a Directory with a Pattern [or not] in filenames
5. cfdpl - Count Files in a Directory with a Pattern [or not] on any line in any file
6. cfdd - Count Files,Lines,& KB in a ALL Sub-Dirs in a Super-Directory
7. cfddt - Count Files,Lines,& KB in a ALL Sub-Dirs in a Super-Directory Totals-Only
8. cfddf - Count Files,Lines,KB in ALL Sub-Dirs in a Super-Dir + 1st few files
9. cfdmm - List Directory: File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#, Dir/Filename
   

   Help screens & sample reports built-in to scripts

You can see a help screen for each script, by entering the script-name only, without required arguments (omit directory). For example, if we enter just 'cfl'

 cfl     <-- enter script name without arguments (directory)
 ===       - displays ERROR msg, Correct Format,& sample report as follows:

 cfl - count lines in 1 file
 cfl filename                  <-- Usage ERROR, arg1 MUST be a FILE
 ============
 cfl /home/uvadm/src/uvcopy.c  <-- Correct Format & Sample Report
 ============================
  24400 lines,   1072 KB in src/uvcopy.c
                         Report--> /home/uvadm/rpts/src_uvcopy.c_cfl

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5B1. Counting Files,Lines,& KB in directories

We will now list the scripts in the same sequence as the v12 help screen menu. Note the help screens & sample reports at the begining of each script. If you have the utilities installed, you can repeat the demos documented on the help screen you get by entering the script name only.

See http://uvsoftware.ca/uvdemos.htm_Part_8 which sugegsts you setup a demo/ directory & copy the $UV/demo/* files to your $HOME/demo/... You would then:

1. login userxx --> /home/userxx
2. cd demo <-- change into demo/ directory
3. cfl <-- run 1st script w/o arg1 directory to see help & sample
4. cfl $UV/src/uvcopy.c <-- rerun with suggested demo dir/file
5. cfl your dir/file <-- then try with your files & directories
   

   cfl - Count Lines & KiloBytes in a File

 #!/bin/ksh
 # cfl - Vancouver Utility script stored at /home/uvadm/sf/util/cfl
 # cfl - Count Lines & kiloBytes in 1 File
 #     - by Owen Townsend, UV Software, updated July 2019
 #
 f1="$1"
 if [[ ! -f "$f1" ]]; then
    echo "cfl filename          <-- Usage ERROR, arg1 MUST be a FILE"
    echo "============"
    echo " - enter 'y' to see sample demo command & report, null to quit"; read reply
    if [[ "$reply" != "y" ]]; then exit 1; fi
    echo " "
    echo "cfl $UV/src/uvcopy.c  <-- Correct Format & Sample Report"
    echo "===================="
    echo "cfl - count Lines & KioloBytes in 1 file"
    echo " 24400 lines,   1072 KB in src/uvcopy.c"
    echo "                        Report--> /home/uvadm/rpts/src_uvcopy.c_cfl"
    exit 9; fi
 #
 typeset -R6 lines fkb
 test -d rpts || mkdir rpts
 f2=$(echo $f1 | tr '/' '_')
 echo "cfl $* - count lines in 1 file" >rpts/${f2}_cfl
 lines=$(wc -l <$f1)
 fdus=$(\du -s $f1 | tr '\t' ' '); fkb=${fdus%% *};
 echo "$lines lines, $fkb KB in $f1"  >>rpts/${f2}_cfl
 echo "Report left in --> $PWD/rpts/${f2}_cfl" >>rpts/${f2}_cfl
 more rpts/${f2}_cfl
 cfhelp  # display help screen summary of 8 scripts to count Files,Lines,KB in directories"
 #-------# export UVEXPERT=n to inhibit
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5C1. Counting Files,Lines,& KB in directories

cfd - Count Files & kiloBytes in a Directory

 #!/bin/ksh
 # cfd - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # cfd - Count Files & kiloBytes in a Directory
 #     - by Owen Townsend, UV Software, updated July 2019
 d1="$1";
 if [[ ! -d "$d1" ]]; then
    echo "cfd directory   <-- Usage ERROR, arg1 MUST be a DIRECTORY"
    echo "============="
    echo " - enter 'y' to see sample demo command & report, null to quit"; read reply
    if [[ "$reply" != "y" ]]; then exit 1; fi
    echo " "
    echo "cfd dat1        <-- Correct Format & Sample Report"
    echo "========"
    echo "cfd - Count Files,Lines,& KiloBytes in a Directory"
    echo "File#    Lines      KB  Directory/Filename 20190722:0819"
    echo "     1     335      32  dat1/CanadaMPs"
    echo "     2      13       4  dat1/CAprovinces"
    echo "     3       8       4  dat1/nameadrs2"
    echo "     4     305      28  dat1/UScities"
    echo "     5     539      40  dat1/UScongress"
    echo "     6      50       4  dat1/USstates"
    echo "******    1250     112  *Totals* in Directory /home/uvadm/demo/dat1"
    exit 1; fi
 #
 typeset -R6 files lines tlines fkb fkbt
 test -d rpts || mkdir rpts
 d2=$(echo $d1 | tr '/' '_')
 echo "cfd $* - Count Files,Lines,KB in Directory" >rpts/${d2}_cfd
 echo "File#    Lines      KB  Directory/Filename $(date +%Y%m%d:%H%M)" >>rpts/${d2}_cfd
 for d1f in $d1/*
   { ((files+=1))
     lines=$(wc -l <$d1f)   # get line count
     ((tlines+=lines))      # count Total lines
     fdus=$(\du -s $d1f | tr '\t' ' '); fkb=${fdus%% *};
     if [[ -d $d1f ]]; then
        echo "$files  $lines  $fkb  ${d1f}/" >>rpts/${d2}_cfd
        else echo "$files  $lines  $fkb  $d1f" >>rpts/${d2}_cfd; fi
     ((fkbt+=fkb))
   }
 if [[ "$d1" == /* ]]; then fd1=$d1; else fd1=$PWD/$d1; fi
 echo "******  $tlines  $fkbt  *Totals* in Directory $fd1" >>rpts/${d2}_cfd
 echo "                         Report left in --> $PWD/rpts/${d2}_cfd" >>rpts/${d2}_cfd
 more rpts/${d2}_cfd
 sleep 1
 cfhelp  # display help screen summary of 8 scripts to count Files,Lines,KB in directories"
 #-------# export UVEXPERT=n to inhibit
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5D1. Counting Files,Lines,& KB in directories

cfdt - Totals-Only version of Count Files,Lines,KB in Directory

 #!/bin/ksh
 # cfdt - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # cfdt - Totals-Only version of Count Files,Lines,& KiloBytes in a Directory
 #      - can count Files in multiple directories if wildcards in quotes
 #      - by Owen Townsend, UV Software, April 12/2012
 d1="$1";
 if [[ ! -d "$d1" ]]; then
    echo "cfdt directory   <-- Usage ERROR, arg1 MUST be a DIRECTORY"
    echo "=============="
    echo " - enter 'y' to see sample demo command & report, null to quit"; read reply
    if [[ "$reply" != "y" ]]; then exit 1; fi
    echo " "
    echo "cfdt /dat1       <-- Correct Format & Sample Report"
    echo "=========="
    echo "cfdt - Count Files,Lines,& KiloBytes in a Directory - Totals-Only"
    echo " Files   Lines      KB  Directory/Filename 20190722:0915
    echo "     6    1250     112  *Totals-Only* in Directory /home/uvadm/demo/dat1
    exit 1; fi
 #
 typeset -R6 files lines tlines fkb fkbt
 test -d rpts || mkdir rpts
 d2=$(echo $d1 | tr '/' '_')
 echo "cfdt $* - Count Files,Lines,& KB in Directory - Totals-Only" >rpts/${d2}_cfdt
 echo " Files   Lines      KB  Directory/Filename $(date +%Y%m%d:%H%M)" >>rpts/${d2}_cfdt
 for d1f in $d1/*
   { ((files+=1))
     lines=$(wc -l <$d1f)   # get line count
     ((tlines+=lines))      # count Total lines
     fdus=$(\du -s $d1f | tr '\t' ' '); fkb=${fdus%% *};
     ##echo "$files  $lines  $fkb  ${d1f}/" >>rpts/${d2}_cfdt ##Totals Only
     ((fkbt+=fkb))
   }
 if [[ "$d1" == /* ]]; then fd1=$d1; else fd1=$PWD/$d1; fi
 echo "$files  $tlines  $fkbt  *Totals-Only* in Directory $fd1" >>rpts/${d2}_cfdt
 echo "                         Report left in --> $PWD/rpts/${d2}_cfdt" >>rpts/${d2}_cfdt
 more rpts/${d2}_cfdt
 cfhelp  # display help screen summary of 8 scripts to count Files,Lines,KB in directories"
 #-------# export UVEXPERT=n to inhibit
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5E1. Counting Files,Lines,& KB in directories

cfdpf - Count Files in Directory with Pattern [or not] in filenames

 #!/bin/ksh
 # cfdpf - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # cfdpf - Count Files in a Directory with a Pattern [or not] in filenames
 #      - can count Files in multiple directories if wildcards in quotes
 #      - by Owen Townsend, UV Software, updated July 2019
 d1="$1"; pat1="$2"; not="$3";
 if [[ -d "$d1" && -n "$pat1" ]]; then : ; else
    echo "cfdpf directory 'pattern' [not] <-- ERROR, arg1 DIRECTORY, arg2 PATTERN in single quotes"
    echo "===============================   - arg3 optional 'not' to select non-matches"
    echo " - enter 'y' to see sample demo command & report, null to quit"; read reply
    if [[ "$reply" != "y" ]]; then exit 1; fi
    echo " "
    echo "cfdpf dat1 'Canada*'            <-- Correct Format & Sample Report"
    echo "====================              - count files with 'Canada' anywhere in filename"
    echo "cfdpf - Count Files,Lines,KB in Directory with Pattern [not] in filenames"
    echo "File#    Lines      KB  Directory/Filename 20190722:0847"
    echo "     1     335      32  dat1/CanadaMPs"
    echo "     2      13       4  dat1/CanadaProvinces"
    echo "******     348      36  *Totals* in dat1 with pattern Canada*"
    exit 1; fi
 #
 typeset -R6 files=0 tlines=0 lines=0 fkb=0 fkbt=0
 test -d rpts || mkdir rpts
 pat2=$(echo $pat1 | tr '*' '_')
 d2=$(echo $d1 | tr '/' '_')
 echo "cfdpf $* - Count Files,Lines,KB in Directory with Pattern [not] in filenames" >rpts/${d2}_cfdpf_$pat2
 echo "File#    Lines      KB  Directory/Filename $(date +%Y%m%d:%H%M)" >>rpts/${d2}_cfdpf_$pat2
 for d1f in $d1/*
   { f1=$(basename $d1f)
     if [[ "$f1" == ${pat1} && "$not" == "not" ]]; then continue; fi
     if [[ "$f1" != ${pat1} && "$not" == ""    ]]; then continue; fi
        ((files+=1))
        lines=$(wc -l <$d1f)   # get line count
        ((tlines+=lines))      # count Total lines
        fdus=$(\du -s $d1f | tr '\t' ' '); fkb=${fdus%% *};
        echo "$files  $lines  $fkb  $d1f" >>rpts/${d2}_cfdpf_$pat2
        ((fkbt+=fkb))
   }
 if [[ "$d1" == /* ]]; then fd1=$d1; else fd1=$PWD/$d1; fi
 echo "******  $tlines  $fkbt  *Totals* in $d1 with pattern \"$pat1\"" >>rpts/${d2}_cfdpf_$pat2
 echo "                         Report left in --> $PWD/rpts/${d2}_cfdpf_$pat2" >>rpts/${d2}_cfdpf_$pat2
 more rpts/${d2}_cfdpf_$pat2
 cfhelp  # display help screen summary of 8 scripts to count Files,Lines,KB in directories"
 #-------# export UVEXPERT=n to inhibit
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5F1. Counting Files,Lines,& KB in directories

cfdpl - Count Files with a Pattern [or not] in any line any file

 #!/bin/ksh
 # cfdpl - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # cfdpl - Count Files in a Directory with a Pattern [or not] on any line in any file
 #       - by Owen Townsend, UV Software, updated July 2019
 d1="$1"; pat1="$2"; not="$3"; v=""; n="";
 if [[ -d "$d1" && -n "$pat1" ]]; then :
    else echo "cfdpl subdir pattern [not] <-- ERROR, arg1 MUST be a DIRECTORY, arg2 MUST be a PATTERN"
    echo "===============================    - arg3 optional 'not' to select non-matches"
    echo " - enter 'y' to see sample demo command & report, null to quit"; read reply
    if [[ "$reply" != "y" ]]; then exit 1; fi
    echo " "
    echo "cfdpl dat1 'Washington'         <-- Correct Format & Sample Report"
    echo "=======================           - count files in ftps/* with 'Washington' on any line"
    echo "cfdpl - Count Files,Lines,KB in Directory with Pattern [not] on any line in the file"
    echo "File#    Lines      KB  Directory/Filename 20190722:0901
    echo "     1     305      28  dat1/UScities
    echo "     2      50       4  dat1/USstates
    echo "******     355      32  *Totals* in /home/uvadm/demo/dat1 with  pattern "Washington"
    exit 1; fi
 #
 typeset -R6 files tlines lines fkb fkbt
 test -d rpts || mkdir rpts
 if [[ "$not" == "not" ]]; then v="-v"; n='!'; fi
 pat2=$(echo $pat1 | tr '*' '_')
 d2=$(echo $d1 | tr '/' '_')
 echo "cfdpl $* - Count Files,Lines,KB in Directory with Pattern [not] on any line in file" >rpts/${d2}_cfdpl_$pat2
 echo "File#    Lines      KB  Directory/Filename $(date +%Y%m%d:%H%M)" >>rpts/${d2}_cfdpl_$pat2
 for d1f in $d1/*
   { cnt=$(grep -c $pat1 $d1f 2>/dev/null)   # count matches in current file
     if ((${n}cnt)); then
        ((files+=1))
        lines=$(wc -l <$d1f)   # get line count
        ((tlines+=lines))      # count Total lines
        fdus=$(\du -s $d1f | tr '\t' ' '); fkb=${fdus%% *};
        echo "$files  $lines  $fkb  $d1f" >>rpts/${d2}_cfdpl_$pat2
        ((fkbt+=fkb))
     fi
   }
 if [[ "$d1" == /* ]]; then fd1=$d1; else fd1=$PWD/$d1; fi
 echo "******  $tlines  $fkbt  *Totals* in $fd1 with $not pattern \"$pat2\"" >>rpts/${d2}_cfdpl_$pat2
 echo "                         Report left in --> $PWD/rpts/${d2}_cfdpl_$pat2" >>rpts/${d2}_cfdpl_$pat2
 more rpts/${d2}_cfdpl_$pat2
 cfhelp  # display help screen summary of 8 scripts to count Files,Lines,KB in directories"
 #-------# export UVEXPERT=n to inhibit
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5G1. Counting Files,Lines,& KB in directories

cfdd - Count Files,Lines,& KB in ALL Sub-Dirs of a Super-Dir

 #!/bin/ksh
 # cfdd - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # cfdd - Count Files,Lines,& KB in ALL Sub-Dirs in a Super-Directory
 #      - assumes the parent directory contains only sub-directories
 #      - any files directly beneath the parent directory are ignored
 #
 pd1="$1"; subs=$(find "$pd1" -type d | wc -l)
 if [[ -d "$pd1" && $subs -gt 1 ]]; then : ; else
    echo "cfdd ParentDir     <-- ERROR, arg1 MUST be a Directory with SubDirs"
    echo "=============="
    echo " - enter 'y' to see sample demo command & report, null to quit"; read reply
    if [[ "$reply" != "y" ]]; then exit 1; fi
    echo " "
    echo "cfdd $UV/sf        <-- Correct Format & Sample Report"
    echo "==========="
    echo "cfdd - Count Files,Lines,& KB in all Sub-Dirs of a Super-Directory"
    echo "  Dir#   Files   Lines      KB  SubDir/ParentDirectory  20190720:2121"
    echo "     1     227    6502     944  sf/adm"
    echo "     2      86    2501     352  sf/demo"
    echo "     3     308   17102    1436  sf/IBM"
    echo "     4     539   21225    2240  sf/util"
    echo "******    1160   47330    4972  *Totals* for SubDirs of SuperDir /home/uvadm/sf/..."
    exit 99; fi
 #
 typeset -R6 dcnt=0 fcnt=0 tfcnt lcnt1=0 lcnt=0 tlcnt=0 fkb=0 fkbt=0
 test -d rpts || mkdir rpts
 pd2=$(echo $pd1 | tr '/' '_')
 echo "cfdd  $* - Count Files,Lines,& KB in SubDirs of SuperDir" >rpts/${pd2}_cfdd
 echo "  Dir#   Files   Lines      KB  SubDir/ParentDirectory  $(date +%Y%m%d:%H%M)" >>rpts/${pd2}_cfdd
 for d1 in $pd1/*
   { if [[ ! -d $d1 ]]; then continue; fi
     ((dcnt+=1))
     if [[ ! "$(ls -A $d1)" ]]; then continue; fi
     fcnt=0; lcnt=0;
     for d1f in $d1/*
       { ((fcnt+=1))
         lcnt1=$(wc -l <$d1f)
         ((lcnt+=lcnt1))
       }
     ((tfcnt+=fcnt))
     ((tlcnt+=lcnt))
     fdus=$(\du -s $d1 | tr '\t' ' '); fkb=${fdus%% *};
     ((fkbt+=fkb))
     echo "$dcnt  $fcnt  $lcnt  $fkb  $d1" >>rpts/${pd2}_cfdd
   }
 if [[ "$pd1" == /* ]]; then fpd1=$pd1; else fpd1=$PWD/$pd1; fi
 echo "******  $tfcnt  $tlcnt  $fkbt  *Totals* for SubDirs of SuperDir $fpd1/..." >>rpts/${pd2}_cfdd
 echo "                                Report left in --> $PWD/rpts/${pd2}_cfdd" >>rpts/${pd2}_cfdd
 more rpts/${pd2}_cfdd
 cfhelp  # display help screen summary of 8 scripts to count Files,Lines,KB in directories"
 #-------# export UVEXPERT=n to inhibit
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5H1. Counting Files,Lines,& KB in directories

cfddt - Count Files,Lines,KB ALL SubDirs in Super-Dir Totals-Only

 #!/bin/ksh
 # cfddt - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # cfddt - Count Files,Lines,& KB in ALL Sub-Dirs in a Super-Directory Totals-Only
 #       - assumes the parent directory contains only sub-directories
 #       - any files directly beneath the parent directory are ignored
 #
 echo "cfddt - Count Files,Lines,KB in all SubDirs of a Super-Dir - Totals-Only (of cfdd)"
 pd1="$1"; subs=$(find "$pd1" -type d | wc -l)
 if [[ -d "$pd1" && $subs -gt 1 ]]; then : ; else
    echo "cfddt ParentDirectory  <-- Usage ERROR, arg1 MUST be a DIRECTORY"
    echo "====================="
    echo " - enter 'y' to see sample demo command & report, null to quit"; read reply
    if [[ "$reply" != "y" ]]; then exit 1; fi
    echo " "
    echo "cfddt $UV/sf             <-- Correct Format & Sample Report"
    echo "============"
    echo "cfddt - Count Files,Lines,KB in all SubDirs of a Super-Dir - Totals-Only (of cfdd)"
    echo "  Dirs    Files   Lines      KB  SubDir/ParentDir 20190722:1230"
    echo "     4     1159   47351    4968  *Totals* for SubDirs of SuperDir /home/uvadm/sf/..."
    exit 99; fi
 #
 typeset -R6 dcnt=0 fcnt=0 tfcnt lcnt1=0 lcnt=0 tlcnt=0 fkb=0 fkbt=0
 test -d rpts || mkdir rpts
 pd2=$(echo $pd1 | tr '/' '_')
 echo "  Dirs    Files   Lines      KB  SubDir/ParentDir $(date +%Y%m%d:%H%M)" >rpts/${pd2}_cfddt
 for d1 in $pd1/*
   { if [[ ! -d $d1 ]]; then continue; fi
     ((dcnt+=1))
     if [[ ! "$(ls -A $d1)" ]]; then continue; fi
     fcnt=0; lcnt=0;
     for d1f in $d1/*
       { ((fcnt+=1))
         lcnt1=$(wc -l <$d1f)
         ((lcnt+=lcnt1))
       }
     ((tfcnt+=fcnt))
     ((tlcnt+=lcnt))
     fdus=$(\du -s $d1 | tr '\t' ' '); fkb=${fdus%% *};
     ((fkbt+=fkb))
     ##echo "$dcnt  $fcnt  $lcnt  $fkb  $d1" >>rpts/${pd2}_cfddt ##for Totals-Only
   }
 if [[ "$pd1" == /* ]]; then fpd1=$pd1; else fpd1=$PWD/$pd1; fi
 echo "$dcnt   $tfcnt  $tlcnt  $fkbt  *Totals* for SubDirs of SuperDir $fpd1/..." >>rpts/${pd2}_cfddt
 echo "      - report left in --> $PWD/rpts/${pd2}_cfddt"
 more rpts/${pd2}_cfddt
 cfhelp  # display help screen summary of 8 scripts to count Files,Lines,KB in directories"
 #-------# export UVEXPERT=n to inhibit
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5I1. Counting Files,Lines,& KB in directories

cfddf - Count Files,Lines,KB ALL SubDirs in SuperDir + 1st few files

 #!/bin/ksh
 # cfddf - Count Files,Lines,KB in ALL Sub-Dirs in a Super-Dir + 1st few files
 pd1="$1"; max="$2"; subs=$(find "$pd1" -type d | wc -l)
 if [[ -d "$pd1" && -n "$max" && $subs -gt 1 ]]; then : ; else
    echo "cfddf ParentDir FilesMax <-- ERROR, arg1 MUST be a Directory with SubDirs"
    echo "========================   - arg2 MUST be number of filenames to list"
    echo " - enter 'y' to see sample demo command & report, null to quit"; read reply
    if [[ "$reply" != "y" ]]; then exit 1; fi
    echo "cfddf backup 1  <-- cfddf sample demo command & report"
    echo "==============    - count Files,Lines,KB in backup/subdirs & show 1st line of each file"
    echo "cfddf - Count Files,Lines,KB in SubDirs of a Super-Dir + 1st few files"
    echo "  Dir#   Files   Lines      KB  SubDir/ParentDirectory  20190722:0829"
    echo "     1      28    1437     356  backup/dat1"
    echo "             1    1437       4  backup/dat1/accents1"
    echo "     2       6    1250     116  backup/dat1"
    echo "             1    1250      32  backup/dat1/CanadaMPs"
    echo "******      34    2687     472  *Totals* for SubDirs of SuperDir /home/uvadm/demo/backup/..."
    exit 99; fi
 typeset -R6 dcnt=0 fcnt=0 fcnt1=0 tfcnt lcnt1=0 lcnt=0 tlcnt=0 fkb=0 fkbt=0
 test -d rpts || mkdir rpts
 pd2=$(echo $pd1 | tr '/' '_')
 echo "cfddf $* - Count Files,Lines,KB in SubDirs of SuperDir + 1st few files" >rpts/${pd2}_cfddf
 echo "  Dir#   Files   Lines      KB  SubDir/ParentDirectory  $(date +%Y%m%d:%H%M)" >>rpts/${pd2}_cfddf
 for d1 in $pd1/*
   { if [[ ! -d $d1 ]]; then continue; fi
     ((dcnt+=1))
     if [[ ! "$(ls -A $d1)" ]]; then continue; fi
     fcnt=0; lcnt=0;
     for d1f in $d1/*
       { ((fcnt+=1))
         lcnt1=$(wc -l <$d1f)
         ((lcnt+=lcnt1))
       }
     ((tfcnt+=fcnt))
     ((tlcnt+=lcnt))
     fdus=$(\du -s $d1 | tr '\t' ' '); fkb=${fdus%% *};
     ((fkbt+=fkb))
     echo "$dcnt  $fcnt  $lcnt  $fkb  $d1" >>rpts/${pd2}_cfddf
     fcnt1=0
     for d1f in $d1/*
       { ((fcnt1+=1))
         lcnt1=$(wc -l <$d1f)
         fdus=$(\du -s $d1f | tr '\t' ' '); fkb=${fdus%% *};
         echo "        $fcnt1  $lcnt  $fkb  $d1f" >>rpts/${pd2}_cfddf
         if ((fcnt1==max)); then break; fi
       }
   }
 if [[ "$pd1" == /* ]]; then fpd1=$pd1; else fpd1=$PWD/$pd1; fi
 echo "******  $tfcnt  $tlcnt  $fkbt  *Totals* for SubDirs of SuperDir $fpd1/..." >>rpts/${pd2}_cfddf
 more rpts/${pd2}_cfddf
 cfhelp  # display help screen summary of 8 scripts to count Files,Lines,KB in directories"
 #-------# export UVEXPERT=n to inhibit
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5J1. Counting Files,Lines,& KB in directories

cfdmm stats: file#,Lines,KB,Minsize/rec#,Maxsize/rec#

 #!/bin/ksh
 # cfdmm - Korn shell script from UVSI stored in: /home/uvadm/sf/util/...
 # cfdmm - list directory stats: file#,Lines,Minsize,Minrec#,Maxsize,Maxrec#, dir/filename
 d1="$1";
 if [[ ! -d "$d1" ]]; then
    echo "usage: cfdmm directory     <-- ERROR - arg1 must be a directory"
    echo "       ==============="
    echo " - enter 'y' to see sample demo command & report, null to quit"; read reply
    if [[ "$reply" != "y" ]]; then exit 1; fi
    echo " "
    echo "cfdmm demo/dat1  <-- cfdmm sample demo command & report"
    echo "===============    - show Record Minsize,Maxsize,Minrec#,Maxrec# for all files in Directory"
    echo " File#  Lines MinRsiz MaxRsiz  MinRnum MaxRnum  Directory/FileName"
    echo "     1    335      94      94        1     335  dat1/CanadaMPs"
    echo "     2     13      13      33       13       5  dat1/CanadaProvinces"
    echo "     3      8      69      74        6       7  dat1/nameadrs1"
    echo "     4    305      74      81        2       1  dat1/UScities"
    echo "     5    539      70      73        2     422  dat1/UScongress"
    echo "     6     50      12      22       15      40  dat1/USstates"
    exit 1; fi
 d2=$(echo $d1 | tr '/' '_')
 test -d rpts || mkdir rpts
 echo "cfdmm $* - show Record Minsize,Maxsize,Minrec#,Maxrec# for all files in Directory" >rpts/${d2}_cfdmm
 echo " File#   Lines      KB MinRsiz MaxRsiz MinRnum MaxRnum  Directory/FileName" >>rpts/${d2}_cfdmm
 typeset -R6 files=0 tfiles=0;
 typeset -R7 tlines=0 fkb=0 tfkb=0 tMinRsz=9999999 tMaxRsz=0;
 for d1f in $d1/*
   { if [[ ! -f $d1f ]]; then continue; fi
     typeset -R7 lines=0 l1cn=0 MinRsiz=9999999 MinRnum=0 MaxRsiz=0 MaxRnum=0;
     fdus=$(\du -s $d1f | tr '\t' ' '); fkb=${fdus%% *};
     ((files+=1))                 # count files
     exec 3< $d1f                 # open file for reading
     while read -u3 line          # read next filename
       do ((lines+=1))            # increment line# in current file
          l1c=${#line}            # get line length
          l1cn=${l1c%% *}         # drop any trailing blanks
          ((l1cn+=1))             # +1 for LineFeed
          if ((l1cn <  MinRsiz)); then MinRsiz=$l1cn; MinRnum=$lines; fi
          if ((l1cn >= MaxRsiz)); then MaxRsiz=$l1cn; MaxRnum=$lines; fi
       done
     exec 3<&-               # close file of filenames
        echo "$files $lines $fkb $MinRsiz $MaxRsiz $MinRnum $MaxRnum  $d1f" >>rpts/${d2}_cfdmm
        ((tlines+=lines))
        ((tfkb+=fkb))
        if ((MinRsiz <  tMinRsz)); then tMinRsz=$MinRsiz; fi
        if ((MaxRsiz >= tMaxRsz)); then tMaxRsz=$MaxRsiz; fi
   }
 echo "$files $tlines $tfkb $tMinRsz $tMaxRsz                  *Totals* in $d1" >>rpts/${d2}_cfdmm
 echo "                               Report also in --> $PWD/rpts/${d2}_cfdmm" >>rpts/${d2}_cfdmm
 more "rpts/${d2}_cfdmm"
 cfhelp  # display help screen summary of 8 scripts to count Files,Lines,KB in directories"
 #-------# export UVEXPERT=n to inhibit
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5K1. Counting Files,Lines,& KB in directories

cfhelp - reminders of scriptnames to count Files,Lines,KB

 #!/bin/ksh
 # cfhelp - display reminders of scriptnames to count Files,Lines,KB in directories
 #        - by Owen Townsend, UV Software, July 2019
 #        - this script called at end of the 8 cfs scripts
 #          (cfl, cfd, cfdt, cfdpl, cfdd, cfddt, cfdpf, cfddf)
 #
 # test env-var to inhibit display cfhelps
 if [[ "$UVEXPERT" == "y" ]]; then
     echo " "
     exit 1; fi
 #
 sleep 1
 echo "--> reply 'y' to display all 9 cf* commands & suggested demo/dirs, or null to exit"; read reply
 if [[ "$reply" != "y" ]]; then
     echo " "
     exit 0; fi
 echo "          ------- cfd... scripts & demo/dirs to count Files,Lines,KB -------"
 echo "1. cfl   \$UV/src/uvcopy.c  # Count Lines & KB in the uvcopy utility program"
 echo "2. cfd   dat1              # Count Files,lines,KB in 1 directory (dat1)"
 echo "3. cfdt  dat1              # Totals-Only of cfd dat1 (6 files counts combined onto 1 line)"
 echo "4. cfdpf dat1 'Canada*'    # Counts for dat1/... filenames begining with 'Canada'"
 echo "5. cfdpl dat1 'Washington' # Counts for dat1/... files with 'Washington' on any line of the file"
 echo "6. cfdd  \$UV/sf            # count Files,Lines,& KB for the 4 subdirs (adm,demo,util,IBM) of $UV/sf"
 echo "7. cfddt \$UV/sf            # Totals-Only of cfdd $UV/sf (4 subdir counts combined onto 1 line)"
 echo "8. cfddf backup 2          # Counts for backup/... (archive,dat1,dat2) + 1st 2 files of each subdir"
 echo "9. cfdmm                   # Directory-Stats: files,lines,Minsize,Maxsize,Minrec#,Maxrec#"
 echo "          ----- add 'export UVEXPERT=y' to your profile to inhibit these displays -----"
 echo " "
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_6 - Vancouver Utility scripts for Conversion & General use - Contents

scripts to manipulate text&data files - contents

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6A1. scripts to manipulate text files

removeCR - remove Carriage Returns from text files

 #!/bin/ksh
 # removeCR - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # removeCR - remove CR's Carriage Returns (x'0D')
 #          - by copy to tmp/... & prompt to copy back
 #          - by Owen Townsend, UV Software, Feb 2014
 #
 echo "removeCR  - copy a file to tmp/... removing CRs & nulls, prompt to copy back"
 echo "removeCR1 - alternate saves file in tmp/ before copy back removing CRs"
 df="$1";
 if [[ -f "$df" && -d tmp ]]; then :
 else echo "usage: removeCR subdir/file"
      echo "       ===================="
      echo " - arg1 must be a text file (with LFs & CRs)"
      echo " - current working directory must also contain a tmp/ subdir"
      echo " - will copy to tmp/ removing CRs & prompt to copy back"
      echo " - also see insertCR to insert CR's (create CR/LF)"
      exit 1; fi
 #
 f=$(basename $df)
 tr -d '\r' <$df >tmp/$f
 #==========================
 linesin=$(wc -l $df); sizein=$(du -bs $df);
 linesout=$(wc -l tmp/$f); sizeout=$(du -b tmp/$f);
 echo "input  lines & bytes:    $linesin $sizein"
 echo "output lines & bytes:    $linesout $sizeout"
 until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "copy back overwriting original y/n ?"; read reply; done
 if [[ "$reply" = "y" ]]; then
    cp tmp/$f $df; echo "copied back to $df"; ls -l $df
 else
    echo "copy back aborted"
 fi
 exit 0

See the test/demo of removeCR on page '6A3' & insertCR on page '6A4'

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6A2. scripts to manipulate text files

insertCR - insert Carriage Returns in text files

 #!/bin/ksh
 # insertCR - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # insertCR - insert CR's Carriage Returns x'0D' in a text file
 #          - by copy to tmp/... inserting CRs & prompt to copy back
 echo "insertCR - copy a file to tmp/ inserting CRs & prompt to copy back"
 echo "removeCR - alternate script to remove Carriage Returns"
 df="$1";
 if [[ -f "$df" && -d tmp ]]; then :
 else echo "usage: insertCR subdir/file"
      echo "       ===================="
      echo " - arg1 must be a text file (with LFs)"
      echo " - current working directory must also contain a tmp/ subdir"
      echo " - will copy to tmp/... inserting CRs & prompt to copy back"
      echo " - also see removeCR to remove CRs"
      exit 1; fi
 f=$(basename $df)
 sed 's/$/\r/' <$df >tmp/$f
 #=========================
 linesin=$(wc -l $df); sizein=$(du -bs $df);
 linesout=$(wc -l tmp/$f); sizeout=$(du -b tmp/$f);
 echo "input  lines & bytes:    $linesin $sizein"
 echo "output lines & bytes:    $linesout $sizeout"
 until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "copy back overwriting original y/n ?"; read reply; done
 if [[ "$reply" = "y" ]]; then
    cp tmp/$f $df; echo "copied back to $df"; ls -l $df
 else echo "copy back aborted"; fi
 exit 0

See the test/demo of removeCR on page '6A3' & insertCR on page '6A4'

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6A3. scripts to manipulate text files

removeCR - remove CR's from Windows text files

Text files from Windows often contain Carriage-Returns x'0D' as well as Line -Feeds x'0A'. Unix text files should be terminated only with a Line-Feed x'0A'. CR's can cause unix scripts to fail (especially if on the 1st line !/bin/ksh..) and you may be mystified because you cannot see them in the unix editors (use uvhd to see them). Here are scripts to remove CR's or insert CR's which might be required if you need to send unix files back to Windows.

Here is a demo using removeCR, 1st we will use 'uvhd' to confirm the presence of CR's (since you cannot see them with unix editors).

 #1. Login yourself --> /home/$HOME

 #2. mkdir tmp tmp1 tmp2  <-- make subdirs for test
     ===================

 #3. cp dat1/prodmas1 tmp1  <-- copy demo file to tmp1
     =====================

 #4. uvhd tmp1/prodmas1 t   <-- display demo file with CR & LF terminators
     ====================     - option 't' for Text files (ended by LFs)
                              - shows 1st record as follows:

                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 BBQ010 BAR-B-Q         0019500..
         44533324452425222222222333333300
         2210100212D2D10000000000019500DA
 rec#=1 rsize=32 fptr=0 fsize=256 rcount=8

 #5. removeCR tmp1/prodmas1   <-- run script to remove CR's
     ====================
     --> copy back overwriting original y/n ?
     --> y <-- reply y to copy back

 #6. uvhd tmp1/prodmas1 t   <-- confirm CR's removed
     ====================

                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 BBQ010 BAR-B-Q         0019500.
         4453332445242522222222233333330
         2210100212D2D10000000000019500A
 rec#=1 rsize=31 fptr=0 fsize=248 rcount=8

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6A4. scripts to manipulate text files

insertCR - insert CR's in unix text files

Now we can demo insertCR, using tmp/prodmas1 (output of from removeCR above).

 #7. insertCR tmp1/prodmas1   <-- run script to insert CR's
     ======================
     --> copy back overwriting original y/n ?
     --> y <-- reply y to copy back

 #8. uvhd tmp1/prodmas1 t   <-- confirm CR's removed
     ====================

                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 BBQ010 BAR-B-Q         0019500..
         44533324452425222222222333333300
         2210100212D2D10000000000019500DA
 rec#=1 rsize=32 fptr=0 fsize=256 rcount=8

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6A5. scripts to manipulate text files

removeCRall - remove CRs from all files in directory

 #!/bin/ksh
 # removeCRall - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # removeCRall - remove CR's Carriage Returns x'0D'
 #               while copying from 1 directory to a 2nd directory
 #             - by Owen Townsend, UV Software, Feb 2014
 #
 echo "removeCRall - copy all files from 1 dir to 2nd dir, removing CRs"
 echo "insertCRall - alternate script to insert Carriage Returns"
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
 else echo "usage: removeCRall indir outdir"
      echo "       ========================"
      echo " - arg1 & arg2 must be directories"
      echo " - also see insertCRall to insert CR's (create CR/LF)"
      exit 1; fi
 #
 integer fn=0
 for df in $d1/*
   { f=$(basename $df)
     ((fn+=1))
     echo "file# $fn - $d1/$f --> $d2 inserting CR's"
     tr -d '\r' <$d1/$f >$d2/$f
     #=========================
   }
 echo "$fn files copied from $d1 to $d2, with CR's removed"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6A6. scripts to manipulate text files

insertCRall - insert CRs into all files in directory

 #!/bin/ksh
 # insertCRall - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # insertCRall - insert CR's Carriage Returns x'0D'
 #               while copying from 1 directory to a 2nd directory
 #             - by Owen Townsend, UV Software, Feb 2014
 #
 echo "insertCRall - copy all files from indir to outdir, inserting CRs"
 echo "removeCRall - alternate script to remove Carriage Returns"
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
 else echo "usage: insertCRall indir outdir"
      echo "       ========================"
      echo " - arg1 & arg2 must be directories"
      echo " - also see removeCRall to remove CR's "
      exit 1; fi
 #
 integer fn=0
 for df in $d1/*
   { f=$(basename $df)
     ((fn+=1))
     echo "file# $fn - $d1/$f --> $d2 removing CR's"
     sed 's/$/\r/' <$d1/$f >$d2/$f
     #============================
   }
 echo "$fn files copied from $d1 to $d2, with CR's inserted"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6B1. scripts to manipulate text files

runsed1 - apply sed script to all files in directory

 #!/bin/ksh
 # runsed1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # runsed1 - run sed script to process all files from 1 dir to a 2nd dir
 #         - will prompt for 1 or more sed script lines
 #
 #usage: runsed1 dir1 dir2
 #       =================
 #
 #example: change all 'lp' to 'llp' in all scripts in the 'sf' directory
 # 1. mv sf sf.old            - change name of existing dir (save)
 # 2. mkdir sf                - remake new dir
 # 3. runsed1 sf.old sf       - run script (will copy files to new sf)
 #    =================
 #    --> s/ lp / llp /g      - enter sed command at prompt
 #        ==============
 #    --> ^D                  - control-D to end command entry
 # 4. diff sf.old/xxx sf/xxx  - spot check results (or alldiff2 all files)
 #
 # runsed2 - follow-on script to use sed command saved by runsed1 in tmp/runsed1
 #         - to rerun same script on other dirs
 #           or could modify script & rerun on same dirs
 #
 # 1. vi tmp/runsed1                 - edit script saved as tmp/runsed1
 # 2. runsed2 dir1 dir2 tmp/runsed1  - 'runsed2' to execute saved script
 #
 # Also see uvcopy jobs rep1d & rep2d in REPjobs.htm
 # - sed is good when you have UNIQUE patterns & dont need qualifiers
 # - use rep1d or rep2d when you need to specify other patterns that
 #   must be present or not present on the same line
 # - rep1d & rep2d also create an audit report showing modified lines
 #
 d1=$1; d2=$2; sedscript=tmp/runsed1
 if [[ -d $d1 && -d $d2 ]]; then :
    else echo "usage: runsed1 dir1 dir2"
         echo "       ================="
         echo "- arg1 & arg2 must be input & output directories"
    exit 9; fi
 if [[ ! -d tmp ]]; then mkdir tmp; fi
 # prompt for sed script entry, using cat ended by control-d
 echo "enter sed script commands, end via control-D"
 echo "--> s/ lp / llp /g <-- sample sed to change ' lp ' to ' llp '"
 echo "--> ^D             <-- enter control-D to end sed lines entry"
 cat >$sedscript         # enter lines, end via ctl-D
 for i in $d1/*
 do f=$(basename $i)
    let x=x+1
    echo "file# $x - $d1/$f"
    sed -f $sedscript $d1/$f >$d2/$f
 done
 echo "total $x files processed from $d1 to $d2 by $sedscript"
 echo "sed script saved in $sedscript, rerun via runsed2 (vs runsed1)"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6B2. scripts to manipulate text files

runsed2 - run sed script to process all files from 1 dir to 2nd dir

 #!/bin/ksh
 # runsed2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # runsed2 - run sed script to process all files from 1 dir to a 2nd dir
 #         - 'driver' script to run previously written 'sed' scripts,
 #           on all files in a directory, while copying to a 2nd directory
 #
 #usage: runsed2 dir1 dir2 sedscript
 #       ===========================
 #
 # Also note alternative 'runsed1' (vs this 'runsed2')
 # - runsed1 will prompt you to enter the sed script command lines
 # - runsed1 saves the script as: tmp/runsed1
 # - you may then use this runsed2 to re-execute the sed script
 #   (& you may use vi to modify the script before reruns)
 #
 #example:
 #
 # 1. mv sf sf.old                  - change name of existing dir (save)
 # 2. mkdir sf                      - remake new dir
 # 3. runsed2 sf.old sf tmp/runsed1 - run script (copies files to new sf)
 #    =============================
 # 4a. diff sf.old/xxx sf/xxx       - spot check results ?
 # 4b. alldiff2 sf.old sf           - or check all files with 'alldiff2'
 #
 d1=$1; d2=$2; sedfile=$3;
 if [[ -d $d1 && -d $d2 && -f $sedfile ]]; then :
    else echo "usage: runsed2 dir1 dir2 sedscript"
         echo "       ==========================="
         echo "- arg1 & arg2 must be directories"
         echo "- arg3 must be a file"
    exit 9; fi
 #
 for i in $d1/*
 do
    f=$(basename $i)
    let x=x+1
    echo "file# $x - $d1/$f"
    sed -f $sedfile $d1/$f >$d2/$f
 done
 echo "total $x files processed from $d1 to $d2 by $sedfile"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6C0. CNVaids: Vancouver Utility Aids

uvfix scripts & uvcopy jobs

If you can't perform your desired data file manipulation using unix 'sed', you can do anything you can imagine with the 'uvcopy' utlity.

'uvcopy' can do many things not possible with the standard unix/linux utilities such as 'sed' - processing 'packed decimal fields', handling 'Indexed files', & 'field addressing by column#' (common in mainframe utilities, but not in unix utilities).

The interpretive nature of uvcopy makes it easy to learn & to use. If you have a mainframe backgound, you may already be familiar with the uvcopy instruction formats since they are patterned after the IBM 360 assembler, but interpretive - no compile required, just edit & execute with the uvcopy interpreter.

uvcopy has over 90 instructions and about half of these are pwerful subfunctions. For example 'fix' will convert delimited files with any specified delimiter (tab, comma, pipe, etc) to fixed formats. 'fix' is very useful to convert the tab or csv delimited files from Excel spread- sheets to fixed format files required by COBOL applications. The 'var' instruction (opposite of 'fix') will convert fixed-formats to delimited formats for importing into spread-sheets or database tables.

uvfix scripts - easy way to use uvcopy

The 'uvfix' scripts make it easy to employ the powerful uvcopy instructions without having to program the uvcopy framework (file I/O, etc). You only need to enter the uvcopy instructions desired when prompted. See http://www.uvsoftware.ca/uvcopy3.htm for the 90 instructions available.

There are 4 'uvfix_' scripts (in /home/uvadm/sf/util/...) that call 4 'uvfix_' uvcopy jobs (from /home/uvadm/pf/util/...). The scripts make the command line as simple as possible - specify only the input file-name & record-size if fixed. The output files are written to the tmp/... subdir (in your current dir) where you can inspect & then copy back overwriting the input file if desired.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6D1. uvfix scripts & uvcopy jobs

demo uvfix1,A,2,B

Here are some demos used to illustrate the 4 uvfix_ scripts available. You can run these demos from /home/uvadm/ using the demo files provided in /home/uvadm/dat1/...

 #0. Login uvadm --> /home/uvadm
 #0a. Or Login yourself if you are in same group as uvadm
 #0b. cd /home/uvadm

uvfix1 ex#1 - modify 1 text file with 'mvc'

 #1. more dat1/names1      <-- investigate Input file
     ================

      Owen Townsend
      UV Software
      4667 Hoskins Rd
      North Vancouver BC Canada
      V7K2R3

 #2. uvfix1 dat1/names1  <-- run uvfix1 & enter instructions
     ==================      at the prompts as follows:
     uop=q1n99r1024 - user option defaults
           n99      - output record count limit (high# copies to EOF)
              r1024 - recsize ONLY for Fixed-Length OUTPUT records if desired
                    - for Fixed-Length, reply RSF/RST at prompt (default text typ=LSTt)
                    - for Text files: accept default recsize & default typ=LSTt
     User OPtion (uop) defaults  = q1r1024n999999

 #2a. -->null accept or override -->  <-- null entry to accept defaults
 #2b. -->LST = default file type -->  <-- null accept typ=LST

 #3. Enter Instructions or '.' to process
     ===================================
 #3a. mvc b27(25),a0    <-- move(copy) cols 1-25 to 28-53"
 #3b. mvc b56(25),a0    <-- move(copy) cols 1-25 to 56-71"
 #3c. .                 <-- enter '.' to end instrns & execute job"

 #4. more tmp/names1    <-- inspect output file
     ===============      - 1st 5 records as follows:

     Owen Townsend              Owen Townsend                Owen Townsend
     UV Software                UV Software                  UV Software
     4667 Hoskins Rd            4667 Hoskins Rd              4667 Hoskins Rd
     North Vancouver BC Canada  North Vancouver BC Canada    North Vancouver BC Canada
     V7K2R3                     V7K2R3                       V7K2R3

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6D2. uvfix scripts & uvcopy jobs

uvfix1 ex#2 - convert Tab-delimited file to Fixed-format

 uvhd /home/owen/dat1/payemails t
 ================================
 rec#=1 rsize=37 fptr=0 fsize=121 rcount=4
                   10        20        30        40        50        60
 r#     1 0123456789012345678901234567890123456789012345678901234567890123
        0 PayrollService.Username.EmailAddress.
          5677666567766605767666604666646676770
          0192FCC352693595352E1D595D19C1442533A
 rec#=2 rsize=32 fptr=37 fsize=121 rcount=4
                   10        20        30        40        50        60
 r#     2 0123456789012345678901234567890123456789012345678901234567890123
       37 acupay.Sheila.Sheila@acupay.com.   <-- '.'s are Tabs or LineFeeds
          66776705666660566666466776726660   <-- Tabs x'09' (vertical hex)
          13501993859C193859C10135019E3FDA   <-- LF's x'0A' (vertical hex)
 rec#=3 rsize=28 fptr=69 fsize=121 rcount=4
                      10        20        30        40        50        60
 r#     3 0123456789012345678901234567890123456789012345678901234567890123
       69 adcomp.Jack.Jack@adcomp.com.
          6666670466604666466666726660
          143FD09A13B9A13B0143FD0E3FDA
 rec#=4 rsize=24 fptr=97 fsize=121 rcount=4
                      10        20        30        40        50        60
 r#     4 0123456789012345678901234567890123456789012345678901234567890123
       97 adp.Linda.Linda@adp.com.
          667046666046666466726660
          1409C9E419C9E410140E3FDA

Fixed-Format created by uvfix1 from tab-delimited

 payrollservice      username            emailaddress
 acupay              sheila              sheila@acupay.com
 adcomp              jack                jack@adcomp.com
 adp                 linda               linda@adp.com

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

uvfix1 ex#2 (continued) Tab-delimited to Fixed-format

 #0.  Login uvadm --> /home/uvadm
 #0a. Or Login yourself if you are in same group as uvadm
 #0b. cd /home/uvadm

 #1. uvhd dat1/payemails t    <-- use uvhd to investigate file
     =====================      - option 't' (Text file, with tab delimiters)

 #2. uvfix1 dat1/payemails    <-- run uvfix, specify input file only
     =====================      - output file will be tmp/payemails
 #2a. -->      <-- prompt for options, reply null
 #2b. --> LSTt <-- prompt for output file type, reply null accept LSTt default
      --> reply to instruction prompts as follows:

 #3.      reply to instruction prompts as follows:
          ========================================
 #3a. --> fix c100(100),a0(80),3,x'09' <-- convert to fixed fields 100 apart
                                - copies from area 'a' to c100,c200,c300
 #3b. --> mvc b00(20),c100    <-- store Employer-Code in cols 1-20
 #3c. --> mvc b20(20),c200    <-- store User-Name in cols 21-40
 #3d. --> mvc b40(60),c300    <-- store Email-SAddress in cols 41-100
 #3e. --> trl b0(100)         <-- ensure all lower-case
 #3f. --> .                   <-- '.' ends input instrns, executes,
          ===========              & writes output to tmp/payemails

 #5. vi tmp/payemails         <-- inspect output
     ================

 #6. cp tmp/payemails dat1/   <-- copyback overwriting original
     ======================     - or change name as desired

Notes

1. See all uvcopy instructions documented at www.uvsoftware.ca/uvcopy3.htm
   
   
2. uvfix1 prompts for instructions until you ennter '.' to end input.
   
   
3. uvfix1 executes as follows:
   

3a. Reads each record into area 'a' & copies to area 'b' for possible output (In example above, we fix to area 'c' & then move to area 'b' for output).

3b. Executes stored instructions on each record.

3c. Writes record from area 'b', would write to output file unchanged if you entered no instructions (just entered '.' to end instruction input).

4. If you wanted to inhibit output, you would enter 'ret<' instruction, which returns with condition code <, which inhibits the ussual write. You would precede with some compare for something & skip 1 (over the ret<) if you want to output the record (ret= is automatic at end your instructions).
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6E1. uvfix scripts & uvcopy jobs

uvfixA - modify ALL Text files in subdir

 #0. Login uvadm --> /home/uvadm
 #0a. Or Login yourself if you are in same group as uvadm
 #0b. cd /home/uvadm

 #1. mkdir tmp2          <-- make outpur subdir
 #1a. rm -f tmp2/*         - or clear all files if tmp2 already exists

 #2. uvfixA mf/cbls tmp2  <-- run uvfixA specifying I/O subdirs
     ===================    - reply to prompts as follows:

      uop=q1r512n999999 - user option defaults
            r512        - output reccsize for fixed length records typ=RSF/RST
                        - for text files: accept default recsize & default typ=LSTt
                n999999 - output record count limit (high# copies to EOF)
      User OPtion (uop) defaults  = q1r512n999999
 #2a. -->null to accept or enter/override -->   <-- null reply, no options required
 #2b. -->did you create outdir (dflt tmp) --> y <-- reply 'y' to continue

 #3.      reply to instruction prompts as follows:
          ========================================
 #3a. --> clr  b0(6),' '   <-- enter instrn to clear cols 1-6
 #3b. --> add  $ca1,1      <-- increment line counter
 #3c. --> mvn  b0(4),$ca1  <-- insert seuw=ence# in cols 1-4
 #3d. --> .                <-- enter '.' to end keyins & execute

sample output - 1st 8 lines of 1st program

 0001   identification division.
 0002  * car200 - test mainframe conversion to micro focus cobol
 0003  *        - vancouver utilities from www.uvsoftware.ca
 0004  *        - list sales details with customer name from custmas
 0005   program-id. car200.
 0006   environment division.
 0007   input-output section.
 0008   file-control.

^^^^ <-- sequence# gnerated by #3b & #3c above

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6F1. uvfix scripts & uvcopy jobs

uvfix2 - modify 1 Fixed-length file

 #1. uvhd dat1/custmas1    <-- investigate Input file
     ================== note - thisyr sales 12 * 5 bytes packed start at 120
                             - lastyr sales 12 * 5 bytes packed start at 180

 rec#=1 rsize=256 fptr=0 fsize=8192 rcount=32
                   10        20        30        40        50        60
 r#     1 0123456789012345678901234567890123456789012345678901234567890123
        0 130140    EVERGREEN MOTORS LTD.    1815 BOWEN ROAD          NANA
          3333332222454545444244545524542222233332445442544422222222224444
          130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
       64 IMO          BC V9S1H1    250-754-5531 LARRY WRENCH     ..4V|...
          4442222222222442535343222233323332333324455525544442222201357000
          9DF00000000002306931810000250D754D55310C12290725E38000000246C000
      128 .........W0....`........)X}..f3.....\.................4V}.......
          0000000005300016000000002570063100095000000000000000013570000000
          0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0246D0000C00
      192 .E|...V}.......................f.....<........f.C 19950531
          0470005700000000880000000018000680001300000000694233333333222222
          35C0046D0000C0023C0000C0083C0056D0012C0000C0016D3019950531000000

 #2. uvfix2 dat1/custmas1 r256  <-- run script, specify input file,& option
     =========================      'r256' for input record-size
                                 - prompts for commands, reply as follows
      uop=q1n99r256s0    - user option defaults
            n99          - output record limit (n99 means all records)
               r256      - Input Record size & output recsize default (if s0)
               r8192     - max record-size 8192
                    s0   - Output Recsize defaulted from Input recsize if s0
                    s80  - may specify value to override the default from insize
      User OPtion (uop) defaults  = q1n99r256s0r256
 #2a. -->null accept or override -->  <-- null reply (options on command line)
 #2b. -->RSF = default file type -->  <-- null accept typ=RSF

 #3.      reply to instruction prompts as follows:
          ========================================
 #3a. --> mvc    b180(60),a120      <-- move thisyear sales to lastyear sales
 #3b. --> mvnx12 b120(5p),0         <-- clear 12 thisyear sales to 5 bytes packed 0
 #3c. --> .                         <-- enter '.' end instructions & execute

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

uvfix2 - move thisyear to last & clear thisyear

 #4. Check output file with uvhd

 #4a. uvhd tmp/custmas1  <-- investigate Output file
      =================    - thisyr sales 12*5 bytes packed 120-179 now all 0
                           - lastyr sales 12*5 bytes packed 180-239
                             now moved from 120-179 (same as 120-179 above Input)

 rec#=1 rsize=256 fptr=0 fsize=8192 rcount=32
                   10        20        30        40        50        60
 r#     1 0123456789012345678901234567890123456789012345678901234567890123
        0 130140    EVERGREEN MOTORS LTD.    1815 BOWEN ROAD          NANA
          3333332222454545444244545524542222233332445442544422222222224444
          130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
       64 IMO          BC V9S1H1    250-754-5531 LARRY WRENCH     ........
          4442222222222442535343222233323332333324455525544442222200000000
          9DF00000000002306931810000250D754D55310C12290725E38000000000C000
      128 ......................................................4V|.......
          0000000000000000000000000000000000000000000000000000013570000000
          0C0000C0000C0000C0000C0000C0000C0000C0000C0000C0000C0246C0000C00
      192 .....W0....`........)X}..f3.....\...............C 19950531
          0000053000160000000025700631000950000000000000004233333333222222
          00C0270D0540C0000C0098D0263C0444C0000C0000C0000C3019950531000000

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6G1. uvfix scripts & uvcopy jobs

uvfixB - modify ALL Fixed-Length files in subdir

 #1. mkdir tmp1 tmp2          <-- make subdirs for demo
 #1a. rm -f tmp1/* tmp2/*       - or clear if already existing

 #2. cp dat1/[123] tmp1/      <-- copy sales1,2,3 from dat1/... to tmp1/...

 #3. uvhd dat1/tmp1 r64       <-- investigate Input, must use uvhd (no LineFeeds)
     ==================         - see sample Input records for uvfix2 above
                                - note 6 digit date in bytes 13-18 (cols 14-19)
 rec#=1 rsize=64 fptr=0 fsize=1280 rcount=20
                   10        20        30        40        50        60
 r#     1 0123456789012345678901234567890123456789012345678901234567890123
        0 130140    21 940802 IN111001  HAM001  00002{ 0000001 00000002{
          3333332222332333333244333333224443332233333723333333233333333722
          130140000021094080209E1110010081D0010000002B00000001000000002B00

 rec#=20 rsize=64 fptr=1216 fsize=1280 rcount=20
                   10        20        30        40        50        60
 r#    20 0123456789012345678901234567890123456789012345678901234567890123
     1216 406082    65 020816 IN441     VAR200  00001R 0009001 00017101R
          3333332222332333333244333222225453332233333523333333233333333522
          406082000065002081609E441000006122000000001200009001000017101200

 #4. uvfixB tmp1 tmp2 r64s66  <-- run script, specify I/O subdirs,& options
     =======================      for in recsize & out recsize
                                - prompts & replies as follows:

      uop=q1n99r256s0    - user option defaults
            n99          - output record limit (n99 means all records)
               r256      - Input Record size & output recsize default (if s0)
               r8192     - max record-size 8192
                    s0   - Output Recsize defaulted from Input recsize if s0
                    s80  - may specify value to override the default from insize
      User OPtion (uop) defaults  = q1r8192n999999r64s66
 #4a. -->null accept or override -->  <-- null reply (options on command line)
 #4b. -->RSF = default file type -->  <-- null accept typ=RSF
 #4c. -->did you create outdir   --> y <-- reply 'y' to continue

 #5.      reply to instruction prompts as follows:
          ========================================
 #5a. --> ins  b13(51),'20'    <-- assume 2000+
 #5b. --> cmc  a13(2),'50'     <-- 1951-1999 ?
 #5c. --> skp< 1
 #5d. --> mvc  b13(2),'19'     <-- yes - change '20' to '19'
 #5e. --> .                    <-- enter '.' to end keyins & execute

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

uvfixB (cont) - Y2K demo check 1st output file

 #6. uvhd dat1/sales1 r64l66  <-- investigate Ouput, must use uvhd (no LineFeeds)
     =======================    - showing only 1st & last records below:

 rec#=1 rsize=66 fptr=0 fsize=1320 rcount=20
                   10        20        30        40        50        60
 r#     1 012345678901234567890123456789012345678901234567890123456789012345
        0 130140    21 19940802 IN111001  HAM001  00002{ 0000001 00000002{
          333333222233233333333244333333224443332233333723333333233333333722
          13014000002101994080209E1110010081D0010000002B00000001000000002B00

 rec#=20 rsize=66 fptr=1254 fsize=1320 rcount=20
                   10        20        30        40        50        60
 r#    20 012345678901234567890123456789012345678901234567890123456789012345
     1254 406082    65 20020816 IN441     VAR200  00001R 0009001 00017101R
          333333222233233333333244333222225453332233333523333333233333333522
          40608200006502002081609E441000006122000000001200009001000017101200
                       ^^ - Note century inserted 19/20 depending ><1950

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6H0. scripts to manipulate text files

dropbl0 - drop blank lines (using 'grep')

 #!/bin/ksh
 # dropbl0 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # dropbl0 - drop blank lines, by copy to tmp/.., inspect & copy back
 #         - by Owen Townsend, UV Software, Dec 2013
 #
 echo "dropbl0 * drop blank lines with grep (copy to tmp/, inspect & copy back)"
 echo "dropbl1 - alternate, same using uvcp with option b2 (gives line counts)"
 echo "dropbl2 - alternate, reduce multi blank lines to 1 (uvcp option b1)"
 echo "dropbl3 - alternate, drop blank lines & COBOL blank '*' lines (uvcopy)"
 df="$1";
 test -d tmp || mkdir tmp
 if [[ -f "$df" && -d tmp ]]; then :
 else echo "usage: dropbl0 subdir/file"
      echo "       ==================="
      echo " - arg1 must be a text file (with linefeeds & max recsize 512)"
      echo " - current working directory must also contain a tmp/ subdir"
      echo " - will copy file to tmp/..., for inspection & copy back"
      exit 1; fi
 #
 f=$(basename $df)
 d=${df%/*}
 grep -v "^ *$" $df >tmp/$f
 #=========================
 echo "file copied to tmp/$f, may inspect & copy back"
 echo "vi tmp/$f      <-- inspect"
 echo "cp tmp/$f $d   <-- copy back"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6H1. scripts to manipulate text files

dropbl1 - drop blank lines (using 'uvcp')

 #!/bin/ksh
 # dropbl1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # dropbl1 - drop blank lines, by copy to tmp/.., inspect & copy back
 #         - by Owen Townsend, UV Software, Dec 2013
 #
 echo "dropbl0 - drop blank lines with grep (copy to tmp/, inspect & copy back)"
 echo "dropbl1 * drop blnak lines using uvcp with option b2 (gives line counts)"
 echo "dropbl2 - reduce multi blank lines to 1 (uvcp option b1)"
 echo "dropbl3 - drop blank lines & COBOL blank '*' lines (uvcopy)"
 df="$1";
 test -d tmp || mkdir tmp
 if [[ -f "$df" && -d tmp ]]; then :
 else echo "usage: dropbl1 subdir/file"
      echo "       ==================="
      echo " - arg1 must be a text file (with linefeeds & max recsize 512)"
      echo " - current working directory must also contain a tmp/ subdir"
      echo " - will copy file to tmp/..., for inspection & copy back"
      exit 1; fi
 #
 f=$(basename $df)
 d=${df%/*}
 uvcp "fili1=$df,rcs=512,typ=LSTtb2,filo1=tmp/$f"
 #===============================================
 echo "file copied to tmp/$f, may inspect & copy back"
 echo "vi tmp/$f      <-- inspect"
 echo "cp tmp/$f $d   <-- copy back"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6H2. scripts to manipulate text files

dropbl2 - reduce multiple blank lines to 1

 #!/bin/ksh
 # dropbl2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # dropbl2 - reduce multiple contiguous blank lines to 1
 #         - by copy to tmp/.., for inspection & copy back
 #         - by Owen Townsend, UV Software, Dec 2013
 #
 echo "dropbl0 - drop blank lines wwith grep (copy to tmp/, inspect & copy back)"
 echo "dropbl1 - drop blnak lines using uvcp with option b2 (gives line counts)"
 echo "dropbl2 * reduce multi blank lines to 1 (uvcp option b1)"
 echo "dropbl3 - drop blank lines & COBOL blank '*' lines (uvcopy)"
 df="$1";
 test -d tmp || mkdir tmp
 if [[ -f "$df" && -d tmp ]]; then :
 else echo "usage: dropbl2 subdir/file"
      echo "       ==================="
      echo " - arg1 must be a text file (with linefeeds & max recsize 512)"
      echo " - current working directory must also contain a tmp/ subdir"
      echo " - will copy file to tmp/..., for inspection & copy back"
      exit 1; fi
 #
 f=$(basename $df)
 d=${df%/*}
 uvcp "fili1=$df,rcs=512,typ=LSTtb1,filo1=tmp/$f"
 #===============================================
 echo "file copied to tmp/$f, may inspect & copy back"
 echo "vi tmp/$f      <-- inspect"
 echo "cp tmp/$f $d   <-- copy back"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6H3. scripts to manipulate text files

dropbl3 - drop blank lines & *comment lines

 #!/bin/ksh
 # dropbl3 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # dropbl3 - drop blank lines, by copy to tmp/.., for inspection & copy back
 #         - option to drop blank COBOL comment lines '*' col 7 & all blank
 #         - by Owen Townsend, UV Software, Dec 2013
 #
 echo "dropbl0 - drop blank lines with grep  (copy to tmp/, inspect & copy back)"
 echo "dropbl1 - drop blnak lines using uvcp with option b2 (gives line counts)"
 echo "dropbl2 - reduce multi blank lines to 1 (uvcp option b1)"
 echo "dropbl3 * drop blank lines & COBOL blank '*' lines (uvcopy)"
 df="$1"; optn="$2"
 test -d tmp || mkdir tmp
 if [[ -f "$df" && -d tmp ]]; then :
 else echo "usage: dropbl3 subdir/file c0/c1"
      echo "       ========================="
      echo " - arg1 must be a text file (with linefeeds & max recsize 512)"
      echo " - arg2 c0/c1 'c1' drop '*' blank comments (as well as blank lines)"
      echo " - current working directory must also contain a tmp/ subdir"
      echo " - will copy file to tmp/..., for inspection & copy back"
      echo " - option c1 to drop lines with only '*'s (as well as blanks)"
      exit 1; fi
 #
 f=$(basename $df)
 d=${df%/*}
 uvcopy dropbl3,fili1=$df,filo1=tmp/$f,uop=$optn
 #==============================================
 echo "file copied to tmp/$f, may inspect & copy back"
 echo "vi tmp/$f      <-- inspect"
 echo "cp tmp/$f $d   <-- copy back"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6H4. scripts to manipulate text files

dropbl3 - uvcopy job to drop blank lines & *comment lines

 # dropbl3 - uvcopy Parameter File stored in: /home/uvadm/pf/util/
 # dropbl3 - drop blank lines with option to drop cobol *cmts
 #Dec14/14 - modify options c1,c2,c4 to drop COBOL *cmts if -,=,blanks
 #
 # uvcopy dropbl3,fili1=indir/infile,filo1=outdir/outfile,uop=c0/1/2/4
 # ===================================================================
 # - copies file to tmp/.., for inspection & copy back
 # - option to drop cobol *cmt lines with only '-'s, '='s, ' 's
 #
 # dropbl3 dir/file c0/c1/c2/c4  <-- script to call uvcopy job (as above)
 # ============================
 #
 # dropbl1 - also see script to drop ALL blank lines
 # dropbl2 - also see script to reduce multi blank lines to 1
 #
 opr='$jobname - drop blank lines with option to drop all "*" lines'
 opr='uop=q1c0j0s0t0 - option defaults'
 opr='    q0           - inhibit prompt to allow user to change options'
 opr='      c0       - do not drop *cmt lines with only "*"s'
 opr='      c1       - drop ALL COBOL *cmts "*" column 7'
 opr='      c2       - drop ALL COBOL *cmts "*" column 1 (cobmaps)'
 opr='      c4       - drop COBOL *cmts ONLY if no text descriptions'
 opr='                 IE - only "-"s, "="s,& blanks'
 opr='                 use c4 in combination with c1 &/or c2, for example:'
 opr='      c5       - drop COBOL *cmts "*" col 7 & only "-"s,"="s,or " "s'
 opr='        j1     - drop //* in cols 1-3 (comments in JCL)'
 opr='        j2     - drop # in col 1 (comments in JCL/scripts)'
 opr='          s1   - Squeeze left to column 1 & only 1 blank between words'
 opr='            t1 - Translate any unprintable characters to Blanks'
 uop=q1c0s0t0    # user option defaults
 fili1=?subdir/filexx,typ=LST,rcs=512
 filo1=?tmp/$fili1,typ=LSTt,rcs=512
 @run
        opn    all                    open files
 #
 # begin loop to get & put records until EOF
 man20  get    fili1,a0               get record into area 'a'
        skp>   man90                  (condition code set > at EOF)
        add    $ca1,1                 count input lines
        mvc    b0(512),a0             copy to output area
 #
 # drop all blank lines
 man24  cmc    a0(80),' '             line all blank ?
        skp=   man20                  yes - return to get next
 #

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

 # test option to drop COBOL *cmt lines '*' col 7 or 1
 man30  tsb    o3(1),x'01'            drop if '*' col 7 ?
        skp!   man35
        cmc    a6(1),'*'              '*' col 7 COBOL comment ?
        skp=   man20
 #
 # test option to drop COBOL *cmt lines '*' col 7 or 1
 man35  tsb    o3(1),x'02'            drop if '*' col 1 ?
        skp!   man40
        cmc    a0(1),'*'              '*' col 1 cobmap comment ?
        skp=   man20
 #
 # qualify drop */1 or */7 if option c4 only other chars '-' '=' ' '
 man40  tsb    o3(1),x'04'            qualify only '-' '=' ' ' ?
        skp!   man50                  no - drop */7 or */1
        mvc    c0(80),a0              copy to work area
        repm   c0(80),'*-=','   '     replace */-/= with blanks ?
        cmc    c0(80),' '             now all blank ?
        skp=   man20                  yes - bypass, return to get next
 #
 # test option to drop //* comments in JCL
 man50  tsb    o10(1),x'01'           drop if '//*' cols 1-3 ?
        skp!   man55
        cmc    a0(3),'//*'            '//*' cols 1-3 ?
        skp=   man20
 #
 # test option to drop //* comments in JCL
 man55  tsb    o10(1),x'02'           drop if '#' col 1 ?
        skp!   man60
        cmc    a0(1),'#'              '#' col 1 ?
        skp=   man20
 #
 # test option to translate any unprintables to blanks
 man60  tsb    o20(1),x'01'           translate unprintables to blanks ?
        skp!   man65
        trt    b0(512),$trtchr
 #
 # test option to squeeze multiblanks to 1 & left adjust
 man65  tsb    o19(1),x'01'           squeeze to col 1 ?
        skp!   man80
        sqzc1  b0(512),' '
 #
 # common point to output current line
 man80  put    filo1,b0               write record to output file
        add    $ca2,1                 count output lines
        skp    man20
 #
 man90  cls    all                    close files
        msgv1  'original file saved in $fili1, in=$ca1, out=$ca2    .'
        eoj                           end job

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6H5. scripts to manipulate text files

dropbl0,1,2,3 demo

Here we will illustrate 4 scripts to drop blank lines in text files. You can see the scripts listed at the link given at bottom of this page.

 #1. Login yourself --> /home/userxx
 #2. mkdir dat1 tmp     <-- make subdirs in your homedir
 #2a. rm tmp/*          <-- or remove tmp/* files if subdirs already existing

 #3. copy demo files from /home/uvadm/dat1 to your $HOME/dat1
     - renaming so we can test the 4 dropbl scripts with no conflicts

 #3a. cp /home/uvadm/dat1/testdropbl dat1/testdropbl0
      ===============================================
      - could copy as above, but better to use $UV as below
        (in case Vancouver Utilities not installed in usual /home/uvadm)

 #3a. cp $UV/dat1/testdropbl dat1/testdropbl0  # to test dropbl0
 #3b. cp $UV/dat1/testdropbl dat1/testdropbl1  # to test dropbl1
 #3c. cp $UV/dat1/testdropbl dat1/testdropbl2  # to test dropbl2
 #3d. cp $UV/dat1/testdropbl dat1/testdropbl3  # to test dropbl3

 #4a. dropbl0 dat1/testdropbl0  <-- drop all blank lines (sed)
      ========================
 #4b. dropbl1 dat1/testdropbl1  <-- drop all blank lines (uvcp option b2)
      ========================
 #4c. dropbl2 dat1/testdropbl2  <-- reduce multi-blank lines to 1 uvcp option b1
      ========================
 #4d. dropbl3 dat1/testdropbl3 c1 <-- drop COBOL blank '*' comments (uvcopy)
      ===========================     as well as blank lines

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

dat1/testdropbl demo INPUT file

 # testdropbl - test scripts to drop blank lines
 #            - see scripts dropbl0, dropbl1, dropbl2, dropbl3

# dropbl0 - uses 'sed' to drop all blank lines

# dropbl1 - uses 'uvcp' option 'b2' to drop all blank lines

# dropbl2 - uses 'uvcp' option 'b1' to reduce multi blank lines to 1

# dropbl3 - uses 'uvcopy' to drop all blank lines & all '*' COBOL comments

# begin test drop COBOL '*' comment blank lines (uvcopy) *

OUTPUT file tmp/testdropbl0 & 1 (same)

 # testdropbl - test scripts to drop blank lines
 #            - see scripts dropbl0, dropbl1, dropbl2, dropbl3
 # dropbl0 - uses 'sed' to drop all blank lines
 # dropbl1 - uses 'uvcp' option 'b2' to drop all blank lines
 # dropbl2 - uses 'uvcp' option 'b1' to reduce multi blank lines to 1
 # dropbl3 - uses 'uvcopy' to drop all blank lines & all '*' COBOL comments
 # begin test drop COBOL '*' comment blank lines (uvcopy)
       *

OUTPUT file tmp/testdropbl2

 # testdropbl - test scripts to drop blank lines
 #            - see scripts dropbl0, dropbl1, dropbl2, dropbl3

# dropbl0 - uses 'sed' to drop all blank lines

# dropbl1 - uses 'uvcp' option 'b2' to drop all blank lines

# dropbl2 - uses 'uvcp' option 'b1' to reduce multi blank lines to 1

# dropbl3 - uses 'uvcopy' to drop all blank lines & all '*' COBOL comments

# begin test drop COBOL '*' comment blank lines (uvcopy) *

OUTPUT file tmp/testdropbl3

 # testdropbl - test scripts to drop blank lines
 #            - see scripts dropbl0, dropbl1, dropbl2, dropbl3
 # dropbl0 - uses 'sed' to drop all blank lines
 # dropbl1 - uses 'uvcp' option 'b2' to drop all blank lines
 # dropbl2 - uses 'uvcp' option 'b1' to reduce multi blank lines to 1
 # dropbl3 - uses 'uvcopy' to drop all blank lines & all '*' COBOL comments
 # begin test drop COBOL '*' comment blank lines (uvcopy)

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6I1. scripts to manipulate text files

printable1 - ensure printable characters

 #!/bin/ksh
 # printable1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # printable1 - script to ensure printable characters in a text file
 #            - copies to tmp/... converting any unprintables to '.' periods
 #
 # Also see uvcopy jobs printable1
 #            - same name, but in $UV/pf/util/... vs $UV/sf/util/...
 # printable1 - same function using a uvcopy job (vs shell script 'tr')
 # printable2 - alternate for fixed record length files w/o LineFeeds
 #            - not possible to process fixed length records with tr script
 #
 echo "printable1 - ensure printable characters in a text file"
 echo "           - copies to tmp/... converting any unprintables to '.' periods"
 echo "printable2 - alternate for fixed record length files w/o LineFeeds"
 df="$1"
 if [[ -f "$df" && -d tmp ]]; then :
    else echo "usage: printable1 filename"
         echo "       ==================="
         echo "example: printable1 dat1/citytax1a"
         echo "         ========================="
         echo " - arg1 must be a file"
         echo " - must be a tmp/ subdir in current working dir"
         exit 1; fi
 #
 f=$(basename $df)
 tr -c '[:print:]\012' '.' <$df >tmp/$f
 #=====================================
 echo "$df copied to tmp/... converting unprintables to '.' periods"
 echo "more tmp/$f  <-- enter to display file with more"; read reply
 more tmp/$f
 #==========
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6J1. scripts to manipulate text files

parmnullfix1 - copy all files, nulls to blanks, using 'sed'

 #!/bin/ksh
 # parmnullfix1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # parmnullfix1 - copy all files from 1 directory to a 2nd directory
 #              - converting any nulls to blanks, using sed
 # parmnullfix2 - alternate script calling 'uvcp' (vs sed)
 #              - also clears any sequence#s in cols 73-80
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
    else echo "usage: parmnullfix1 dir1 dir2"
         echo "       ======================"
         echo "- arg1 & arg2 must be input & output directories"
    exit 9; fi
 if [[ ! -d tmp ]]; then mkdir tmp; fi
 ls $d2 >tmp/$d2.emptytest
 if [[ -s tmp/$d2.emptytest ]]; then
    echo "the output directory must be empty $d2";exit 93; fi
 integer fcount=0 lcount=0 tlcount=0 nlcount=0 nfcount=0
 for d1f in $d1/*
 do
    f=$(basename $d1f)
    ((fcount+=1)); nlcount=0;
    nlcount=$(grep -Pac '\x00' $d1/$f)
    if ((nlcount)); then ((nfcount+=1)); fi
    wcount=$(wc -l $d1/$f 2>/dev/null); lcount=${wcount% *}; ((tlcount+=lcount));
    echo "file#=$fcount lines=$lcount nulls=$nlcount file=$d1/$f"
    sed 's/\x00/ /g' $d1/$f >$d2/$f
    #==============================
 done
 echo "total $fcount files, $nfcount with nulls converted to blanks"
 echo "total $fcount files, $tlcount lines, copied from $d1 to $d2"
 echo "- also see parmnullfix2 also clears 73-80 (using uvcp vs sed)"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6J2. scripts to manipulate text files

parmnullfix2 - nulls to blanks & clear 73-80, using 'uvcp'

 #!/bin/ksh
 # parmnullfix2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # parmnullfix2 - copy all files from 1 directory to a 2nd directory
 #              - using 'uvcp' to convert nulls to blanks & clear cols 73-80
 # parmnullfix1 - alternate script using 'sed' (does not clear 73-80)
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
    else echo "usage: parmnullfix2 dir1 dir2"
         echo "       ======================"
         echo "- arg1 & arg2 must be input & output directories"
    exit 9; fi
 if [[ ! -d tmp ]]; then mkdir tmp; fi
 ls $d2 >tmp/$d2.emptytest
 if [[ -s tmp/$d2.emptytest ]]; then
    echo "the output directory must be empty $d2";exit 93; fi
 integer fcount=0 lcount=0 tlcount=0 nlcount=0 nfcount=0
 for d1f in $d1/*
 do
    f=$(basename $d1f)
    ((fcount+=1)); ncount=0;
    nlcount=$(grep -Pac '\x00' $d1/$f)
    if ((nlcount)); then ((nfcount+=1)); fi
    wcount=$(wc -l $d1/$f 2>/dev/null); lcount=${wcount% *}; ((tlcount+=lcount));
    if ((ncount)); then ((nfcount+=1)); fi
    echo "file#=$fcount nulls=$ncount file=$d1/$f"
    uvcp "fili1=$d1/$f,rcs=256,typ=LSTt,filo1=$d2/$f,clr=72(28),rop=i15"
    #==================================================================
 done
 echo "total $fcount files, $nfcount with nulls converted to blanks"
 echo "total $fcount files, $tlcount lines, copied from $d1 to $d2"
 echo "- using uvcp to clear cols 73-80 (vs parmnullfix1/sed blank nulls only)"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_7 Vancouver Utility scripts for Conversion & General use

copying & sorting files from 1 dir to a 2nd dir

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

7A1. copying & sorting files from 1 dir to a 2nd dir

copycpio1 - copy subdirs/files from indir to outdir

 #!/bin/ksh
 # copycpio1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # copycpio1 - copy a directory tree to a 2nd directory
 #           - input may have sub-directories to any level
 #           - output will have exactly the same tree structure
 #           - you must be in input directory
 #           - output directory must be empty
 #           - you must specify full pathname of indir & outdir
 #           - fail-safe, paranoid script
 #           - also see copycpio2, similar to this *copycpio1, except:
 #*copycpio1 - requires you to be in input directory
 # copycpio2 - requires you to be in output directory
 #
 # Example - copy /p2/proddata/... to /p3/backup/proddata
 # 1. cd /p2/proddata                  <-- change to input directory
 # 2. ls -l /p3/backup/proddata | more <-- ensure outdir OK to erase
 # 3. rm -rf /pr/backup/proddata/*     <-- clear output directory
 # 4. copycpio1 /p2/proddata /p3/backup/proddata <-- do it
 #    ==========================================
 #
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
    else echo "usage: copycpio1 indir outdir"
         echo "       ======================"
         echo " - arg1 & arg2 must be directories (input & output)"
         exit 91; fi
 #
 dpath=$(pwd)                      # capture current directory path
 if [[ $dpath != $d1 ]]
    then echo "you must be in the input directory $d1";
          exit 92; fi
 #
 ls $2 >/tmp/copycpio_emptytest
 if [[ -s /tmp/copycpio_emptytest ]]
    then echo "the output directory must be empty $d2";exit 93; fi
 #
 echo "copy $d1 to $d2 OK ? (or kill)"; read reply
 cd $d1                          # <-- change to indir
 find . -print | cpio -pdmv $d2  # <-- copy all files to outdir
 #=============================
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

7B1. copying & sorting files from 1 dir to a 2nd dir

copycpio2 - copy subdirs/files to outdir from indir

 #!/bin/ksh
 # copycpio2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # copycpio2 - copy a directory tree to a 2nd directory
 #           - input may have sub-directories to any level
 #           - output will have exactly the same tree structure
 #           - you must be in output directory & it must be empty
 #           - you must specify full pathname of outdir & it must match .
 #           - fail-safe, paranoid script
 #           - also see copycpio1, similar to this *copycpio2, except:
 # copycpio1 - requires you to be in input directory
 #*copycpio2 - requires you to be in output directory
 #
 # Example - copy /p2/proddata/... to /p3/backup/proddata
 # 1. cd /p3/backup/proddata         <-- change to output directory
 # 2. ls -l | more                   <-- ensure outdir OK to erase
 # 3. rm -rf *                       <-- clear output directory
 # 3a. rm -rf /p3/backup/proddata/*  <-- safer (avoid unqualified '*')
 # 4. copycpio2 /p2/proddata /p3/backup/proddata <-- do it
 #    ==========================================
 #
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
    else echo "usage: copycpio2 indir outdir"
         echo "       ======================"
         echo " - arg1 & arg2 must be directories (input & output)"
         exit 91; fi
 #
 dpath=$(pwd)                      # capture current directory path
 if [[ $dpath != $d2 ]]
    then echo "you must be in output directory (& it must be empty) $d2";
          exit 92; fi
 #
 ls . >/tmp/copycpio_emptytest
 if [[ -s /tmp/copycpio_emptytest ]]
    then echo "the current directory (output) must be empty $d2";exit 93; fi
 #
 echo "copy $d1 to current dir $d2 OK ? (or kill job)"; read reply
 cd $d1                          # <-- change to indir
 find . -print | cpio -pdmv $d2  # <-- copy all files to outdir
 #=============================
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

7C1. copying & sorting files from 1 dir to a 2nd dir

copydate - copy file to a directory, appending date-stamp

 # copydate - copy a file, appending a date stamp
 #          - by Owen Townsend, UV Software, Jan30/06
 # copydatetime - alternative script if you want time as well as date
 #
 #usage: copydate inputfile outputdirectory
 #       ==================================
 #       - arg1 must be a file, arg2 must be a directory
 #
 #example: copydate data/test1 tmp
 #         =======================
 #         - output would be: tmp/test1_yymmdd
 #
 if [[ -f "$1" && -d "$2" ]]; then :
 else echo "usage: copydate inputfile outputdirectory"
      echo "       =================================="
      echo "       - arg1 must be a file, arg2 must be a directory"
      exit 1; fi
 #
 fil="$1"; dir="$2";          # capture input-file-name & output-directory-name
 fbn=$(basename $fil)         # extract file basename, dropping any directories
 fbndt=${fbn}_$(date +%y%m%d) # append date stamp
 #
 cp $fil $dir/$fbndt          # copy inputfile to outdir appending date stamp
 #===================
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

7D1. copying & sorting files from 1 dir to a 2nd dir

cpnew1 - copy files to a 2nd subdir newer than x days

 #!/bin/ksh
 # cpnew1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # cpnew1 - copy files from 1 directory to a 2nd directory
 #          that are newer than a specified number of days
 #
 #usage:  cpnew1 indir outdir days
 #        ========================
 #
 #note - this script illustrates reading text lines from a file
 #       (a powerful feature of the KORN shell)
 #       'exec 3<' to open, 'read -u3 field' to read, 'exec 3<&-' to close
 #
 if [ -d "$1" -a -d "$2" -a -n "$3" ]; then :
 else echo "USAGE: cpnew1 indir outdir days"; exit 1; fi
 #
 # create tmp dir in current dir if not already present
 if [[ ! -d tmp ]]; then mkdir tmp; fi
 #
 # create list of filenames newer than specifed no of days
 find $1/* -mtime -$3 -print >tmp/cpnew_files
 #===========================================
 #
 x=0;
 exec 3< tmp/cpnew_files    # open namesfile (using file descriptor #3)
 while read -u3 fp
 do
   let x=x+1
   fn=${fp##*/}                   # strip off left path to filename only
   echo "copy file# $x - $1/$fn "
   cp -p $1/$fn $2/$fn
   #==================
 done
 exec 3<&-           # close file descriptor #3
 echo "$x files (newer than $3 days) copied from $1 to $2"
 exit

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

7D2. copying & sorting files from 1 dir to a 2nd dir

cpnew2 - copy files within a modification date range

 #!/bin/ksh
 # cpnew2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # cpnew2 - copy files from 1 directory to a 2nd directory whose last
 #          modification times are within a specified date range
 #          specify dates as for the 'touch' command--> YYMMDDhhmm
 #
 #usage:  cpnew2 indir outdir YYMMDDhhmm1 YYMMDDhhmm2
 #        ===========================================
 #
 #example: cpnew2 dir1 dir2 9804010000 9804302359   copy April 98 files
 #         ======================================
 #
 if [[ (-d "$1") && (-d "$2") && (-n "$3") && (-n "$4") ]]; then :
 else echo "USAGE: cpnew2 indir outdir YYMMDDhhmm1 YYMMDDhhmm2 "; exit 1; fi
 #
 if (( (${#3} == 10) && (${#4} == 10) )); then :
 else echo "arg3 & arg4 must be 10 digits (YYMMDDhhmm)"; exit 2; fi
 #
 # create tmp dir in current dir if not already present
 if [[ ! -d tmp ]]; then mkdir tmp; fi
 #
 # create 2 files & use touch to set dates to the specified dates
 >tmp/cpnew2date1; >tmp/cpnew2date2
 touch -am -t $3 tmp/cpnew2date1
 touch -am -t $4 tmp/cpnew2date2
 #
 x=0; y=0;
 for f in $1/*
 do
   let x=x+1
   if [[ (-f $f) && ($f -nt tmp/cpnew2date1) && ($f -ot tmp/cpnew2date2) ]]
   then let y=y+1
        fn=${f##*/}                     # strip left to filename only
        echo "copy file# $x - $1/$fn "
        cp -p $1/$fn $2/$fn
        #==================
   fi
 done
 echo "$y files of $x total (dated $3 to $4) copied from $1 to $2"
 exit

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

7E1. copying & sorting files from 1 dir to a 2nd dir

sortcpio - copy/sort all subdirs/files from dir1 to dir2

 #!/bin/ksh
 # sortcpio - Korn shell script from UVSI stored in: /home/uvadm/sf/adm/
 # sortcpio  - sort filenames & copy a directory to a 2nd directory
 #           - so filenames will be in sequence for backups, etc
 #             (ls sorts filenames, but not tar, cpio, du, etc)
 #           - input may have sub-directories to any level
 #           - output will have exactly the same tree structure
 #             except directory names & filenames will be sorted
 #
 if [[ -d $1 && -d $2 ]]; then :
    else echo "usage: sortcpio indir outdir"
         echo "       ====================="
         echo "- you must be outside of dirs & outdir must be empty"
         exit 1; fi
 #
 echo "current directory must be outside of indir & have tmp subdir - OK ?";read
 echo "outdir can be anywhere outside of indir & must be empty - OK ?";read
 #
 cwd=$(pwd)
 cd $1; d1=$(pwd); cd $cwd
 cd $2; d2=$(pwd); cd $cwd
 if [[ ! -d tmp ]];then mkdir tmp; fi
 cd $d1
 #
 find . -print >$cwd/tmp/files1
 #=============================
 sort -o$cwd/tmp/files2 $cwd/tmp/files1
 #=====================================
 cat $cwd/tmp/files2 | cpio -pdmv $d2
 #===================================
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

7F1. copying & sorting files from 1 dir to a 2nd dir

sortall - sort all text files from dir1 to dir2

 #!/bin/ksh
 # sortall - sort all text files from 1 subdir to a 2nd subdir
 #         - maintaining same filenames in output directory
 #         - using unix/linux 'sort' utility vs 'uvsort' (sortallu1/u2)
 #
 d1=$1; d2=$2; u="$3"; u="";
 if [[ -d "$d1" && -d "$d2" ]]; then :
    else echo "usage: sortall indir outdir [-u]"
         echo "       ========================="
         echo " - arg1 & arg2 must be subdirs"
         echo " - arg3 optional, '-u' to sort UNIQUE"
    exit 1; fi
 #
 if [[ -n "$3" ]]; then u="$3"; fi
 x=0
 for df in $d1/*
   { f=${df##*/}
     sort $u -o $d2/$f $d1/$f
     let x=x+1
   }
 echo "$x files sorted $3 from $d1 to $d2"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_8 Vancouver Utility scripts for Conversion & General use

system administration scripts

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

8A1. Notes about system/application administration scripts

1. These scripts are also documented in ADMjobs.htm, along with other system administration scripts such as backup scripts & console log processing scripts.
   
   
2. We should use 'root' only when absolutely necessary (as in killuser).
   
   
3. The backup scripts (& most scripts ADMjobs.doc) do not need root, because they backup the application data & libraries (not the unix/linux system).
   
   
4. The backup scripts are run by cron, but the crontab is not owned by root. The nightly 'crontabs' are owned by 'appsadm' which is in the same group as the programmers & operators, and has access to the data & libraries.
   
   
5. Note that scripts run by 'cron' do not get the owner's profile executed automatically, so these scripts 1st '.' dot execute the profile (/home/appsadm/env/stub_profile). Then these scripts have all the usual environmental variables & PATHs setup to run the same as for console operation.
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

8B1. system/application administration scripts

killuser1 - kill user by username (vs process#)

 #!/bin/ksh
 # killuser1 - Korn shell script from UVSI stored in: /home/uvadm/sf/adm/
 # killuser1 - kill any 1 user (must be run by root)
 #           - by Owen Townsend, www.uvsoftware.ca May 2002
 #           - also see killuser2 to kill all users running bash
 #             (killuser2 intended for cron at 3AM or whatever)
 #
 # verify arg1 specified
 user="$1"
 if [ -z "$user" ]; then
    echo "arg1 userid must be specifeid"
    echo "usage: killuser1 userid"
    echo "       ================"
    exit 1; fi
 #
 # redirect ps output to a tmp file
 # - use '-o' output option for 3 fields only (COMMAND, PID,& RUSER)
 ps -u $user -ocomm -opid -oruser >/tmp/kill$user
 #===============================================
 #    ** sample output lines **
 # COMMAND            PID RUSER
 # init                 1 root
 # bash              9022 root
 # bash              9077 uvadm
 # ps                9226 uvadm
 #
 # - open the file & read back into variables for easier manipulation
 exec 3< /tmp/kill$user        # open file #3
 #
 x=0; y=0
 while read -u3 comm pid ruser
   do if [[ ("$ruser" = "$user") && ("$comm" = "bash") ]]
         then let x=x+1
              kill -9 $pid
              #===========
              if [[ $? = 0 ]]; then kok=OK; let y=y+1; else kok=NAK; fi
              echo "#$x kill $comm $pid $ruser - $kok"
      fi
   done
 exec 3<&-                    # close file #3
 echo "$x kills attempted, $y killed OK $(date)"

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

8B2. system/application administration scripts

killuser2 - kill all users who forgot to logoff

 #!/bin/ksh
 # killuser2 - Korn shell script from UVSI stored in: /home/uvadm/sf/adm/
 # killuser2 - kill users who did not logout
 #           - by Owen Townsend, www.uvsoftware.ca, March 2002
 #
 # - intended to be run by crontab_root at 11:45 PM or whenever
 # - killuser2 is run before logfixN which processes user console logs
 # - this kills user 'script' & closes the script output file
 # - also see 'killuser1' script for interactive use to kill any 1 user
 #
 if [ "$1" != "all" ]; then
    echo "usage: killuser2 all"
    echo "       ============="
    echo " - arg1 must be 'all'"
    exit 1; fi
 #
 # redirect ps -f output to a tmp file
 # - use '-o' output option for 3 fields only (COMMAND, PID,& RUSER)
 ps -e -ocomm -opid -oruser >/tmp/psef
 #====================================
 #    ** sample output lines **
 # COMMAND            PID RUSER
 # init                 1 root
 # bash              9022 root
 # bash              9077 uvadm
 # ps                9226 uvadm
 #
 # - open the file & read back into variables for easier manipulation
 exec 3< /tmp/psef            # open file #3
 #
 x=0; y=0
 while read -u3 comm pid ruser
 do if [[ ("$comm" = ksh || "$comm" = bash) && ("$ruser" != root) ]]
       then let x=x+1
            kill -9 $pid
            #===========
            if [[ $? = 0 ]]; then kok=OK; let y=y+1; else kok=NAK; fi
            echo "#$x kill $comm $pid $ruser - $kok"
    fi
 done
 exec 3<&-                    # close file #3
 echo "$x kills attempted, $y killed OK $(date)"

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

8B3. system/application administration scripts

crontab_root - to schedule 'killuser2' at 3 AM

 # crontab_root - crontab file to run under root
 #              - by Owen Townsend, UV Software, May 2008
 # - supplied in /home/uvadm/sf/adm/crontab_root
 # - copy to /home/appsadm/sf/crontab_root before customization & activation
 # - see details at www.uvsoftware.ca/admjobs.htm#Part_5
 #
 # Only for backups, cleanups, etc that require 'root'
 # - see crontab_apps1 for backups & applications (daily,weekly,monthly,etc)
 # - 'crontab_apps1' runs under 'appsadm' (not root) & much safer
 #
 # Actual crontab for root depends on the unix/linux OS
 # - you need to find it & add desired lines
 # OR, you could use the following procedure so you could maintain
 # all crontabs in 1 place /home/appsadm/sf/...
 # - after you have retrieved & combined root crontabs with this file
 # - for RHEL 5.1 there is no root crontab, so just copy this
 #
 # suggested procedures for updating root crontab:
 # 1. logon as root & cd to /home/appsadm
 # 2. crontab -l (if 1st setup)   - list to see if any crontab exists for root
 # 3. crontab -l >sf/crontab_root - ifso, redirect to sf/appsadm
 # 4. vi sf/crontab_root          - edit this file as required
 # 4a. :r /home/uvadm/sf/adm/crontab_root   <-- append this supplied file
 # 5. crontab -r                  - remove old crontab file for root
 # 6. crontab sf/crontab_root     - activate new crontab for root
 # 7. crontab -l                  - list crontab file to confirm installation
 #
 # minute hour day-of-mth mth-of-yr day-of-week <----command---->
 #
 15 00 * * * /home/appsadm/sf/killuser2 all  # kill users at 12:15AM every night
 #=========================================
 # - killuser2 kills all 'ksh' & 'bash' users who did not log off
 # - also see killuser1 to interactively kill any 1 specified userid
 #
 30 00 * * 2-6 /home/appsadm/sf/setperms1 all # set perms at 12:30AM Tues-Sat
 #===========================================
 # At 12:30 AM Tues-Sat, run setperms1 to set permissions on data & libraries
 # - ensure directories 775, data-files 664, script-files 775
 # - ensure owner:group appsadm:apps (see details in setperms1 script)
 #
 00 01 * * 0 /sbin/shutdown -g0 -y -i6  # reboot at 01:00AM Sunday
 #====================================    (above command for traditional unix)
 # reboot Sunday 1 AM -g0(no wait) -y(auto reply y to prompt) -i6(reboot)
 # 00 01 * * 0 /sbin/shutdown -r now    #<-- can use this for Linux
 # =================================
 #
 #Note: Add commands here to bring up software packages, such as:
 # - Micro Focus COBOL license manager
 # - Online systems (MTO, unikix, CICS6000, etc)

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

8C1. system/application administration scripts

chmod1 - set perms on all levels - using 'find'

 #!/bin/ksh
 # chmod1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/...
 # chmod1 - change permissions on subdirs & files under a specified superdir
 #        - using 'find' to process all levels of directories & files
 #        - need to manually add 'x' perms on any bin/* & script/* dirs
 #        - by Owen Townsend, April 26/2006
 #
 # chmod1 directory dir-perms file-perms   <-- command format
 # ======================================
 # chmod1 directory 775 664                <-- recommended permissions
 # =========================
 #
 # - also see alternatives chmod2 & chmod3 (to this *chmod1)
 #*chmod1 - change perms on dirs & files, using 'find'
 # chmod2 - change perms on dirs & files, using 'recursion'
 # chmod3 - change perms on dirs & files, using 'recursion'
 #        - sets 'x' perm by testing for known names of bin/ & script/ subdirs
 #
 # After running chmod1 or chmod2, you must manually fix permissions on
 # executable programs & scripts via -->chmod 775 bin/*; chmod 775 scripts/*
 #
 # capture arguments & force perms integers
 dir="$1";
 typeset i dperm="$2"; typeset i fperm="$3";
 #
 # ensure arg1 is directory & length of perms are 3 digits
 dpl=${#dperm}; fpl=${#fperm};
 #
 if [[ -d "$dir" ]] && ((dpl==3 && fpl==3)); then :
    else echo "usage: chmod1 directory dir-perms file-perms"
         echo "       ====================================="
         echo "example: chmod1 dirxx 775 664"
         echo "         ===================="
         echo " - arg1 must be dir, args 2 & 3 must be 3 digits"
         exit 90; fi
 #
 echo -n "chmod1: set perms $dperm/$fperm from: $dir - enter to continue"
 read reply
 #
 find $dir -type d -exec chmod $dperm {} \;
 #=========================================
 #
 find $dir -type f -exec chmod $fperm {} \;
 #=========================================
 #
 echo "chmod1: perms set $dperm/$fperm for all subdirs & files within: $dir"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

8C2. system/application administration scripts

chmod2 - set perms on all levels - using 'recursion'

 #!/bin/ksh
 # chmod2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/...
 # chmod2 - change permissions on subdirs & files under a specified superdir
 #         - using 'recursion' to process all levels of subdirs
 #         - need to manually add 'x' perms on any bin/* & script/* dirs
 #         - by Owen Townsend, June 13/2006
 #
 # chmod2 directory dir-perms file-perms    <-- command format
 # =====================================
 # chmod2 directory 775 664     <-- recommended permissions
 # ========================
 # - also see alternatives chmod1 & chmod3 (to this *chmod2)
 # chmod1 - change perms on dirs & files, using 'find'
 #*chmod2 - change perms on dirs & files, using 'recursion'
 # chmod3 - change perms on dirs & files, using 'recursion'
 #        - sets 'x' perm by testing for known names of bin/ & script/ subdirs
 # After running chmod2 or chmod3, you must manually fix permissions on
 # executable programs & scripts via -->chmod 775 bin/*; chmod 775 scripts/*
 # capture arguments & force perms integers
 dir="$1";
 typeset i dperm="$2"; typeset i fperm="$3";
 # ensure arg1 is directory & length of perms are 3 digits
 dpl=${#dperm}; fpl=${#fperm};
 if [[ -d "$dir" ]] && ((dpl==3 && fpl==3)); then :
    else echo "usage: chmod2 directory dir-perms file-perms"
         echo "       ====================================="
         echo "example: chmod2 dirxx 775 664"
         echo "         ===================="
         echo " - arg1 must be dir, args 2,3 must be 3 digits"
         exit 90; fi
 echo -n "chmod2: Begin directory $dir - enter to continue"
 read reply
 #note - remove the pause (read reply) above if desired
 integer nd=0 nf=0
 for df in $dir/*
   { if [[ -d $df ]]
        then chmod $dperm $df; ((nd+=1))
        else chmod $fperm $df; ((nf+=1))
     fi
     # if current entry in current dir is a subdir
     # - call this script again to process its files & subdirs
     # - using 'recursion' to process all existing levels of subdirs
     if [[ -d $df ]]; then
        ls $df >/tmp/chmod2
        if [[ -s /tmp/chmod2 ]]; then
           chmod2 $df $dperm $fperm
           #=======================
        fi
     fi
   }
 echo "chmod2: End dir $dir, subdirs=$nd, files=$nf"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

8C3. system/application administration scripts

chmod3 - set perms using recursion & add x to bin/script

 #!/bin/ksh
 # chmod3 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/...
 # chmod3 - change permissions on subdirs & files under a specified superdir
 #        - using 'recursion' to process all levels of subdirs
 #        - adds 'x' perms on any bin/* & script/* dirs, by testing
 #          for known names of bin & script subdirs
 #        - by Owen Townsend, June 13/2006
 #
 # chmod3 directory dir-perms file-perms xfile-perms   <-- command format
 # ==================================================
 # chmod3 directory 775 664 775    <-- recommended permissions
 # =============================
 # - also see alternatives chmod1 & chmod2 (to this *chmod3)
 # chmod1 - change perms on dirs & files, using 'find'
 # chmod2 - change perms on dirs & files, using 'recursion'
 #*chmod3 - change perms on dirs & files, using 'recursion'
 #        - sets 'x' perm by testing for known names of bin/ & script/ subdirs
 # After running this script, you must manually fix permissions on executable
 # programs & scripts via -->chmod 775 bin/*; chmod 775 scripts/*
 dir="$1";
 typeset i dperm="$2"; typeset i fperm="$3"; typeset i xperm="$4";
 # ensure arg1 is directory & length of perms are 3 digits
 dpl=${#dperm}; fpl=${#fperm}; xpl=${#xperm};
 if [[ -d "$dir" ]] && ((dpl==3 && fpl==3 && xpl==3)); then :
    else echo "usage: chmod3 directory dir-perms file-perms xfile-perms"
         echo "       ================================================="
         echo "example: chmod3 dirxx 775 664 775"
         echo "         ========================"
         echo " - arg1 must be dir, args 2,3,4 must be 3 digits"
         exit 90; fi
 echo -n "chmod3: Begin directory $dir - enter to continue "; read reply
 #note - remove the pause (read reply) above if desired
 integer nd=0 nf=0 nxf=0
 for df in $dir/*
   { if [[ -d $df ]] then chmod $dperm $df; ((nd+=1))
        elif [[ $df == *script* || $df == *sf* || $df == *bin*
                                 || $df == *program* || $df == jcls ]]
             then chmod $xperm $df; ((nxf+=1))
        else chmod $fperm $df; ((nf+=1))
     fi
     # if current entry in current dir is a subdir
     # - call this script again to process its files & subdirs
     # - using 'recursion' to process all existing levels of subdirs
     if [[ -d $df ]]; then
        ls $df >/tmp/chmod3
        if [[ -s /tmp/chmod3 ]]; then
           chmod3 $df $dperm $fperm $xperm
           #==============================
        fi
     fi
   }
 echo "chmod3: End dir $dir, subdirs=$nd, files=$nf, xfiles=$nxf"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

8D1. system/application administration scripts

ftrigger51 - test file presence & run a script

 # ftrigger51 - script to test for a file presence & run a 2nd script
 #            - could use to run a JCL/script when a file is FTP'd to server
 #            - by Owen Townsend, March 30/2005
 #            - 2 versions of this script (ftrigger51 & ftrigger52)
 #*ftrigger51 - simple version with hard-coded filename & jobname
 #            - need to copy/rename/modify for other filenames & jobnames
 # ftrigger52 - general purpose version with 2 args for filename & jobname
 #
 jobset51;  # call setup function common to scripts converted from mainframe JCL
 #            - remove 'jobset51' for scripts unrelated to mainframe conversions
 # 'jobset51' - a Vancouver Utility function to setup the job environment
 #            - changes directory to $RUNDATA (with subdirs ftp, etc)
 #            - see MVSJCL.htm#1C3 explanation & listing at MVSJCL.htm#3C1
 # assign variables for hard-coded file/job names up front in 1 place only
 # - to make script easier to copy/rename/modify for other filenames/jobnames
 file1="ftp/customer.master"     # hard-code filename here (1 place only)
 script1="jar160.ksh"            # hard-code jobname here (1 place only)
 logmsg "$0 waiting for $file1 to run $script1"
 # sleep 2 seconds repeatedly, until the arg1 file is detected
 until [[ -f $file1 ]]
    do sleep 2
    done
 #
 # file presence detected - run arg2 JCL/script & rename the file
 file2=${file1}_$(date +%y%m%d:%H%M%S)  # add _date:time stamp on filename
 $script1        # run the JCL/script with no arguments (vs ftrigger52)
 #=======
 cc=$?           # capture return status
 if (($cc != 0))
    then logmsg "$script1 $file1 FAILED (return code $cc)"; exit $cc; fi
 mv $file1 $file2   # rename the input file (so we can rerun this script)
 #===============
 logmsg "$script1 ran OK & $file1 renamed to $file2"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

8D1. system/application administration scripts

ftrigger52 - G.P. test file presence & run a script

 # ftrigger52 - script to test for a file presence & run a 2nd script
 #            - could use to run a JCL/script when a file is FTP'd to server
 #            - by Owen Townsend, March 30/2005
 #            - 2 versions of this script (ftrigger51 & ftrigger52)
 # ftrigger51 - simple version with hard-coded filename & jobname
 #            - would need to copy/rename/modify for other filenames & jobnames
 #*ftrigger52 - general purpose version with 2 args for filename/jobname
 #
 jobset51;  # call setup function common to scripts converted from mainframe JCL
 #            - remove 'jobset51' for scripts unrelated to mainframe conversions
 # 'jobset51' - a Vancouver Utility function to setup the job environment
 #            - changes directory to $RUNDATA (with subdirs ftp, etc)
 #            - see MVSJCL.htm#1C3 explanation & listing at MVSJCL.htm#3C1
 #
 if [[ -n "$1" && -n "$2" ]]; then :
 else echo "usage: ftrigger52 filename JCL/script  <-- 2 arguments must be coded"
      echo "       ============================="
      echo "example: ftrigger52 ftp/customer.master jar160.ksh"
      echo "         ========================================"
      exit 1; fi
 #
 file1=$1; script1=$2;   # name the variables
 logmsg "$0 waiting for $file1 to run $script1"
 #
 # sleep 2 seconds repeatedly, until the arg1 file is detected
 until [[ -f $file1 ]]
    do sleep 2
    done
 #
 # file presence detected - run arg2 JCL/script & rename the file
 file2=${file1}_$(date +%y%m%d:%H%M%S)  # add _date:time stamp on filename
 #
 $script1 $file1    # run the JCL/script with filename as arg1
 #==============
 cc=$?              # capture return status
 if (($cc != 0))
    then logmsg "$script1 $file1 FAILED (return code $cc)"; exit $cc; fi
 #
 mv $file1 $file2   # rename the input file (so we can rerun this script)
 #
 logmsg "$script1 ran OK & $file1 renamed to $file2"
 exit 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Permuted Index of Keywords from ** Headings **

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Visitor Counters for ThisYear and LastYear