Training Guide Unix for Beginners

www.caosys.com [email protected]

UNIX For Beginners

PLEASE NOTE This training guide was produced many years ago and is now offered to you free of charge. We make no claim that the material is free from error nor to its relevance or accuracy but we do hope that you find it useful as a beginners guide.

Training Guide Unix for Beginners

www.caosys.com [email protected]

Course Introduction

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 2

Training Guide Unix for Beginners

www.caosys.com [email protected]

Course Objectives This course is designed to give each delegate a basic understanding/awareness of the following: Basic UNIX commands  Basic Shell Programming/Scripting  Source Control System

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 3

Training Guide Unix for Beginners

www.caosys.com [email protected]

Course Objectives This course is designed to give each delegate a basic understanding of the following topics:  Basic UNIX commands. The course covers some of the most commonly used commands, from logging in and out of the system to editing files.  Basic Shell Programming is covered; the delegate should be equipped with enough knowledge to enable them to build basic shell scripts.  Source Control System is covered. During the course there will be several labs exercises to perform.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 4

Training Guide Unix for Beginners

www.caosys.com [email protected]

Course Agenda The course is split into 3 sections: Section 1 : Basic UNIX commands  Section 2 : Basic Shell Programming (other useful UNIX commands)  Section 3 : Source Code Control System

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 5

Training Guide

www.caosys.com [email protected]

Unix for Beginners

Course Agenda Section 1 – Basic UNIX Commands Before We Start – Basic Concepts Logging In/Out Getting Help File Management Searching Text & File Utilities Processes vi – The Text Editor Working with the Shell Summary

8 9 14 16 17 50 55 67 78 83 94

Section 2 – Basic Shell Programming What is a Shell program/Script? Executing/Running a Shell Script Shell Variables Command Line Parameters Basic String Operators Arithmetic Expressions Decisions Decisions Loop the Loop Summary

96 99 101 104 107 110 119 122 131 135

Section 3 – Source Code Control System (SCS) What is the Source Code Control System? Directory Structures SCS Commands Summary

137 140 142 145 160

What is Next?

161

Answers to Exercises

165

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 6

Training Guide Unix for Beginners

www.caosys.com [email protected]

Section One Basic UNIX Commands

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 7

Training Guide Unix for Beginners

www.caosys.com [email protected]

Basic UNIX Commands  Before We Start – Basic Concepts  Logging In/Out  Getting Help  File Management  Searching  Text & File Utilities  Processes  vi – the text editor  Working with the Shell

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 8

Training Guide Unix for Beginners

www.caosys.com [email protected]

Before We Start – Basic Concepts Be we start on the main course work, we should discuss a few UNIX concepts: Multi-user  Multi-tasking  Ownership & Permissions  The Command Line  Shells  Input And Output  Pipes and Redirection  UNIX Filesystem  Paths – Absolute and Relative

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 9

Training Guide Unix for Beginners

www.caosys.com [email protected]

Before We Start – Basic Concepts Before we start on the main coursework, we should quickly discuss a few UNIX concepts:Multi-User UNIX is a multi-user system, meaning that more than one user can be using a single machine at the same time. The UNIX operating system will easily support a vast number of users at the same time, the only real limitation is the specification of the hardware. Each user is usually given their own home directory, this is the directory you will be placed in when you first log in, it can be used to store your own work. Mutli-Tasking The UNIX O/S is a true multi-tasking operating system, in that it can process more than one operation at the same time. This is obvious if there are many different users using the system at the same time, but not so obvious when only a single user is logged in. A single user can execute literally hundreds of different processes all at once. Ownership & Permissions The concept of Mutli-User leads us to Ownership & Permissions. If there are to be many users working on many files at the same time, then there needs to be a method of controlling who can do what. Files and directories have ownership, meaning which user actually owns the files, and permissions determine who can edit, view and run these files. We cover commands used to alter ownership and permissions later. The Command Line The command line is where you enter UNIX commands. Everything UNIX can do can be done from the command line. There are GUI‟s that work on UNIX, such as X-Windows, these are very similar to MS Windows but you will probably have no need for them in your environment. Shells When you enter UNIX commands at the command line, you are entering into a UNIX shell. Shell can be thought of as the old DOS Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 10

Training Guide Unix for Beginners

www.caosys.com [email protected]

prompt or any system that is command text driven. Shells provide an interface to the UNIX operating system as well as providing a whole host of other facilities, from allowing you to customise your environment, to providing very powerful scripting tools. There are many Shells available but most provide very similar environments. The most common flavours are:-

 sh – The Bourne Shell. The most common shell that is available on all versions of UNIX. Provides simple script tools.  csh – The C-Shell (pronounced seashell). As you might have guessed takes it‟s name from the C programming language. It is very similar to Bourne but not entirely compatible. It provides more advanced scripting tools.  ksh – The Korn Shell. Named after its author, the Korn shell is a backward compatible extension to Bourne Shell. It provides many more facilities than Bourne and has a more advanced scripting language. Input & Output (I/O) The concept of input/output on UNIX is very simple and absolutely essential. I/O is in effect the method in which programs communicate. Input can be thought of as a stream of data being passed to a process, the process will act upon this stream. Output is a stream of data that comes out of a program, output could be displayed on screen to the user, or even used as input to another program. Pipes and Redirection I/O can be processed in two ways, Redirection and Pipes. Redirection is the source or destination of a stream of data, for example, output from one program can be redirected to a file, and this is the destination. Input can be taken from a file using redirection, this is the source. A Pipe is a method of directing a stream of data into another program. UNIX Filesystem The UNIX filesystem is made up of what is called a root directory, this is a single top level directory, and it is a parent to ALL other directories. Other directories are subdirectories of root arranged in a sort of upside down tree shape. Root directory can contain files Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 11

Training Guide Unix for Beginners

www.caosys.com [email protected]

and directories, each directory in root can contain files and directories and so on. Filenames on UNIX are case-sensitive and can have almost any format, typically, a filename is given an extension, this is some text at the end of a filename following a . (dot). Typically extensions are used to signify the type of file. Paths – Absolute & Relative A path is essentially the address at which something (a file) can be found, for example, the ls command is typically located on the path (address) /usr/bin. This address says, from root directory /, go to the usr directory, then go to the bin subdirectory, this is an absolute path, meaning the directory is specified exactly. A relative path could be ../bin, which means, from our current directory, go back to the previous directory (..), then go to the bin directory. When you enter a UNIX command at the command line, UNIX searches down set of pre-defined paths, these paths are held in the environment variable called PATH*.

* Unless something is defined as a function within the shell, functions are beyond the scope of this course.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 12

Training Guide Unix for Beginners

www.caosys.com [email protected]

Basic UNIX Commands The course covers some of the most commonly used commands, they can be categorised into the following categories: Logging In/Out – Very simple, how to actually log into UNIX and how to get out.  File Management – Here we cover all basic file commands, from copying files to changing file permissions. We also cover how commands process input and output and how you can pipe output from one command to another, we also cover redirection.  Searching – This section covers the various commands available for searching using UNIX.  Text Processing - Here we cover common commands used for processing text.  vi – A quick tutorial on using the „vi‟ editor  Customising Your Environment – We learn how to customise your environment. We will perform simple exercises with each of the above topics.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 13

Training Guide Unix for Beginners

www.caosys.com [email protected]

Logging In/Out Logging in…..  First, you need to be given a login username and password (System Administrator will do this)  Open a UNIX session (if using a PC)  Enter username and password Logging Out….  From the UNIX prompt, type, logout  or press D

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 14

Training Guide Unix for Beginners

www.caosys.com [email protected]

Lab 1 1 2

Using the username and password provided, try logging in Log out of the system using both methods

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 15

Training Guide Unix for Beginners

www.caosys.com [email protected]

Getting Help UNIX provides a very comprehensive online help system. The help system comes in the form of manual pages for almost every command available. You access online manual pages using the man command.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 16

Training Guide Unix for Beginners

www.caosys.com [email protected]

Getting Help UNIX provides a very comprehensive online help system. The help system comes in the form of manual pages for almost every command available. You access online manual pages using the man command. The man command is very easy to use, simply type man followed by the command your want help on, for example, to get the online manual for the man command itself, type, man man

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 17

Training Guide Unix for Beginners

www.caosys.com [email protected]

File Management File management covers many things: Listing Files and Directories  Navigating The Filesystem  Copying, Moving & Deleting Files  Permissions & Ownership  Viewing File Contents  Printing  A Little more on Pipes and Redirection

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 18

Training Guide Unix for Beginners

www.caosys.com [email protected]

File Management At the very least, you should be familiar with how to navigate around directories and list or view files. The following topics are covered:Listing Files and Directories Here we cover how to list the contents of a directory. Navigating The Filesystem We cover commands used to help you navigate around the UNIX filesystem. Copying, Moving & Removing Files Commands related to the creation and deletion of files and directories are covered here. Permissions & Ownership How to change ownership and permissions is covered here. Viewing File Contents Several commands are available to allow the user to view the contents of a file or portions of it.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 19

Training Guide Unix for Beginners

www.caosys.com [email protected]

Listing Files & Directories So, you have logged in, and you want to know what files are in your current directory, you use the ls command. Using ls, you can: List files in a specified directory  View file/directory details such as permissions, owner, size, date last modified  Sorting output  Recursively list files and directories  Many more …

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 20

Training Guide Unix for Beginners

www.caosys.com [email protected]

Listing Files and Directories You can list the contents of any directory (for which you have permission) using the ls command. ls is very powerful and its full functionality is beyond the scope of this course. Some common uses follow …

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 21

Training Guide Unix for Beginners

www.caosys.com [email protected]

List Files in a Specified Directory In its most basic form, ls will list files and directories found in a specified directory, its syntax is:Syntax ls Examples List files with a filename beginning with mywork and ending in .dat, in the current directory. ls mywork*.dat List all files ending in .spl in the directory above your current directory. ls ../*.spl List all files in the directory /usr/users/user10 with it in the filename. ls /usr/users/user10/*it*

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 22

Training Guide Unix for Beginners

www.caosys.com [email protected]

View File Details You can view file details using ls such as, who owns the file, the file size, when it was last modifed,..etc. ls has many options available, -l is a common one, it means long listing. Examples Give a long listing of all files beginning with tmp, ls –l tmp* this will give output similar to the following:drwxrwxr—x drwxrwxr—x -rwxrwxr—x -rwxrwxr—x

root root root root

512 512 4013 1678

Dec Dec Dec Dec

17 15 17 17

16:01 09:37 16:01 16:01

tmpfile101 tmpdata4 tmp.dat tmp10124.spl

The first column of gibberish is the file type and permission, the first character indicating if it is a file or directory, d for directory and – for file. The next nine characters are file permission, 3 permissions of read, write and execute for three permission types, owner, group and everyone else. These will be discussed later. Then comes the owner of the file, in this case, the owner is root. This is followed by the file size. Then the date last modified and time, and finally the filename itself.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 23

Training Guide Unix for Beginners

www.caosys.com [email protected]

Sorting Output The output from ls can be sorted:Examples Give a long list of all files beginning with z in the current directory sorted by the date the files were modified, with newest being the first. ls –lt z* You can reverse the sort order with the –r option, it will list all files sorted by the date modified with the oldest file being listed first. ls –ltr

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 24

Training Guide Unix for Beginners

www.caosys.com [email protected]

Navigating The File System How do you move around the UNIX filesystem? UNIX allows you do the following: Display Current Directory  Change Directory

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 25

Training Guide Unix for Beginners

www.caosys.com [email protected]

Navigating The Filesystem Moving around the UNIX filesystem is essential to using the system, there are two basic commands for filesystem navigation. Display The Current Directory The pwd command simply displays the current directory. Change The Current Directory You use the cd command to change the current directory. You can specify the directory as an absolute or relative path, for example cd /u01 will change directory to /u01 whereas cd u01 will go to a directory called u01, which is found within the current directory. You can specify the previous directory using .., for example, cd ../u01 will go up a directory and then to a directory called u01. Syntax cd Examples Change the current directory to /u01/app/applmgr cd /u01/app/applmgr A special feature of the cd command is the ability to jump back and forth between two directories, this is useful when working with directories with long path names, for example, lets say your current directory is /usr/users/home/myhome/yourhome and you change the directory to /etc/tmp/datout, you could type, cd – this command will toggle between the two directories.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 26

Training Guide Unix for Beginners

www.caosys.com [email protected]

Copying, Moving and Deleting Files Now you can list files and navigate around, how do you copy, move or remove files. There are four very simple and very powerful commands available, cp, mv, rm and rmdir.  Copying Files  Moving Files  Removing Files  Removing Directories

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 27

Training Guide Unix for Beginners

www.caosys.com [email protected]

Copying, Moving, Deleting Files You may often want to copy a file to another file, move a file, delete a file or directory, UNIX provides many commands for these kind of things. Copying Files You can copy a file using the cp command. cp allows you to copy one or more files from one directory to another, or to make another copy with a different name. Syntax cp [options] cp [options] Options -i Interactive overwrite. This prompts the user before overwriting any existing files. -p Preserve modification data/time -r Recursive copy, can be used to copy whole directory structures from one place to another Examples Create a new file called myfile2.dat, which is exactly the same as myfile1.dat cp myfile1.dat myfile2.dat Copy all files starting with qa and ending in .log to a directory called /usr/logs. The –i option ensures the user confirms any files the could be overwritten. cp –i qa*.log /usr/logs

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 28

Training Guide Unix for Beginners

www.caosys.com [email protected]

Moving Files Files and directories can be moved (renamed) using the mv command. Syntax mv [options] Options -- Use this option if one of the files to be moved starts with a – (there is no spaces between the –„s) -f Force move, suppress messages about restricted access modes -i Same as –i for cp Examples Move/rename a file called oldfile to newfile mv oldfile newfile Move all files beginning with z to a directory called /u01/datafiles mv z* /u01/datafiles Note After using the mv command, the original file will no longer exist.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 29

Training Guide Unix for Beginners

www.caosys.com [email protected]

Deleting Files & Directories Files can be removed using the rm command. Directories can be removed using the rmdir command. This is a very dangerous command, unlike Windows Os‟s, files are not moved to a trashcan for recovery later, once deleted, that is it. Syntax rm [options] rmdir [options] rm options -f Remove write-protected files without prompting -i Interactive mode, prompt before removing -r Be careful, this option will recursively remove a directory and all its contents including subdirectories rmdir options -P If, as a result of removing a directory, its parent directory becomes empty, then this directory will also be removed Examples Remove all files starting with tmp and prompt before removing rm –i tmp* Remove a directory call olddata and all of its contents rm –r olddata Remove an empty directory called myfiles rmdir myfiles

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 30

Training Guide Unix for Beginners

www.caosys.com [email protected]

Permissions & Ownership UNIX is a multiuser/multiprocess system, it needs a way of dealing with security of files and directories, this is implemented in the form of file permissions and ownership.  All files have 3 sets of permissions, owner, group and world  All files are owned by someone  All files belong to a group  All users belong to at least one group

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 31

Training Guide Unix for Beginners

www.caosys.com [email protected]

Permissions & Ownership UNIX is a multi-user/mutliprocess operating system, many users can be working with many files at the same time, there needs to be a way of ensuring files are secure and cannot be viewed, changed or executed by someone who is not supposed to. UNIX implements this security using file permissions and ownership. Ownership All files are owned by a particular user, this is usually the user that created the file. All files also belong to a group. A group is a method of grouping users and files into certain categories, for example, your system may have several groups; administrators, developers, users, guests,…etc; by assigning these groups to files and users, you can quickly ensure only the right people have access to the right files. Permissions File permissions say who can do what to a particular file. There are 3 basic permissions, read, write and execute. These permissions are given to a file in 3 groups, owner, group and world, these permission groups determine what the owner can do to a file, what other users in the group can do to the file and what everyone else can do. Permissions can be viewed using the ls command, they are the first 10 characters of the output. Commands There are a few simple commands that are used to maintain file permissions and ownership, these are chmod, chgrp and chown

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 32

Training Guide Unix for Beginners

www.caosys.com [email protected]

Changing Permissions – chmod The chmod command is used to change the permissions on a file providing you have permissions to do so. Syntax chmod [options] Options -R Recursively go through directories and change mode Mode The mode specifies what permission you want to add or remove from the file. It has two methods of use:Method 1 Method 1 involves providing the command with 3 things, Who, Opcode and Permission, Who u User g Group o Other a All Opcode + Add permission Remove permission = Assign a specific permission, this removes permissions not specified Permission r Read w Write x Execute Other permissions are available (see online help using the man command) You specify the Who followed by the Opcode and end with the permission, for example u+x, means add execute permission for user Method 2 Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 33

Training Guide Unix for Beginners

www.caosys.com [email protected]

You can also specify permission using a 3-digit number. The 1st digit represents owner permissions, the 2nd is for group permissions and the 3rd is for everyone else. The 3-digit number is calculated as follows:4 2 1

Read Write Execute

So, if you want to calculate permission as read, write and execute for the owner and read and execute for everyone else:User – Read, write and execute = 4 + 2 + 1 = 7 Everyone else – Read and execute = 4 + 1 = 5 then the 3-digit number would be 755. Examples Add execute permission to group for specified file chmod g+x myfile Remove read for everyone and also add execute for owner chmod +r o+x myfile Set permissions to everything for everyone chmod 777 myfile

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 34

Training Guide Unix for Beginners

www.caosys.com [email protected]

Changing Owner - chown You can change to owner of a file using the chown command, providing you have permission to do so. Syntax chown [options] Options -R Same as –R for chmod Examples Change owner of all files in current directory to user10 chown user10 *

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 35

Training Guide Unix for Beginners

www.caosys.com [email protected]

Changing Group - chgrp You can change the group of a file using the chgrp command, providing you have permission to do so. Syntax chgrp [options] Options -R Same as –R for chmod Examples Change group of all files in current directory to developer chgrp developer *

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 36

Training Guide Unix for Beginners

www.caosys.com [email protected]

Viewing File Contents Any file on the system, so long as its contents are of a viewable nature and you have permission to do so, can be view using a number of different command.  Catenating/viewing files  File Contents – A Page at a Time  Viewing portions of a file

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 37

Training Guide Unix for Beginners

www.caosys.com [email protected]

Viewing File Contents At some time or other you are going to want to view the contents of a file. UNIX provides several commands for this. Any file can be viewed so long as it is of a viewable nature (ie, plain ASCII) and you have permission to do so. Catenating/Viewing Files - cat A file can be easily viewed using the cat command. This command is also used for catenating many files together. The command takes filenames as parameters, catenates them together, one after the other and gives them back to you. Used with redirection you can easily create new files. Syntax cat Examples View the contents of myfile01 cat myfile01 View all files starting with data cat data* Create a new file containing the files ab1 and ab2 cat ab1 ab2 > newfile Append to newfile the contents of oldfile cat oldfile >> newfile

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 38

Training Guide Unix for Beginners

www.caosys.com [email protected]

Files Contents – A Page at a Time The cat command is very useful, but what if you want to view a file that is several screens in size,….you enter the cat command and whooosh, its gone, displayed so fast that you never even saw it, the answer is to use a pager, not the beeping kind, the UNIX kind. The two most common commands are more and pg. Using more The more command has many options and many built in commands, at any time when more is running you can press h to get help on how to use it. Press to get the next line or to get the next page. Syntax more [options] Options – There are many option available, here are a few -c Clear screen between pages -d Display user friendly prompt between pages -n Set pagesize to n lines Examples View the contents of myfile more myfile View the contents of myfile, clearing the screen between each page more –c myfile

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 39

Training Guide Unix for Beginners

www.caosys.com [email protected]

Using pg The pg command is very similar to the more command. Press to get next page or h for help. Syntax pg [options] Options - There are many option available, here are a few -c Same as –c for the more command -n Same as –n for the more command -pstr Set prompt to str between pages Examples View file myfile, setting prompt to „Page ‟ pg –p ‗Page %d‘ myfile

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 40

Training Guide Unix for Beginners

www.caosys.com [email protected]

Viewing Portions of a File UNIX provides a couple of useful commands for viewing only portions of a file. The most common are head and tail. head allows you to view the top portion and tail the end. Using head Syntax head [-n] Options -n Display n lines. Default is 10 Examples Display first 10 lines of myfile head myfile Display first 100 lines of myfile a page at a time. Here we use a UNIX pipe to send output from head to the more command. head –100 myfile | more

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 41

Training Guide Unix for Beginners

www.caosys.com [email protected]

Using tail Syntax tail [options] Options - There are many options available, here are a few -f Don‟t quit at end of file, follow file as it grows, this is very useful for viewing log files. -r Reverse output -nitem Display from the nth item in file (default is 10). Item can be l for lines (default), b for block or c for character. Examples View last 10 lines of myfile tail myfile View last 10 lines of myfile and continue to view as file grows tail –f myfile View last 5 characters of a variable called $string. Here we use a UNIX pipe to direct the output from the echo command to the tail command. echo $string | tail –5c

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 42

Training Guide Unix for Beginners

www.caosys.com [email protected]

Printing Printing on UNIX is very simple. You determine what you want to be printed and send it to the printer with one simple command.  Sending files to the printer  The print queue

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 43

Training Guide Unix for Beginners

www.caosys.com [email protected]

Printing At sometime or another you are going to want to print a file. This is done very easily using the lp command. When you submit a file for printing it is added to the print queue, the queue is used to help deal with the fact that many users may be printing at the same time on the same printer. Sending Files to the Printer You print using the lp command, it is very easy to use and has many options, below it is described in its most basic form. If you specify one of the files as „-„ then the lp command will print anything on the standard input. Syntax lp [options] Options -d printer -o option

This specifies the printer to be used Options can be nobanner to suppress printing the print banner, nofilebreak to suppress form feeds between files, there are many more options available.

Examples Print a file named myfile on the printer named IT_01 and suppress the banner. lp –d IT_01 –o nobanner myfile Print all files beginning with ab on the IT_01 printer. Here we use a UNIX pipe and the cat command. cat ab* | lp –d IT_01 –

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 44

Training Guide Unix for Beginners

www.caosys.com [email protected]

The Print Queue All files sent to print are put onto the print queue, this controls what is printed when and where. You can view the print queue using the lpstat command. Syntax lpstat [options] Option - There are many options available, here are a few -s Show summary information about all printers. -r Show if print scheduler is running -t Show everything Examples View status of print queue lpstat View entire print queue a page at a time. lpstat –t | more

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 45

Training Guide Unix for Beginners

www.caosys.com [email protected]

A Little more on Pipes and Redirection Redirection is a method of directing output from a program to a file or taking input from a file. Redirection is implemented as follows: command > file - Redirects output from command into file  command >> - file - Appends output from command into file  command < file – command takes input from file A Pipe is a method of sending output from one program to another program as input. It is implemented as follows: command1 | command2 – Pipe output from command1 into command2

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 46

Training Guide Unix for Beginners

www.caosys.com [email protected]

A Little more on Pipes and Redirection Redirection Redirection is a method of redirecting the output or input to, or from a file. Redirection is very useful and allows you to do things such as saving the results of a command to a file. Redirection is implemented with the following operators: command > file - Redirects output from command into file. If file does not exist then it will be created, otherwise it is overwritten.  command >> - file - Appends output from command into file. If the file does not exist it will be created, otherwise it will be appended to.  command < file – command takes input from file. You can mix the redirection operators, for example, you could redirect a file into a command and redirect the output of that command into a file, for example :spell < myfile > outfile

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 47

Training Guide Unix for Beginners

www.caosys.com [email protected]

Pipes Pipes allow you to redirect the output of one command into another command. It is implemented as follows: command1 | command2 – Pipe output from command1 into command2 This is very useful and can be used to link together several commands, for example, to list all files starting with a and pipe the output through the more command, ls –lt a* | more Pipes can be mixed with redirection too, for example, to create a file called myfile which contains the first 10 lines of a file called yourfile, cat yourfile | head > myfile

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 48

Training Guide Unix for Beginners

www.caosys.com [email protected]

Lab 2 Before you start, type cd lab2 1 2 3 4 5

6 7

8 9 10 11

List all files in the current directory whose name start with file. List all files in the current directory in reverse order Change your current directory to /usr and then back again Copy a file called testdata.spl to testdata.bak Copy all files the subdirectory called current, found in your home directory into a directory called previous, also found in your home. Rename the directory called previous to backup Remove read and write permissions to all files in your current directory, then get a long directory listing, then re-add permissions Display the contents of file01 and file02 and redirect the output to file03 View file03 one page at a time View the middle 5 lines of file04, this file is 15 lines long (tip, use tail and head together) List the first five files in your current directory

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 49

Training Guide Unix for Beginners

www.caosys.com [email protected]

Searching At some point you will want to find files on the filesystem when you don‟t actually know the files name or path. To enable you to do this you must first know something about the files you are looking for, this could be: Part of the filename  When the file was created  Some of the contents Once you have this information, you can use a couple of standard UNIX commands to help you find what you are looking for. The most common commands used for searching are: find  grep

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 50

Training Guide Unix for Beginners

www.caosys.com [email protected]

Searching At some point you will want to find files on the filesystem where you do not know the files name or which directory it is in. To enable you to do this you first need to know something about the file or files you are looking for, this could be: You know part of the filename, for example, you may know that the file you want starts with abc or ends with .dat.  You know when the file was created, for example, the file you are looking for may have been created on the 10th of April or it was created within the last week.  You know some of the contents of the file, for example, you may be searching for a text file that contains the words „Version 2.0‟ Once you have this information, you can use a couple of standard UNIX commands to help you find what you are looking for. The most commonly used commands are: find – You use the find command to find files by name or date created.  grep – You use the grep command to find files where you know some of the files contents

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 51

Training Guide Unix for Beginners

www.caosys.com [email protected]

Using find The find command is very useful for finding files based on things like portions of the filename, file age, file size,..etc. It can also be used to execute other UNIX commands on many files. It is a very complicated command with many options, I suggest you read the online help to get further information. Its basic use is described below. Syntax – For finding by filename find -name -print Examples To search for all files starting with pic, which are in the directory called /u01/data or a subdirectory thereof, find /u01/data –name pic* -print To search from the current directory for all files ending with .dat, find . –name *.dat –print Syntax – For find by file age find -ctime -print Example To find all files created or modified within the last 2 days (48 hours), find /u01/data –ctime 2 –print

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 52

Training Guide Unix for Beginners

www.caosys.com [email protected]

Using grep & Regular Expressions grep is a very powerful command for searching through the body of file for a particular word or pattern. grep comes in three flavours, grep, egrep and fgrep. The differences between each version is the complexity of the regular expression they can handle. A regular expression defines a pattern of text that can be used to search files, simple regular expressions are:* . [] ^ $

Match Match Match Match Match

anything a single character a range of characters the start of a line the end of a line

e.g. e.g. e.g. e.g. e.g.

*ab fr.d a[c-f] ^abc $xyz

Syntax – (See online help for egrep and fgrep) grep [options] Options – Many options are available, but here are a few -c Print only a count of matching lines -i Ignore upper and lower case, treat as same -v Print only lines that don‟t match -l Print only the filename if a match is found Examples To print all lines in a file called backup which contains the word success, grep success backup To count all lines starting with error, grep –c ―^error‖ backup To find all files which have a line starting with A, B or C, grep ―^[A-C]‖ *

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 53

Training Guide Unix for Beginners

www.caosys.com [email protected]

Lab 3 Before you start, type cd ../lab3 1 2 3

Using find, find all files in a directory called /usr/bin that start with the letter l. Using grep, find all files in your current directory that contain the word unix. List all lines that do not contain the word ‗the‘ but do contain the word ‗string‘ in a file called grepit.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 54

Training Guide Unix for Beginners

www.caosys.com [email protected]

Text & File Utilities UNIX provides many commands for working with text files. We will briefly cover some of the most common commands: File Statistics with wc  Sorting using sort  Comparing using diff  Text Processing with awk

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 55

Training Guide Unix for Beginners

www.caosys.com [email protected]

Text & File Utilities UNIX provides many commands for working with text files. We will briefly cover some of the most common commands:   

Obtain quick file statistics using wc Sorting files and output from other commands using sort Comparing files using diff Text processing using awk

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 56

Training Guide Unix for Beginners

www.caosys.com [email protected]

File Statistics Using wc The wc command allows you to get quick and basic statistics about a file or many files. wc produces the following statistics: Number of lines in file  Number of characters in file  Number of words in a file

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 57

Training Guide Unix for Beginners

www.caosys.com [email protected]

File Statistics using wc The wc command can be used to get quick statistics on a file. The statistics wc gives are number of lines, number of character and number of words. Syntax wc [options] Options -c Printer character count only -l Print line count only -w Print word count only Examples Count all lines in a file called backup wc –l backup Display number of files in current directory, ls | wc –l Count number of users logged in, who | wc –l Count number of words in a few text files, wc –w textfile.[123]

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 58

Training Guide Unix for Beginners

www.caosys.com [email protected]

Sorting Using sort You can sort information using the sort command. You can easily sort the contents of a file or the output of another command. sort has the following features: Reverse sort order  Sort by a specific field  Show only unique lines  Take input from a file or from output of another command

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 59

Training Guide Unix for Beginners

www.caosys.com [email protected]

Sorting using sort Sorting can be done very easily using the sort command, it is very flexible and very fast. Syntax sort [options] Options -u Only show unique lines -r Reverse sort order +n Skip n fields before sorting -tc Specify field separator as c Examples List all files sorted by number of lines, highest first wc –l * | sort –r Sort the contents of a file by the fourth field, field separator is :, sort +3 –t: myfile

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 60

Training Guide Unix for Beginners

www.caosys.com [email protected]

Comparing Files with diff You can easily compare the contents of two files using the diff command.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 61

Training Guide Unix for Beginners

www.caosys.com [email protected]

Comparing Files with diff Lets say you have two files that are very large and almost the same, how do you find the differences other than looking through each file,… you use the diff command. diff basically checks each file and displays output which tells you the differences. Syntax diff Example Create a file called diffs containing the differences between file01 and file02, diff file01 file02 > diffs

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 62

Training Guide Unix for Beginners

www.caosys.com [email protected]

Text Processing using the Almighty awk awk is a very powerful text-processing tool. It is almost a full language in itself, it is very similar to the C language in syntax. awk can do almost anything to its input, from reformatting it, to producing summaries of the information passed to it.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 63

Training Guide Unix for Beginners

www.caosys.com [email protected]

Text Processing using the Almighty awk awk is a very powerful command, it is used for processing text files. awk is really a full programming language in its own right and can do many things. I suggest you read the online help on awk or even read one of the many books available on awk programming. Here we cover only very basic use:Syntax awk [program] Program This is the actual awk script itself, this can be entered directly on the command line or can be taken from a file. As mentioned awk is a full (almost) programming language so is well beyond the scope of this course. We will cover two things that awk can do: print things  add things up program has a very complicated syntax, in its simplest form we can do, {print $0} this will print the entire line of input, whereas {print $1} will print the first field. Program has three basic sections, BEGIN, BODY and END. BEGIN is executed before any input is processed, BODY is processed for each line of input and END is processed after all input has been processed, for example, BEGIN{print ―Start‖} {print ―Body ― $0} END{print ―End‖} if the above script was executed on a file containing:line 1 line 2 then the output would be:Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 64

Training Guide Unix for Beginners

www.caosys.com [email protected]

Start Body line 1 Body line 2 End. Okay, lets say we had a file containing lots of numbers, how could we use awk to add all these numbers up:BEGIN{total=0} {total+=$0+0} END{print ―Total=‖ total} the above starts by declaring a variable, total to be 0, then it adds the the contents of each line of the file (the +0 is used to ensure we are working with numeric values), finally, at the end it by printing the variable total, easy!!! Examples To print fields 3 and 5 from a file called myfile, awk ‗{print $5 $5}‘ myfile To print a file containing several columns, the last column should be totalled, here we use the NF built-in variable, this variable contains the number of fields on the current line, awk ‗BEGIN {total=0} {print $0;total+=$NF} END {print ―total=‖total}‘ myfile

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 65

Training Guide Unix for Beginners

www.caosys.com [email protected]

Lab 4 Before you start, type, cd ../lab4 1

Print the total number of lines in all files in your current directory. (tip, use wc and tail)

2

Create a file containing the filename and size of all files in your home directory. (tip, use ls and awk)

3

Using diff, create a file called diff3 which contains the differences between diff1 and diff2, then view the differences.

4

Print the total size of all the files in your current directory. (tip, use wc and tail)

5

Display the first and last columns from the output produced by ls –l.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 66

Training Guide

www.caosys.com [email protected]

Unix for Beginners

Processes Everything you have done so far on this course and everything you will ever do will create processes on the system. Every command creates a process that UNIX takes care of until the process is finished or killed. UNIX allows you to do the following with processes: View details about current processes  Run processes in the background  Kill processes  Schedule processes

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 67

Training Guide Unix for Beginners

www.caosys.com [email protected]

Processes Everything you have done so far on this course and everything you will ever do on UNIX will create processes. Every command creates a process that UNIX takes care of until either the process finishes naturally or it is killed. UNIX allows you to do many things with processes, here are a few: View details about current processes. You can use the ps command to view details about any process on the system  Run processes in the background. You can make almost any process you own run in the background, meaning that there is no interaction from the user and no direct output.  You can kill a process you own using the kill command  You can schedule processes to start overnight using cron.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 68

Training Guide Unix for Beginners

www.caosys.com [email protected]

Viewing Current Processes You can view information about any process on the system using the ps command. Using ps you can view process information about a specific process, processes belonging to a specific user or even all processes. The information ps provides varies in detail depending on the options you specify when running the command.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 69

Training Guide Unix for Beginners

www.caosys.com [email protected]

Viewing Current Processes If you want to see what processes are running on the system or view what your own processes are doing, you do this with the ps command. ps has many options, a few are covered here:Syntax ps [options] Options -e -uuser -f

List all processes List processes for specified user Produce a full list of all information about each process

Examples The get a full process list of all processes, ps –ef Display a process list for all processes owned by a user called fred, ps -ufred

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 70

Training Guide Unix for Beginners

www.caosys.com [email protected]

Running a Process in the Background UNIX allows you to run many processes at the same time. You could for instance, run a command that formats a large text file and still be working on another file using vi. This is achieved by running command in the background. You would typically want to run something in the background so as not to tie up your terminal.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 71

Training Guide Unix for Beginners

www.caosys.com [email protected]

Running a Process in the Background You may want to run a command that you know will take a very long time, you need to do it but you don‟t want to tie up your terminal, the answer is to run the command in the background. Putting a Command into the Background There are two basic ways to make a job run in the background:1 Start the command and then press +z, this will suspend the command, then type bg, this will put it into the background 2 On the command line, add the & character to the end of the line, for example, process & Either way is okay, but be aware that if the job is running in the background then your command should not print anything to the screen or ask for user interaction. Viewing Background Processes You can view a process running in the background in two ways:1 Use the ps command 2 Use the jobs command

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 72

Training Guide Unix for Beginners

www.caosys.com [email protected]

Killing a Process All processes must end at some point, unless they are broken of course. A process can end in two ways, naturally, ie. letting it finish on its own or by using the kill command. The kill command allows you to kill (stop) any process that is owned by you (unless you are a superuser such as root).

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 73

Training Guide Unix for Beginners

www.caosys.com [email protected]

Killing a Process You will at some point start a process that you know will not finish, will take a long time or will do something it shouldn‟t, if this does happen then you will need to stop, or kill the process. You do this with the kill command. I suggest you read the online help for information about kill, it is a very dangerous command in the wrong hands. You, as a normal user can only kill processes you own. Syntax Kill kill Process ID This is the process ID given to all processes, you can find the process ID using the ps command. Job Number The job number is a number given to all processes that you manually execute in the background. Examples Kill a process using process ID, kill 12345 Kill a process using the job number, kill %1 Notes kill will try to stop a job in a graceful way, if this fails then you will have to use a more severe version of kill, kill –9 12345 the –9 part forces kill to kill the specified process. Be aware that if you kill a process that has child processes then these child processes will also be killed, for example, when you log in, you create a process, if you then start another process and kill your login process, all process you started since login will be killed.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 74

Training Guide Unix for Beginners

www.caosys.com [email protected]

You can get around this problem with the nohup prefix to a command, nohup &

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 75

Training Guide Unix for Beginners

www.caosys.com [email protected]

Scheduling a Process UNIX allows you to schedule a process to execute at a specific time. This is done using the cron facility. cron easily allows you to schedule jobs to run a regular intervals. This facility is typically used to perform housekeeping and backup tasks that need to be run overnight or at weekends.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 76

Training Guide Unix for Beginners

www.caosys.com [email protected]

Scheduling a Process You may want to execute certain commands at a time when noone will be there to start the command, or you may want to regularly run a command that does some kind of housekeeping, you do this using the cron facility. Using cron is beyond the scope of this course and is usually done by administrators, so we will just cover what cron actually does rather than learn how to use it. cron is a process that is constantly running on a UNIX system, it checks a file called crontab for the commands it should run. The crontab file is basically a simple text file that specifies what should be executed and at what interval. You can create your own personal crontab file, which can be added to the main crontab file.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 77

Training Guide Unix for Beginners

www.caosys.com [email protected]

vi – the Text Editor Upto this point we have worked with files without actually having manually created them, to create or edit text files on a UNIX system you will need some kind of editor, there are many to choose from but the most widely used is probably vi, so we will only cover that one. vi has the following features (amongst many others): Full screen based text editor  Very non user friendly  Very powerful  Very annoying to start with Hmmm, you may well say “Well why use it then?”, there are a few good answers to that: Not much choice (ed, ex, emacs are alternatives)  It is very good for programmers and is immensely powerful, although it takes time to realise this. We will cover only the basics of using vi, you will pick up skills in this area over time.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 78

Training Guide Unix for Beginners

www.caosys.com [email protected]

vi – The Text Editor Upto this point we have worked with files without having to manually create them. To create or edit text files you need a text editor. There are many editors available on UNIX but the most widely used is probably vi (some pronounce it as vee-eye and some just vye, but who cares anyway). This course will only cover vi and the very basics of it, as full editor training is beyond the scope of this course. vi has the following features: It is a full-screen text editor, whereas something like ed, is line based.  It is not user friendly at all  It is very powerful, once you are proficient as using vi you will wonder how you ever managed without it (maybe)  It can be very annoying to use to start with. After reading the above you may well say “Well why use the thing then?”, there are a few good answers to that: You don‟t have much choice, well not on a text based UNIX box anyway, there are other editor available such as ed, ex and emacs.  vi is very good for programmers and is immensely powerful, but it may well take you some time to realise this.  vi is a universal editor, you will find it on every UNIX machine you ever work on.  vi has a very small footprint (executable size) and is very efficient therefore runs very quickly.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 79

Training Guide Unix for Beginners

www.caosys.com [email protected]

Using vi First of all, you need to run vi, do this as follows:vi if exists then vi will open the file, otherwise you will see a blank screen. Okay, lets say you opened a file that didn‟t exist, so, you see nothing on screen at all,… very odd and very confusing. The first thing you need to understand is that vi has two modes of operation:1 command mode – this is used to enter vi commands, from this mode you can do things like, saving the file, performing mass changes or even access the OS 2 insert mode – this is the mode where you actually enter text into the file. Below is a list of some of most commonly used commands:Mode Command Insert/Command Command

Key Action l, h, j, k Move cursor around Cursor keys and Move cursor around l,h,j,k x Delete character dd Delete line a Change to insert mode and append after cursor A Change to insert mode and append to end of line i Change to insert mode and insert at cursor position I Change to insert mode and insert a start on line :w Save file as :w Save file :q Quit :q! Quit without saving nyy Yank n lines into buffer

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 80

Training Guide Unix for Beginners

www.caosys.com [email protected]

p P Insert

Esc key Backspace and Del

Open new line below cursor and inset contents of buffer Open new line above cursor and insert contents of buffer Change from insert to command mode Delete character before cursor

To explain how to use vi fully would take an entire book, I suggest for now you look at commands above and have a go with it, also, read the online help too. You will eventually become good at using vi but it takes a little patience.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 81

Training Guide Unix for Beginners

www.caosys.com [email protected]

Lab 5 As a quick practice on using vi, following the instruction below. 1 2 3 4 5 6 7 8

From the command prompt type vi firstvifile Once in vi press the i key Enter some text and press Enter some more text Press the key Press : then w Press : then q From the command prompt type cat firstvifile

Can you see your file?

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 82

Training Guide Unix for Beginners

www.caosys.com [email protected]

Working with the Shell When using the shell* on UNIX there are several things you can do to make life easier: Customise your Environment  Filename Completion  Command History

* This course assumes you are using Korn Shell

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 83

Training Guide Unix for Beginners

www.caosys.com [email protected]

Working with the Shell When working with the shell on a UNIX machine, there are a number of things you can do to make life easier: Customise Your Environment – you can set the command prompt to almost anything you like, you can create command alias‟s as shortcuts for often used commands  Filename Completion – when entering filenames in the shell you can use filename completion to help save typing.  Command History - is another facility that saves typing.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 84

Training Guide Unix for Beginners

www.caosys.com [email protected]

Changing Your Command Prompt UNIX allows you to change your command prompt to almost anything you like. You could, for example, set it to one or many of the following: Your login name  The hostname (machine name)  The current directory

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 85

Training Guide Unix for Beginners

www.caosys.com [email protected]

Changing the Command Prompt You can change your command prompt to be almost anything you like, you may for instance want the prompt to indicate who you are logged in as or even show the current directory, here‟s how you do it. When you log into the system a few things happen: 1 Global system login file is executed 2 Personal login file is executed 3 Personal shell settings file is executed We are interested in points 2 and 3 above. Your personal login file is usually called .profile (.login on csh), this file contains various commands used to set-up your environment, such as setting the path or running command. You can add to this file anything you want, such as a custom variable to hold information that you want to be available to you, while you are logged in. The other file is your personal settings file, on Korn shell this is called .kshrc. It is in this file that you usually make change to your environment such as changing the prompt or setting up aliases. The command prompt is set by changing the value of a special environment variable called PS1, for example, if your .kshrc contains the following, PS1=‘mysession>‘ then the command prompt would say mysession>. To set the command prompt to your current directory do the following:PS1=‘$PWD >‘ If your current directory was /u01/data then your prompt would be, /u01/data >

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 86

Training Guide Unix for Beginners

www.caosys.com [email protected]

Command Alias’s You may find that while using UNIX you are repeating yourself many times. There maybe some lengthy commands you use that you use very often. UNIX allows you to set up what are called alias‟s to save you having to re-type commonly used command over and over. You could, for example, set up alias‟s to do the following: Give a directory listing a page at a time  Change the current directory to a commonly used directory  Ensure that when using the rm command, that you are always asked for confirmation before deleting.  You may want to simply make some UNIX commands have more user-friendly names.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 87

Training Guide Unix for Beginners

www.caosys.com [email protected]

Setting up a Command Alias A command alias is simply a shortcut you use to a command or sequence of commands, for example, if you use the ls command very frequently with specific options, ls –l | more you could create an alias for this, which would save typing. To create an alias you simply make an entry in your .kshrc file using the alias command, alias ll=‘ls –l | more‘ and that‟s it, now whenever you type ll, shell will execute ls –l | more. You can alias almost anything. Here are a few more examples, Clear the screen, alias cls=clear List directories, alias dir=‘ls –lt | grep ‗^d‘ | more‘ Replace grep for a more meaningful name, alias search=grep You can clear an alias with the unalias command, for example, unalias search

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 88

Training Guide Unix for Beginners

www.caosys.com [email protected]

Filename Completion Filename Completion is another typing saver. Basically, UNIX works out the file or path name you want before you have finished typing. You must provide enough of the name to avoid ambiguity and then press a couple of keys, UNIX will then attempt to finish of the file or pathname for you.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 89

Training Guide Unix for Beginners

www.caosys.com [email protected]

Filename Completion Filename Completion is a very neat little trick you can use when entering filenames or pathnames at the command prompt. Basically, while typing in a filename or pathname, once you have typed enough of the filename to uniquely identify it, if you press \, the shell will complete the filename for you. For example, if the current directory contain the following files, thisfile.dat thatfile.dat theotherfile.dat and you type, ls –l this and press \, the shell will complete the filename for you and the command line will now look like this, ls –l thisfile.dat

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 90

Training Guide Unix for Beginners

www.caosys.com [email protected]

Command History Every command you enter at the command prompt is record by the shell to a command history file. UNIX allows you to move back through this history file and re-execute or edit previously entered command. Once you get used to using command history you will wonder how you ever managed without it.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 91

Training Guide Unix for Beginners

www.caosys.com [email protected]

Command History When working with UNIX you will find that you will at some point need to repeat a command you entered earlier, some of these commands could be very long. You will soon discover how annoying having to re-type command is, shell provides a facility called Command History which allows you to recall, edit and execute previously entered commands. Full use of command history is beyond the scope of this course so will shall cover only the very basics. All commands you enter are saved into a file specified by the HISTFILE environment variable. There are several key combinations you can use to allow you to go back through this file while at the command prompt. For example, lets say you have entered the following commands, ls clear vi myfile ls –lt | more now, you want to use the vi command again, you could just retype it, but you could also type, r vi this will execute the last command you entered starting with vi, or you could type, /vi this will search through all commands entered looking for the string vi, it allows you to use the j and k key to scroll through all matching commands. Editing commands entered at the command prompt is the same as editing text in the vi editor. At anytime, while in command mode, if you press v, vi will open with your command there, ready to edit, leaving vi will execute the command.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 92

Training Guide Unix for Beginners

www.caosys.com [email protected]

Lab 6 Before you start, type, cd ../lab2 1 2 3 4 5

Change your command prompt to be “Enter Command Here ->”. Create a command alias for vi called edit Create an alias call newlist, which lists the newest 5 files in the current directory Have a little practice with filename completion Have a little practice using command history

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 93

Training Guide Unix for Beginners

www.caosys.com [email protected]

Section One - Summary In summary then, we can see that UNIX is a very simple operating system to use with many features. We briefly covered the following: Basic Concepts – multi-user, multi-tasking, ownership & permissions, command Line, shells, input & output, pipes & redirection, filesystem and paths  Logging in and out of UNIX  File Management – files & directories, navigation, copying/moving/deleting, viewing files, printing  Searching – Searching using partial filenames and file contents  Text & File Utilities – File Statistics, sorting, comparing, text Processing  Processes – Viewing, putting in the background, killing and scheduling  vi – Using the text editor  Working with the Shell – Customising, typing savers

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 94

Training Guide Unix for Beginners

www.caosys.com [email protected]

What is next? In section two we cover basic shell programming (or sometimes referred to as scripting). This section gives an overview of how to create very basic shell scripts.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 95

Training Guide Unix for Beginners

www.caosys.com [email protected]

Section Two Basic Shell Programming

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 96

Training Guide Unix for Beginners

www.caosys.com [email protected]

Basic Shell Programming Topics covered in this section: What is a Shell Program/Script?  Executing a Shell Script  Shell Variables  Command Line Parameters  Basic String Operators  Arithmetic Expressions  Decisions Decisions  Loop The Loop

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 97

Training Guide Unix for Beginners

www.caosys.com [email protected]

Basic Shell Programming This section of the course is designed to give the delegate enough knowledge to enable them to produce simple shell scripts. Shell scripting is a complex topic and would require a course devoted to it alone to cover most of the features of shell programming, therefore, in this course we only cover the very basics. It should be enough to give you a feel for what can be achieved using shell. This section covers the following topics:  What is a Shell Program/Script – Here we briefly discuss what a shell script actually is.  Executing a Shell Script – Here we learn how to execute/run a shell script.  Shell Variables – We learn about shell variables and how you implement them in a script  Command Line Parameters – Here we briefly cover passing parameters to a shell script from the command line  Basic String Operators – We quickly cover some basic string operators  Decisions Decisions – This section covers the decision making aspects of the language  Loop The Loop – You will learn how to create simple loops using shell

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 98

Training Guide Unix for Beginners

www.caosys.com [email protected]

What is a Shell Program/Script? Okay, so you‟ve heard someone say, “I‟ve got a shell script that does that”. What are they talking about? A Shell script is basically a file, which contains several UNIX commands, you can then execute this file as if it were a UNIX command. Another way of thinking of it is that a Shell Script is a file containing what you would normally enter at the command line, only this time what you typed is saved in a file for later use. Shell scripts can be very simple two-liners or very large and complex scripts, for example, the Source Control System you will be using is written entirely with Shell (and a little awk for good measure). Shell Scripts might not have the elegance and consistency of a conventional programming language but they are comparable in power and flexibility.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 99

Training Guide Unix for Beginners

www.caosys.com [email protected]

What is a Shell Program/Script? In simple terms, a Shell Script is a file that contains several standard UNIX commands that you can execute as if the file were a command itself. Shell Scripts can be very simple two-liners or large and complex systems, for example, you may have a shell script that makes a backup of crucial files or a series of Shell Scripts that implement a Source Control System. (The source control system you will be using is written entirely in shell). Shell Scripts may not have the elegance or consistency of a conventional programming language but they are comparable in power and flexibility. A Shell script is usually created using a text editor such as vi but you could also dynamically create a shell script using another program. Creating Shell Scripts on the fly is beyond the scope of this course but it is something you may need to do in the future.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 100

Training Guide Unix for Beginners

www.caosys.com [email protected]

Executing/Running a Shell Script Once you have created a shell script you will want to execute/run it. There are two ways you can run a shell script: Using the dot (.) method – this causes the script to run as if you had typed the commands in directly at the command prompt  Using the name only method – this causes the shell script to run in what is know as a subshell

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 101

Training Guide Unix for Beginners

www.caosys.com [email protected]

Executing/Running a Shell Script Once you have created a Shell Script, you will probably want to run it. There are two ways in which a Shell Script can be run:  Using the dot (.) method – this causes the script to literally execute as if the commands it contains were actually entered at the command prompt. This method runs the script in the same process as the calling process, in other words, no sub-process is started to execute the shell script.  Using the name only method – for this method to work you must first ensure the file has been given execute permission. NOTE At the top of all your shell scripts you should specify in what shell the script should run, for korn shell, the first line of your script should read #!/bin/ksh Example using Dot(.) Method Lets say you have a Shell Script called process, from the command line simply type, . process this will run the shell script as if the commands were entered at the command line. One Thing to be aware of is that any environment variables which are set in a script run in this manner will be visible either until you log out or unset them. Example using Name only Method Again, lets assume you have a Shell Script called process, to run it using the name only method, simply enter the shell script name, process this method will only work if you have execute permission set on the file, add this permission like this, chmod +x process

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 102

Training Guide Unix for Beginners

www.caosys.com [email protected]

Lab 7 Before you start, type cd ../lab7 1 2

In your current directory there is a shell script called runme, run this script using both methods described Using vi, create a shell script called firstshell which contains the following:print ―Welcome!‖ Now run the script using the name only method.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 103

Training Guide Unix for Beginners

www.caosys.com [email protected]

Shell Variables As with all programming languages, a major part of the shells functionality relates to shell variables. To recap, a variable is simply a named place to store data. Within the shell, a variables value is obtained by preceding its name with a dollar sign ($), for example, print $myvar When setting the value of a variable, you do not precede it with a dollar, for example, myvar=craig

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 104

Training Guide Unix for Beginners

www.caosys.com [email protected]

Shell Variables As with all programming languages, a major part of the shells functionality relates to shell variables. To quickly recap, a variable is a named place to store data. Shell variables are no different to any other variable from any other language, they allow you to store various types of data in a place you specify. You can obtain the value of a shell variable by preceding its name with a dollar sign ($), for example, print $myvar When setting a variables value, do not use the dollar sign, for example, myvar=craig The print command is useful for displaying output to the user, you can also use the echo command which is almost the same but print is a more user-friendly command. You can easily set a variable to be the output of almost any command by enclosing the command in back single quotes, for example, curdir=`pwd` this command would set the variable curdir to the current directory path.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 105

Training Guide Unix for Beginners

www.caosys.com [email protected]

Lab 8 Before you start, type cd ../lab8 1

Create a shell script called vars, which sets a variable to the value 10, then copy this variable into another variable and display the new variable.

2

What is wrong with the following script? print “A Small Shell Script” $myvar = 10 print “The value of myvar is myvar”

3

Using a shell variable, display the first file found in your current directory. (tip, ls & head will be needed)

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 106

Training Guide Unix for Beginners

www.caosys.com [email protected]

Command Line Parameters Parameters can be passed to a shell script very easily from the command line. All that you need to do is specify your parameters after the command name, for example, runme param1 param2 The above code executes a shell script called runme and passes it two parameters, param1 and param2. You can then read these parameters from within the script using $1 and $2 shell variables.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 107

Training Guide Unix for Beginners

www.caosys.com [email protected]

Command Line Parameters Parameters can be passed to a shell script from the command line very easily. All you need to do is to specify your parameters after the command name, for example, runme param1 param2 the above code will run a script called runme and pass it two parameters, param1 and param2. You can then read these parameters inside the script using $1 and $1 shell variables, for example, at the command line you enter, runme bob dave the script runme contains, print ―parameter 1 = $1‖ print ―parameter 2 = $2‖ you will get the following output, parameter 1 = bob parameter 2 = dave You can read all parameters at once using, print ―all parameters = $*‖ print ―all parameters = $@‖

or

You can count parameters using, print ―number of parameters = $#‖

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 108

Training Guide Unix for Beginners

www.caosys.com [email protected]

Lab 9 1

Create a shell script called params, which accepts upto 3 parameters, print all the parameters and displays a count of the number of parameters

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 109

Training Guide Unix for Beginners

www.caosys.com [email protected]

Basic String Operators In shell programming, string operators are very powerful, they can be used to do things like the following: Set default values  Pattern match and cut strings  Get a variables length

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 110

Training Guide Unix for Beginners

www.caosys.com [email protected]

Basic String Operators In shell programming, string operators are very powerful, they can be used to do things like the following: Set default values – You can ensure variables always have a default value and also test for existence  Pattern match and cut strings – You can search for patterns in a string and cut the string into pieces  Get a variables length There are many more string operators available, I suggest reading the online help for more information.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 111

Training Guide Unix for Beginners

www.caosys.com [email protected]

Set Default Values You can ensure a variable has a default value using string operators very easily:myvar=${count:-0} this would set myvar to the value of count, if count is not set the myvar would equal 0, count would be left unset. You could also ensure count is also set to 0 by using :myvar=${count:=0} You can test for a variable existence and error if it does not exist very easily as follows:myvar=${count?‘missing variable‘} this would test the existence of count, if it does not exist then the message ―count : missing variable‖ would be displayed and the shell script would end.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 112

Training Guide Unix for Beginners

www.caosys.com [email protected]

Set Default Values You can ensure a variable always has a value using string operators very easily using the following code: myvar=${count:-0} this would set myar to the value of count, if count does not exist then myvar would be set to 0, count would be left unset. You can also ensure that count is set using the following code: myvar=${count:=0} this would do the same as the previous code but it would also set the count variable to 0. You can also test for the existence of a variable in two ways using string operators, the first is to display an error message and stop processing if a variable does not exist: myvar=${count:?‖missing variable‖} the above code would print the following message:count : missing variable and exit if count does not exist. The second is to set a variable to something if another variable does not exist: varexist=${count:+‖true‖} the above code would set varexist to true if count exists.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 113

Training Guide Unix for Beginners

www.caosys.com [email protected]

Pattern Match and Cut Strings You can use string operators to search for patterns in a string and then cut the string into pieces. A common use for these operators is stripping parts of filenames and pathnames out.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 114

Training Guide Unix for Beginners

www.caosys.com [email protected]

Pattern Match and Cut Strings You can use string operators to search for patterns in a string and then cut the string into pieces. A common use for these operators is stripping parts of filenames and pathnames out. For example, lets say we have a variable called fullpath and it contains /usr/users/homes/myhome/long.file.name, you might want to cut out the path and just leave to filename or cut-out everything after the first dot (.) and leave the rest. Here is how you do it: To cut out the entire path and only leave to filename use the following code: myvar=${fullpath##/*/} the above code search through fullpath from the start, look for the pattern /*/, this is anything starting and ending with a slash (/), the ## symbol says, find a match and cut-out the longest match and leave the rest, so you would end up with, long.file.name you could also cut-out the shortest match using:myvar=${fullpath#/*/} this would leave:users/homes/myhome/long.file.name To do the same but start searching from the end of the string, use a % symbol in place of the #, for example, to cut off the extension .name you would use the following code:myvar=${fullpath%.*}

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 115

Training Guide Unix for Beginners

www.caosys.com [email protected]

Get a Variables Length There are a number of ways you can get the length of the contents of a variable, but the easiest is probably using a string operator. To get the length of a variable use the following code:length=${#varname}

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 116

Training Guide Unix for Beginners

www.caosys.com [email protected]

Get a Variables Length There are a number of ways you can get the length of a variable but the easiest method is to use a string operator. You get the length of a variable as follows: varname=thisvar length=${#varname} print $length the above code sets the varname variable to the value thisvar, the next line sets the variable length to the length of varname and the last line prints the length variable.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 117

Training Guide Unix for Beginners

www.caosys.com [email protected]

Lab 10 Before you start, type, cd ../lab10 1 2 3

Create a shell script that uses a string operator to set a variables default value Create a shell script that accepts two parameters, default the second to the first if it is not entered, print both parameters Create a shell script that accepts two parameters, the first is a string and the second is another string to be cut from start of the first, print the remaining string after the cut.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 118

Training Guide Unix for Beginners

www.caosys.com [email protected]

Arithmetic Expressions You can perform arithmetic in the shell using the let command. Arithmetic is done using one or more integer expressions. Expressions consist of numbers, operators and shell variables. Using the let command you can perform:  Addition  Subtraction  Multiplication  Division  Remainder  Bitwise operations

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 119

Training Guide Unix for Beginners

www.caosys.com [email protected]

Arithmetic Expressions You can perform arithmetic in the shell using the let command. Arithmetic is done using one or more integer expressions. Expressions consist of numbers, operators and shell variables. Using the let command you can perform:      

Addition Subtraction Multiplication Division Remainder Bitwise Operators (beyond the scope of this course)

If the expression contains any white space then you must surround the expression in quotes unless you use the (( )) syntax, see example below. Some Examples let ―i = i + 1‖ let i=i+1 (( i = i + 1 )) all the above examples do exactly the same thing.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 120

Training Guide Unix for Beginners

www.caosys.com [email protected]

Lab 11 Before you start, type, cd ../lab11 1

Create a script which adds two numbers and prints the result.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 121

Training Guide Unix for Beginners

www.caosys.com [email protected]

Decisions Decisions At some point you will want to make a decision within a shell script, this decision may be based upon parameters passed to your shell script or based on the number of files in a particular directory, in any case you will need to use one of the shells flow control statements: The if statement  The case statement

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 122

Training Guide Unix for Beginners

www.caosys.com [email protected]

Decisions Decisions At some point you will want to make a decision inside your shell script, this decision could be based on almost anything, from the parameters passed to the script, to the number of files in a particular directory. In any case you will use one of the shells flow control statements:  The if statement – used to make decisions, takes the format if [ condition ] then statement [elif condition then statement ] [else statement ] fi  The case statement – used to make decisions, allows for wildcard conditions, and is useful for performing many if then‟s one after the other, takes the format case variable in pattern1 ) statements ;; pattern2 ) statements ;; esac

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 123

Training Guide Unix for Beginners

www.caosys.com [email protected]

Using The if Statement The if statement allows you to make decisions based on a specific condition. You could, for example, say, if string1 = string2 then copy this file to that file. In fact the condition could be almost anything. You would use an if statement when the shell script needs to act differently depending on the situation. There are many types of conditional expression that can be used in an if statement:  String comparison  File Attribute checking  Integer comparison  Multiple conditions – logical operators

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 124

Training Guide Unix for Beginners

www.caosys.com [email protected]

Using The if statement The if statement allows you to make decisions based on a specific condition. You could, for example, say, if string1 = string2 then copy this file to that file. In fact the condition can be almost anything. The components of the condition need to be in quotes unless you use the [[ ]] syntax, for example, both the following lines of code do the same thing:if [ ―$str1‖ = ―$str2‖ ] if [[ $str1 = $str2 ]] You would use an if statement when the shell script needs to act differently depending on the situation. Conditions – String Comparison There are many types of conditions you can use, here are some of the most common: Operator str = pat str != pat str1 < str2 str1 > str2 -n str -z str

True if… str matches pat str does not match pat str1 is less than str2 str1 is greater than str2 str is not null (has length greater than 0) str is null (has length 0)

Some Examples if [[ $myvar = $yourvar ]] ; then print ―Match found‖ else print ―No match found‖ fi if [[ -z $myvar ]] ; then print ―you need to set myvar‖ fi

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 125

Training Guide Unix for Beginners

www.caosys.com [email protected]

Conditions – File Attributes You can also check file attributes in a conditional expression, there are many file attribute operators available, here are some of the most common: Operator -f file -d file file1 –nt file2 file1 –ot file2

True if… file exists file is a directory file1 is newer than file2 file1 is older than file2

Some Examples if [[ -f $myfile ]] ; then print ―file found‖ fi if [[ $file1 –nt $file2 ]] ; then mv $file2 ${$file2%.*}.backup cp $file1 $file2 fi Conditions – Integer Comparison Operator -lt -le -eq -ge -gt -ne

True if… less than less than or equal equal greater than or equal greater than not equal

Some Examples if [[ $num1 –eq $num2 ]] ; then print ―numbers are equal‖ fi

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 126

Training Guide Unix for Beginners

www.caosys.com [email protected]

Multiple Conditions You can use multiple conditions at the same time that adhere to standard logical operators such as AND and OR. The AND operator is implemented with &&, if [[ $str1 = $str2 && -a $file ]] ; then print ―string1 is equal to string2‖ print ―AND‖ print ―the file $file exists‖ fi The OR operator is implemented with ||, if [[ $str1 = $str2 || -a $file ]] ; then print ―string1 is equal to string2‖ print ―OR‖ print ―the file $file exists‖ fi Negate A Condition You can negate any condition using the ! symbol, for example, if [[ -f myfile ]] …. the above code says, if the file myfile exists then return true, if [[ ! –f myfile ]] … the above says, if the file myfile does not exist, then return true.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 127

Training Guide Unix for Beginners

www.caosys.com [email protected]

Using The case Statement The case statement allows you to make series of if-then-else statements in a concise way. case also lets you test strings against patterns that can contain wildcards, which makes it very useful, especially when working with filenames.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 128

Training Guide Unix for Beginners

www.caosys.com [email protected]

Using the case statement The case statement allows you to make a series of if-thenelse statements in a clean and concise way. It can make code clearer and easier to read and work with. case also lets you test strings against patterns that can contain wildcards, which make it very useful, especially when working with filenames. case takes the form, case expression in pattern1 ) statements ;; pattern2 ) statements ;; … esac An Example case $filename in *.txt ) print ―$filename is a text file‖ *.dat ) print ―$filename is a datafile‖ *.spl ) print ―$filename is a spool file‖ *.tmp ) ;; * ) print ―Filename type not specified‖ return 1; esac the above code uses wildcard pattern matching to determine what extension a file has, then it prints a message based on the extension, the exceptions are, if a file ends in .tmp then it is ignored and the *) part at the end catches all other files.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 129

Training Guide Unix for Beginners

www.caosys.com [email protected]

Lab 12 1 2

Create a shell script that takes a single parameter, displays the message, “Must provide a parameter” if no parameter is given when running the script Create a script that displays whether a file is a directory or a file. The filename should be provided as a parameter

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 130

Training Guide Unix for Beginners

www.caosys.com [email protected]

Loop the Loop The reason for wanting to create a shell script is usually to save you having to re-enter long strings of commands time and time again, well you may need the same convenience from within a shell script. You may, for example, want to process many files at a time, or continue processing until a certain condition is met. Shell provides two constructs for doing just that: The for loop  The while loop

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 131

Training Guide

www.caosys.com [email protected]

Unix for Beginners

Loop the Loop The reason for wanting to create a shell script is usually to save you having to re-enter long strings of commands time and time again, well, at some point you may need the same convenience within a shell script. You may, for example, want to process many files at a time, or continue processing until a certain condition is met. Shell provides two constructs for doing just that: The for loop A for loop allows you to repeat a command or commands a fixed number of times. You could for example, have a script which displays whether a file specified on the command line is a file or directory, you could use a for loop to process more than one file. The for loop has the following syntax: for name [in list] do statements that can read $name… done in list is the list that is used to populate the name variable on each iteration of the loop. If in list is not specified, then it defaults to $@ which is the command line arguments specified on the command line. in list can also be a statement so long as it is enclosed in reverse quotes (``). The korn shell for loop in not like the for loop in other languages, in that it is not really suitable for doing things like:for n=1 to 10 statements … next n but the for loop is ideal for working with command line arguments and lists of files. Some Examples

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 132

Training Guide Unix for Beginners

www.caosys.com [email protected]

To get information about specific UNIX users using the finger command: for names in john fred richard craig ; do finger $names done To copy all files in a specified directory into another directory and log the copy in a log file: for files in `ls /usr/bin/data/` ; do cp $files /usr/bin/backup print ―copied $file‖ >> logfile done The while loop The while loop is used when you need to repeat a command or set of commands until a certain condition is met. The while loop has the following syntax:while condition do statements … done condition can be any condition that is valid in the if statement. An Example To keep processing so long as the list variable is less than 100 digits long:counter=0 list= while [ ${#list} -lt 100 ] do list=$list―XXXX‖ let counter=$counter + 1 done print ―iterations : $counter‖

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 133

Training Guide Unix for Beginners

www.caosys.com [email protected]

Lab 13 Before you start, type, cd ../lab13 1 2 3

Using a for loop, create a script which displays command line parameters with the parameter number. Using a for loop, create a script which lists all files in the current directory and says whether they are a file or directory Using a while loop, create a script which runs in the background (put & at end of command line). The script should add a line of text to a file called whilelog.log every second (use the sleep command) so long as a file called keep_on_running exists. Run your script and wait a while then type, rm keep_on_running this will remove the keep_on_running file, your script should now stop running, use the more command to look at the file whilelog.log

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 134

Training Guide Unix for Beginners

www.caosys.com [email protected]

Section Two - Summary In summary then, we can see that UNIX provides a very powerful scripting language, known as shell. We have covered basic programming using the Korn Shell. We covered the following topics: What is a Shell Program/Script?  Executing a Shell Script  Shell Variables  Command Line Parameters  Basic String Operators  Arithmetic Expressions  Decisions Decisions  Loop The Loop

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 135

Training Guide Unix for Beginners

www.caosys.com [email protected]

What is next? In section three we cover the Source Code Control System you will be using. We cover all basic Source Control commands and do a number of exercises which involve actually using the system.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 136

Training Guide Unix for Beginners

www.caosys.com [email protected]

Section Three Source Code Control System (SCS)

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 137

Training Guide Unix for Beginners

www.caosys.com [email protected]

Source Code Control System (SCS) In this section of the course we learn how to use the Source Code Control System. We cover the following topics: What is SCS?  Directory Structures  SCS Commands  Registering an application and file with SCS  Editing a file in SCS  Moving a file to TEST or PROD  Viewing the audit trail  Other useful commands

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 138

Training Guide Unix for Beginners

www.caosys.com [email protected]

Source Code Control System (SCS) In this section of the course we learn how to use the Source Code Control System. We will cover the following topics:  What is the Source Code Control System?  Directory structures – This section goes through the directory structures used for Oracle Applications custom development and SCS  SCS commands. Here were go through actually using the SCS, we cover the following: Registering an application and file with SCS  Editing a file in SCS  Moving a file to TEST or PROD  Viewing the audit trail  Other useful commands

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 139

Training Guide Unix for Beginners

www.caosys.com [email protected]

What is the Source Code Control System? Exactly what is the Source Code Control System? Well, it is exactly what the name says, a system for controlling source code. What the Source Code Control System does it to carefully control and monitor the movement of source code in and out of

test and production codesets, ie. all movement of code should be audited and only privileged users should be able to put something onto the production system. The Source Code Control System basically does the following: Provides a simple command set for moving files around  Audits everything that happens to source code  Provides access to production codesets only via a privileged user  Keeps multiple versions of a file, although is sometimes referred to as Revision/Version control  Provides other useful utilities that aid software development

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 140

Training Guide Unix for Beginners

www.caosys.com [email protected]

What is the Source Code Control System? So, exactly what is the Source Code Control System? Well it is just as its name suggests, a system for controlling source code. What the Source Code Control System does is to carefully control and monitor the source code that moves into and out of the test and production codesets. The Source Code Control System basically does the following:  Provide simple and easy to use commands for moving files around. Basically, all files held within source control are just normal files within a well defined directory structure, moving files around manually would be a tedious task, having to remember all those large pathnames as well as having to type them in, mistakes being made would be very likely. Source Control commands basically wrap standard UNIX commands making them more user-friendly and secure.  Audit everything that happens to source code, because all movement of code is done using Source Control commands, the system audits everything you do, from editing, to installing into the production codeset.  Provide access to the production codeset only via a privileged user. Once a piece of code has been successfully tested and the user is happy with it, you will need to put the code live, Source Control restricts this „putting live‟ process to only allow privileged users to perform this task.  Keep multiple versions of a file, although this is sometimes under the remit of a Revision or Version control systems. Source control can keep multiple versions of a file that was installed into the test or production codeset, this is useful if you ever need to go back to a previous version.  Source Control provides other useful utilities that aid software development

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 141

Training Guide Unix for Beginners

www.caosys.com [email protected]

Directory Structures All source code used for Oracle Applications custom development is stored in a well-defined directory structure. Before we start on the Source Code Control System itself, it will be useful to briefly explain this directory structure and how it works with SCS.  There a 3 main codesets: DEV – Development Codeset  TEST – Test Codeset  PROD – Production Codeset  For TEST and PROD, SCS creates another directory for storing old versions of code  Each Custom Oracle Applications module has its own directory which contains further directories: BIN – For storing executables and SQL*Loader Scripts  SQL – For storing PL/SQL & SQL scripts  SRW – For storing Reports 2.5 modules  INSTALL/SQL – For storing object creation/installation scripts

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 142

Training Guide Unix for Beginners

www.caosys.com [email protected]

Directory Structures All source code used for Oracle Applications custom development is stored in a well-defined directory structure. Before we start on the Source Code Control System itself, its will be useful to briefly explain this directory structure and how it works with SCS.  There a 3 main codesets, each with there own directory: DEV – Development Codeset  TEST – Test Codeset  PROD – Production Codeset  For TEST and PROD, SCS creates another directory for storing previous versions of code  Each Custom Oracle Applications module (such as Order Entry) has its own directory which contains the following directory: A product version directory, for example, 1.0.0, this directory then contains the actual code directories, as follows: BIN – For storing executables and SQL*Loader Scripts  SQL – For storing PL/SQL & SQL scripts  SRW – For storing Reports 2.5 modules  INSTALL/SQL – For storing object creation/install All these directories are stored as subdirectories of a custom development base directory which can be anything your administrator decides, for example, you may have a base directory of, /u01/app/applmgr/custom all custom modules would be below this directory. This base directory is commonly referred to as CUSTOM_TOP. See next page for a complete example. You can see from the diagram on the following page that manually controlling files in all these directories would be open to error and very tedious. SCS solves these problems.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 143

Training Guide Unix for Beginners

www.caosys.com [email protected]

Example Directory Structure Lets assume we have a Custom Application, Order Entry. The custom shortname given to this application may be something like CUMOE, meaning Cummins Orders Entry. Following the standard set by Oracle, we would have a directory structure as follows:-

CUSTOM TOP

DEV

CUMOE

TEST

PROD

The same directory structure can be found below TEST and PROD

1.0.0

BIN

SRW

SQL

INSTALL

SQL

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 144

Training Guide Unix for Beginners

www.caosys.com [email protected]

SCS Commands SCS has been built to aid the process of software development. Its main aim is to control and audit all movement of source code. This is achieved by providing the user with a number of easy to use commands designed specifically for working with 3 codeset environments. SCS also provides a number of other useful commands which are of use to the developer. The most commonly used SCS commands are: sch

Online help for SCS

 new_app

Register a new application

 new

Register a new file

 me

Edit a file

 put

Put file into TEST and/or PROD

 take

Take file from TEST or PROD

 info

View audit/file information

 comp

Compile code

 lock

Lock a file/View locks

 unlock

Unlock a file

 Other useful commands

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 145

Training Guide Unix for Beginners

www.caosys.com [email protected]

SCS Commands SCS has been built to aid the process of software development. Its main aim is to control and audit all movement of source code. This is achieved by providing the user with a number of easy to use commands designed specifically for working with 3 codeset environments (DEV, TEST & PROD). Some of more commonly used commands are listed below, we will cover each in detail.          

sch new_app new me put take info comp lock unlock

Online help for SCS Register a new application Register a new file Edit a file Put file into TEST and/or PROD Take file from TEST or PROD View audit/file information Compile code Lock a file/View locks Unlock a file

Logging into SCS Before you can use the SCS, you must first log into the system. This is done at actual UNIX login time. After entering your username and password, you will be prompted for your SCS username, assuming you have a valid SCS username, simply enter it at this point. You should see an SCS welcome message. You are now logged into SCS and all SCS commands are available for use.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 146

Training Guide Unix for Beginners

www.caosys.com [email protected]

SCS Command : sch At any time while logged into SCS you can get help by using the sch command. It is easy to use, at the command prompt simply type, sch

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 147

Training Guide Unix for Beginners

www.caosys.com [email protected]

SCS Command : new_app All files must exist in what is know as an application, this application determines the directories where the files are stored (see CUMOE in example directory structure diagram). Before you can add files to an application you must first register the application. You do this using the new_app command. new_app is very easy to use, at the command prompt simply type, new_app application application : the new application shortname. new_app will create all the directories required for your new application. Example new_app CUMOE

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 148

Training Guide Unix for Beginners

www.caosys.com [email protected]

SCS Command : new To allow SCS to control a file, the file must first be registered with SCS. This is done using the new command. The syntax for new is, new file application type file : file to be registered application : registered application type : type of file you are registering, which must one of: sql – If your file is a SQL script of any kind  srw – For Reports 2.5 modules  bin – For exe files and SQL*Loader scripts  install/sql – For install/object creation files When you use new, it checks to see if the file is in your current directory, if it is, then it copies it to the correct application directory. Using new will lock the file. Some Examples new CUMPTOE.pkg cumoe sql new CUMRTOE.rdf cumgl srw

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 149

Training Guide Unix for Beginners

www.caosys.com [email protected]

SCS Command : me If your file is a SQL script or any kind of ASCII readable text then you can edit/view the file using the me command. The me command uses the DEV codeset. me uses the vi editor. The syntax for me is, me [-e] [-v] [[file [application]] file : file to edit/view. If you omit file then the last file you worked on will be opened. application : application where file can be found. This is optional as me will work it out for you. [-e] : prompt for error line number. Sometimes the PL/SQL compiler gives an error line number, this does not necessarily relate to the actual line number in the file, using the [-e] will try to find the real line number. [-v] : open editor in view only mode. Using me without the [-v] option will lock the file. Some Examples me –v CUMPTOE.pkg me –e CUMRTOE.rdf me

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 150

Training Guide Unix for Beginners

www.caosys.com [email protected]

SCS Command : put At some point you will want to put your file to the test and/or prod codesets. This is done using the put command. The syntax for put is, put [-t] file codeset [application] file : file to be put codeset : where to put file. Must be test, prod or both [-t] : this option is used to put from test to prod instead of putting from dev to prod Once you have put a file to the prod codeset, the file will be unlocked. Some Examples put HOERTOE.rdf both put HOERTOE.rdf test

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 151

Training Guide Unix for Beginners

www.caosys.com [email protected]

SCS Command : take You may want to get take a copy of the code in test or prod and put it back into dev, you do this with the take command. The syntax for take is, take [-t] file[-backup] [application] file : file you want to take -backup : specify a previous version application : specify where file can be found. Optional as take will work this out for you. [-t] : this option is used if you want to take from prod to

test instead of prod to dev. The take command will put a lock on the file. Some Examples take HOERTOE.rdf prod take HOERTOE.rdf-4 test

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 152

Training Guide Unix for Beginners

www.caosys.com [email protected]

SCS Command : info Any SCS command you use that involves working with source code is audited. This audit trail can be viewed using the info command. info also allows you to view codeset information about a particular file. The syntax for info is, info [-d [file [application]]] [search] [-d] : if you specify this option, info will display codeset and special instruction info about the specified file. If you omit file then the last file you worked on will be displayed. search : if not using [-d] then you can enter a search string, info will search through the entire audit trail, displaying any info containing the string search. If you omit search then all info is displayed. Some Examples info registered info CUMOE | grep sql info –d HOETTOE.rdf

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 153

Training Guide Unix for Beginners

www.caosys.com [email protected]

SCS Command : comp If you are working on a PL/SQL program, whether it be a package, procedure or trigger, you will at some point need to compile it, SCS provides the comp command to do this. The syntax for comp is, comp [file application [codeset]] file application : specifies the file you want to compile. If you omit this then the last file you worked on will be compiled. codeset : specifies on which codeset you want to compile. This must be dev, test or prod. dev is the default. Code is compiled on the current database instance, so if you want to actually compile into the prod database, then you must first ensure your environment points to prod. Some Examples comp comp HOEPTOE.pkg cumoe

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 154

Training Guide Unix for Beginners

www.caosys.com [email protected]

SCS Command : lock If you know you are going to want to work on a file in the future and you want to make sure nobody else works with the file in the meantime, you can lock the file using the lock command. lock can also be used to display all files that are currently locked and who is locking them. The syntax for lock is, lock [file [application]] file application : specifies the file you want to lock,

application is optional. If you omit file then all current locks are displayed. Some Examples lock lock HOEPTOE.pkg lock HOEPTOE.pkg cumoe

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 155

Training Guide Unix for Beginners

www.caosys.com [email protected]

SCS Command : unlock Although SCS generally takes care of the locking and unlocking of files, you may find that you have locks on files that you don‟t want (if you use me without [-v] to view a file for instance). The syntax for unlock is, unlock [-a] [file [application]] file application : specifies the file you want to unlock, application is optional. [-a] : will unlock all files you currently have locked Some Examples unlock -a unlock HOEPTOE.pkg unlock HOEPTOE.pkg cumoe

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 156

Training Guide Unix for Beginners

www.caosys.com [email protected]

Other Useful Commands SCS provides several other useful commands. Here is a brief description of each.  prep & inst – Used together for putting many files at the same time  cdh – goto your SCS home directory  dev, tst, prd – change directory to the different codesets  cls – clear the screen  sql – Shortcut for sql*plus  dir – list directories  lt – list files in the current directory  src – search through all registered source code  nreg – list unregistered files found in registered applications  desc – describe a DB object

 out – list files that are out of step between codesets  del – remove a file from SCS  li – find real line number (like me –e)  si – add special instructions to a file  xpl – quick explain plan utility.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 157

Training Guide Unix for Beginners

www.caosys.com [email protected]

Other Useful Commands continued….

 hmak & hmdo – create and execute a SCS makefile. useful for go live situations

 lps – print a file registered in SCS to a default printer  obj – list all DB all matching a string you enter Future Commands There are a couple of commands that have not been implemented yet,

 save – make a quick, safe backup of a file on dev. Useful if you need to make a quick change to a file in

test or prod.

 reco – used to get back a file saved with save

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 158

Training Guide Unix for Beginners

www.caosys.com [email protected]

Lab 14 1 2 3 4 5 6 7 8 9 10 11 12 13

Ensure you have logged into SCS. Create a new application called app_, for example, app_craig Create a file in your current directory called file_, for example file_craig Register the file created in step 3 in the application created in step 2, you can choose the file type Put the file to the test codeset View the audit trail and the file info Put the file to prod Take the file back from prod Edit the file and change something Put the file to test and prod Take the first backup file from prod View the audit trail Try to edit a file that someone else is working on

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 159

Training Guide Unix for Beginners

www.caosys.com [email protected]

Section Three - Summary In summary then, SCS provides an easy to use and controlled way to work with files on multiple codesets. SCS also provides many other useful commands.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 160

Training Guide Unix for Beginners

www.caosys.com [email protected]

What is Next? As far as UNIX is concerned, that is it!! You should now be equipped with enough knowledge to allow you to use the UNIX OS effectively, you should also be able to understand and create simple shell scripts as well as being able to use the SCS. I suggest you try to use UNIX as much as possible to further enhance your knowledge, re-read these notes and any other documentation you can lay your hands on, including the online manual pages.

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 161

Training Guide Unix for Beginners

www.caosys.com [email protected]

Notes

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 162

Training Guide Unix for Beginners

www.caosys.com [email protected]

Notes

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 163

Training Guide Unix for Beginners

www.caosys.com [email protected]

Notes

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 164

Training Guide Unix for Beginners

www.caosys.com [email protected]

Answers to Exercises Lab 2

Exercise 1 2 3 4 5 6 7 8 9 10 11

Answer ls file* ls –r cd /usr cd cp testdata.spl testdata.bak cp current mv previous backup chmod –rw * ls -l chmod +rw * cat file01 file02 > file03 more file03 tail –10 file04 | head –5 ls | head –5

Lab 3

Exercise 1 2 3

Answer find /usr/bin –name l* -print grep unix * grep string grepit | grep –v the

Lab 4

Exercise 1 2 3

Answer wc –l | tail -1 ls –lt | awk ‗{print $5 ― ― $9}‘ > filesizes diff diff1 diff2 > diff3 cat diff3 wc –c | tail –1 ls –l | awk ‗{print $1 ― ― $9}‘

4 5 Lab 6

Exercise 1 2 3

Answer PS1=‖Enter Commands Here->‖ alias edit=vi alias newlist=‖ls –lt | head –6‖

Lab 7

Exercise 1

Answer . runme

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 165

Training Guide Unix for Beginners 2

Lab 8

Exercise 1

2 3

www.caosys.com [email protected]

runme vi firstshell chmod +x firstshell firstshell Answer #!/bin/ksh var=10 var2=$var1 print $var2 The second line should not prefix the variable with a dollor ($). #!/bin/ksh firstfile=`ls | head –1` print $firstfile

Lab 9

Exercise 1

Answer #!/bin/ksh print ―Parameter print ―Parameter print ―Parameter print ―Parameter

Lab 10

Exercise 1

Answer #!/bin/ksh myvar=${yourvar:-10} print $myvar #!/bin/ksh param1=${1:-‗craig‘} param2=${2:-$1} print $param1 print $param2 #!/bin/ksh result=${1#$2} print $result

2

3

Lab 11

Exercise 1

1 = $1‖ 2 = $2‖ 3 = $3‖ Count = $#‖

Answer #!/bin/ksh result=10 ((result = result + 50 )) print $result

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 166

Training Guide Unix for Beginners Lab 12

Exercise 1

www.caosys.com [email protected]

Answer #!/bin/ksh if [[ $1 = ―‖ ]] ; then print ―Must enter a parameter‖ fi or

2

Lab 13

Exercise 1

2

3

Lab 14

Exercise 2 3 4 5 6

#!/bin/ksh if [ $# -eq 0 ] ; then print ―Must enter a parameter‖ fi #!/bin/ksh if [[ -f $1 ]] ; then print ―$1 is a file‖ fi if [[ -d $1 ]] ; then print ―$1 is a directory‖ fi Answer #!/bin/ksh par_number = 0 for params in $* ; do (( par_number = par_number + 1 )) print ―Parameter $par_number = $params) done #!/bin/ksh for files in `ls` ; do if [[ -f $files ]] ; then print ―$files is a file‖ elif [[ -d $files ]] ; then print ―$files is a directory‖ fi done #!/bin/ksh while [[ -f keep_on_running ]] ; do print ―some text‖ >> whilelog.log sleep 1 done Answer new_app app_craig vi file_craig new file_craig app_craig sql put file_craig test info info –d (or info –d file_craig)

Copyright © 1999 - 2009 CaoSys Limited

UNIX For Beginners - 167

Training Guide Unix for Beginners 7 8 9 10 11 12 13

www.caosys.com [email protected]

put file_craig prod take file_craig prod me (or me file_craig) put file_craig both take file_craig-1 prod info me