Introduction to Unix

Brian Taesung Kim [email protected]

Supercomputing Facility Texas A&M University Since 2009

man format and display the on-line manual pages an adult male person, as distinguished from a boy or a woman.

Contents Concepts ● Unix,Linux,Operating System,Kernel,Shell,Process ● Account,Filesystem,Home Directory,Path Remote Access ● Remote login,File transfer,Check availability Commands ● Online Manual ● Edit text file ● File,Directory,Search ● I/O Redirection,Pipe,Alias ● Permission Programming Environment ● Process, Stack, Heap, BSS, Data References

Unix Operating system Multitasking Multi-user 1969 AT&T Running on PDP-7

http://www.faqs.org/docs/artu/ch02s01.html

http://academic.udayton.edu/SaverioPerugini/courses/cps346/lecture_notes/UNIXphilo.html

Evolution of Unix and Unix-like OS

From Wikipedia

Linux Unix-like free Operating system 1991 ~ Linus Tovalds & many others Linux distribution ● Unix-like operating system based on "Linux kernel" Used by all of world top 10 Supercomputers(June, 2011)

Operating System

Remote Access Windows

Linux/Mac

Login

putty

ssh

Copy files

winscp

scp

Check availability

ping

ping

Free download from putty http://www.chiark.greenend.org.uk/~sgtatham/putty/ *Configuration http://sc.tamu.edu/help/access/windows.php

winscp http://winscp.net/

Online Manual Display manual by command name ● $man command Display manual by keyword ● $man -k keyword Display manual in certain sections ● $man -S section_number command Contents of manual ● Name Manual Sections 1 Commands ● Synopsis 2 System calls ● Description 3 C library ● See also 4 Special files Search within manual 5 File formats /keyword n(next),N(previous)

Copy files and directories $man cp CP(1)

User Commands

CP(1)

NAME cp - copy files and directories SYNOPSIS cp [OPTION]... [-T] SOURCE DEST cp [OPTION]... SOURCE... DIRECTORY cp [OPTION]... -t DIRECTORY SOURCE... DESCRIPTION Copy SOURCE to DEST, or multiple SOURCE(s) to DIRECTORY.

Move(rename) files $man mv MV(1)

User Commands

MV(1)

NAME mv - move (rename) files SYNOPSIS mv [OPTION]... [-T] SOURCE DEST mv [OPTION]... SOURCE... DIRECTORY mv [OPTION]... -t DIRECTORY SOURCE... DESCRIPTION Rename SOURCE to DEST, or move SOURCE(s) to DIRECTORY.

Remove files or directories $man rm RM(1)

User Commands

RM(1)

NAME rm - remove files or directories SYNOPSIS rm [OPTION]... FILE... DESCRIPTION This manual page documents the GNU version of rm. rm removes each specified file. By default, it does not remove directories.

File & Directory File

Directory

Create

vi

mkdir

Modify

mv

mv

Copy

cp

cp

Delete

rm

rmdir

View

cat,more,less

ls

Permission

chmod

chmod

Edit text file vi ● Standard editor on Linux ● Text based ● Command mode and Input mode ● Reference ○ http://www.atmos.albany. edu/deas/atmclasses/atm350/vi_cheat_sheet.pdf emacs ● Text based / GUI based nedit/gedit(not always available) ● GUI based *For Microsoft Windows user ● Need to convert text file format for Unix/Linux ○ $dos2unix filename

Unix Standard Directory Hierarchy

Absolute path ● Path to file/directory from /(root) ○ $dir /usr/local Relative path ● Path to file/directory from not /(root) ○ $cd /usr/bin ○ $dir ../local Display directory tree ● $tree

Special symbol . : Current directory .. : Parent directory

Search files Search pattern from files ● $grep option pattern file ● Ex) To see if result.txt contains 0.2487 ○ $grep 0.2487 result.txt ● Ex) Search multiple patterns ○ $grep 'pattern_a\|pattern_b' result.txt ● Ex)Search non-matching lines ○ $grep -v 'pattern' result.txt Search files by name ● $find path expression ● Ex) To find file under current directory with ending ’.txt’ ○ $find . -name '*.txt' ● Ex) Search and Run ○ $find . -name "*.txt" -exec wc {} \; Search files by size ● Ex) To find file larger than 50KB ○ $find . -size +50k

Decompression File type

Decompress

Compress

file.tar

tar -xf file.tar

tar -cf file.tar files

file.tar.gz

tar -zxf file.tar.gz

tar -czf file.tar.gz files

file.tar.bz2

tar -jxf file.tar.bz2

tar -cjf file.tar.bz2 files

file.gz

gzip -d file.gz

gzip file

Process An instance of a program that is being executed ● Identified by PID(Process ID) ● Created by Parent process(PPID) Shows current status of processes ● $ps ○ -A : All ○ -l : Long Format ● $top ○ -u USER_NAME : Filter by user Kill process ● Send kill 'signal' to process ○ $kill PID ○ $kill -9 PID

Advanced Process Management Report status immediately ● $set -b Check exit status of process ● $echo $?

Run command in background ● $command& Wait for background process to finish ● $wait Run multiple commands in background ● $(command1;command2)& Control dependency between commands ● Run command2 only when command1 succeeds ○ $command1 && command2 ● Run command2 only when command1 fails ○ $command1 || command2

Signal Mechanism to notify a process of a particular event Example) ● Terminate process ○ $kill PID ● Stop process ○ $kill -sigstop PID ● Continue process ○ $kill -sigcont PID Default action can be List of signals redefined ● $kill -l $trap arg signals 1) SIGHUP 6) SIGABRT 11) SIGSEGV 16) SIGSTKFLT 21) SIGTTIN 26) SIGVTALRM ....

2) SIGINT 7) SIGBUS 12) SIGUSR2 17) SIGCHLD 22) SIGTTOU 27) SIGPROF

3) SIGQUIT 8) SIGFPE 13) SIGPIPE 18) SIGCONT 23) SIGURG 28) SIGWINCH

4) SIGILL 9) SIGKILL 14) SIGALRM 19) SIGSTOP 24) SIGXCPU 29) SIGIO

5) SIGTRAP 10) SIGUSR1 15) SIGTERM 20) SIGTSTP 25) SIGXFSZ 30) SIGPWR

Input/Output redirection Redirect standard input, output, or error of a process ● STDIN(0),STDOUT(1),STDERR(2) Redirect standard output to file ● $ls > output.txt Redirect standard error to file ● $ls 2> error.txt Combine standard error & standard output ● $ls > output.txt 2>&1 Append standard output to existing file ● $ls >> ls.txt Redirect input Prevent being overwritten ● $sort < city.txt ● $set -o noclobber Pipe ● Send output of command to other command as input ○ $cat city.txt | wc

Example of I/O Redirection Redirect output of multiple commands into one file $(command1;command2) > output.txt Redirect output and error to different file $command > output.txt 2> error.txt Send text file through email $mail -s "Test result" [email protected] < cf.txt

Send text file as an attachment of email $echo | mutt -s "Test result" -a cf.txt help@sc. tamu.edu

Alias Abbreviation or a ‘nickname’ for an existing command User definable Useful if ● command has a number of options or arguments ● the syntax is difficult to remember Define in environment file(.bashrc) to make it permanent. Define alias ● $alias myls=’ls -alF’ Run alias ● $myls Override alias Display all defined alias ● $\ls ● $alias Delete alias ● $unalias myls

Environment Shell initialization files for customizing working environment such as 'environment variables' and 'alias' ● Login shell ○ ~/.bash_profile ● Non login shell ○ ~/.bashrc Environment variables ● Show all variables ○ $set ● Show a single variable ○ $echo $PATH ● Define variables ○ For current shell only ■ $CLASS=unix ○ For all subsequent shell ■ $export CLASS=unix

Permission Methods of administering permissions or access rights to specific users and groups of users $ls -l hello.exe -rwx------ 1 tskim staff 4882 2010-02-10 11:17 hello.exe

u g o r: read w: write x: execute u: user g: group o: others

Remove execution permission from user $chmod u-x hello.exe Remove write permission from user $chmod u-w hello.exe Remove execution permission from group $chmod g-x hello.exe Remove write permission from group $chmod g-w hello.exe Remove write permission from others $chmod o-w hello.exe

Anatomy of Permission 0 --- no permission 1 --x execute 2 -w- write 3 -wx write and execute 4 r-- read 5 r-x read and execute 6 rw- read and write 7 rwx read, write and execute 0: 000 1: 001 2: 010 3: 011 4: 100 5: 101 6: 110 7: 111

Set all permission for all $chmod u+rwx,g+rwx,o+rwx hello.exe $chmod 777 hello.exe Set read,write for all $chmod u+rw,g+rw,o+rw hello.exe $chmod 666 hello.exe Set read for all $chmod u+r,g+r,o+r hello.exe $chmod 444 hello.exe Set read,write,execut for only me $chmod u+rwx,g-rwx,o-rwx hello.exe $chmod 700 hello.exe

Split large file into smaller files Split largefile into multiple smaller files $split -b1m largefile $ls -lh -rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--

1 1 1 1 1 1

tskim tskim tskim tskim tskim tskim

staff staff staff staff staff staff

5.0M 1.0M 1.0M 1.0M 1.0M 1.0M

Feb Feb Feb Feb Feb Feb

2 2 2 2 2 2

11:44 11:47 11:47 11:47 11:47 11:47

largefile xaa xab xac xad xae

Combine parts into one large file $cat x* > largefile $ls -l largefile -rw-r--r-- 1 tskim staff 5242880 Feb

2 11:52 largefile

Resource limit

Building program Compiling ● Converting source code into object code Linking ● Combining object codes to generate executable file Makefile ● Script to automate building procedure Library ● Pre-compiled object code for certain computation *Building program is sensitive to environment such as ● Operating system version ○ $cat /etc/*release ● Kernel version ○ $uname -a ● Compiler version ○ $icc -V # Intel compiler ○ $gcc -v # GNU compiler

Anatomy of Building Program

Sample program

Process Memory Layout

BSS section

Data section

Stack section

Heap section

References on the web For beginner ● http://www.ee.surrey.ac.uk/Teaching/Unix/ ● http://linuxcommand.gds.tuwien.ac.at/learning_the_shell. php For experienced user ● http://www.arachnoid.com/linux/shell_programming.html ● http://www.linuxcommand.org/writing_shell_scripts.php