cosc362 - fall 2015

Awk Introduction Tutorial – 7 Awk Print Examples by SASIKALA on JANUARY 6, 2010

This is the first article on the new awk tutorial series. We’ll be posting several articles on awk in the upcoming weeks that will cover all features of awk with practical examples. In this article, let us review the fundamental awk working methodology along with 7 practical awk print examples. Note: Make sure you review our earlier Sed Tutorial Series.

Awk Introduction and Printing Operations Awk is a programming language which allows easy manipulation of structured data and the generation of formatted reports. Awk stands for the names of its authors “Aho, Weinberger, and Kernighan.” The Awk is mostly used for pattern scanning and processing. It searches one or more files to see if they contain lines that matches with the specified patterns and then perform associated actions. Some of the key features of Awk are:



Awk views a text file as records and fields.



Like common programming language, Awk has variables, conditionals and loops



Awk has arithmetic and string operators.



Awk can generate formatted reports

Awk reads from a file or from its standard input, and outputs to its standard output. Awk does not get along with non-text files.

Syntax: awk '/search pattern1/ {Actions} /search pattern2/ {Actions}' file

In the above awk syntax:



search pattern is a regular expression.



Actions – statement(s) to be performed.



several patterns and actions are possible in Awk.



file – Input file.



Single quotes around program is to avoid shell not to interpret any of its special characters.

1|Page

cosc362 - fall 2015

Awk Working Methodology 1.

Awk reads the input files one line at a time.

2.

For each line, it matches with given pattern in the given order, if matches performs the corresponding action.

3.

If no pattern matches, no action will be performed.

4.

In the above syntax, either search pattern or action are optional, But not both.

5.

If the search pattern is not given, then Awk performs the given actions for each line of the input.

6.

If the action is not given, print all that lines that matches with the given patterns which is the default action.

7.

Empty braces without any action does nothing. It won't perform default printing operation.

8.

Each statement in Actions should be delimited by semicolon.

Let us create employee.txt file which has the following content, which will be used in the examples mentioned below. $cat employee.txt

5 2 -> 5 3 -> 4 4 -> 5 5 -> 4

7. Awk FILENAME Example: Name of the current input file FILENAME variable gives the name of the file being read. Awk can accept number of input files to process.

$ awk '{print FILENAME}' student-marks student-marks student-marks student-marks student-marks student-marks

In the above example, it prints the FILENAME i.e student-marks for each record of the input file.

14 | P a g e

cosc362 - fall 2015

8. Awk FNR Example: Number of Records relative to the current input file When awk reads from the multiple input file, awk NR variable will give the total number of records relative to all the input file. Awk FNR will give you number of records for each input file. $ awk '{print FILENAME, FNR;}' student-marks bookdetails student-marks 1 student-marks 2 student-marks 3 student-marks 4 student-marks 5 bookdetails 1 bookdetails 2 bookdetails 3 bookdetails 4 bookdetails 5

In the above example, instead of awk FNR, if you use awk NR, for the file bookdetails the you will get from 6 to 10 for each record.

15 | P a g e

cosc362 - fall 2015

7 Powerful Awk Operators Examples (Unary, Binary, Arithmetic, String, Assignment, Conditional, Reg-Ex Awk Operators) by SASIKALA on FEBRUARY 3, 2010

This article is part of the on-going Awk Tutorial Examples series. In our earlier awk articles, we discussed about awk print, awk user-defined variables, and awk built-in variables. Like any other programming language Awk also has lot of operators for number and string operations. In this article let us discuss about all the key awk operators. There are two types of operators in Awk. 1.

Unary Operator – Operator which accepts single operand is called unary operator.

2.

Binary Operator – Operator which accepts more than one operand is called binary operator.

Awk Unary Operator OperatorDescription + Positivate the number – Negate the number ++ AutoIncrement — AutoDecrement Awk Binary Operator There are different kinds of binary operators are available in Awk. It is been classified based on its usage.

Awk Arithmetic Opertors The following operators are used for performing arithmetic calculations.

OperatorDescription + Addition – Subtraction * Multiplication / Division % Modulo Division

Awk String Operator 16 | P a g e

cosc362 - fall 2015

For string concatenation Awk has the following operators.

OperatorDescription (space) String Concatenation

Awk Assignment Operators Awk has Assignment operator and Shortcut assignment operator as listed below.

OperatorDescription = Assignment += Shortcut addition assignment -= Shortcut subtraction assignment *= Shortcut multiplication assignment /= Shortcut division assignment %= Shortcut modulo division assignment

Awk Conditional Operators Awk has the following list of conditional operators which can be used with control structures and looping statement which will be covered in the coming article.

OperatorDescription > Is greater than >= Is greater than or equal to < Is less than =35 && $4 >= 35 && $5 >= 35) print $0,"=>","Pass"; else print $0,"=>","Fail"; }' student-marks Jones 2143 78 84 77 => Pass Gondrol 2321 56 58 45 => Pass RinRao 2122 38 37 => Fail Edwin 2537 87 97 95 => Pass Dayan 2415 30 47 => Fail

The condition for Pass is all the test score mark should be greater than or equal to 35. So all the test scores are checked if greater than 35, then it prints the whole line and string “Pass”, else i.e even if any one of the test score doesn’t meet the condition, it prints the whole line and prints the string “Fail”.

23 | P a g e

cosc362 - fall 2015

3. Awk If Else If Example: Find the average and grade for every student $ cat grade.awk { total=$3+$4+$5; avg=total/3; if ( avg >= 90 ) grade="A"; else if ( avg >= 80) grade ="B"; else if (avg >= 70) grade ="C"; else grade="D";

print $0,"=>",grade; } $ awk -f grade.awk student-marks Jones 2143 78 84 77 => C Gondrol 2321 56 58 45 => D RinRao 2122 38 37 => D Edwin 2537 87 97 95 => A Dayan 2415 30 47 => D

In the above awk script, the variable called ‘avg’ has the average of the three test scores. If the average is greater than or equal to 90, then grade is A, or if the average is greater than or equal to 80 then grade is B, if the average is greater than or equal to 70, then the grade is C. Or else the grade is D.

4. Awk Ternary ( ?: ) Example: Concatenate every 3 lines of input with a comma. $ awk 'ORS=NR%3?",":"\n"' student-marks Jones 2143 78 84 77,Gondrol 2321 56 58 45,RinRao 2122 38 37 Edwin 2537 87 97 95,Dayan 2415 30 47,

We discussed about awk ORS built-in variable earlier. This variable gets appended after every line that gets output. In this example, it gets changed on every 3rd line from a comma to a newline. For lines 1, 2 it’s a comma, for line 3 it’s a newline, for lines 4, 5 it’s a comma, for line 6 a newline, etc.

24 | P a g e

cosc362 - fall 2015

Caught In the Loop? Awk While, Do While, For Loop, Break, Continue, Exit Examples by SASIKALA on FEBRUARY 24, 2010

This article is part of the on-going Awk Tutorial Examples series. In our earlier awk articles, we discussed about awk print, awk user-defined variables, awk built-in variables, awk operators, and awk conditional statements. In this article, let us review about awk loop statements – while, do while, for loops, break, continue, and exit statements along with 7 practical examples. Awk looping statements are used for performing set of actions again and again in succession. It repeatedly executes a statement as long as condition is true. Awk has number of looping statement as like ‘C’ programming language.

Awk While Loop Syntax:

while(condition) actions



while is a keyword.



condition is conditional expression



actions are body of the while loop which can have one or more statement. If actions has more than one statement, it has to be enclosed with in the curly braces.

How it works? — Awk while loop checks the condition first, if the condition is true, then it executes the list of actions. After action execution has been completed, condition is checked again, and if it is true, actions is performed again. This process repeats until condition becomes false. If the condition returns false in the first iteration then actions are never executed.

1. Awk While Loop Example: Create a string of a specific length $awk 'BEGIN { while (count++0; i--) print $i," "; print "\n"; }' student-marks 77

84

78

2143

Jones

45

58

56

2321

Gondrol

37

38

2122

95

97

87

47

30

2415

RinRao

2537

Edwin

Dayan

We discussed about awk NF built-in variable in our previous article. After processing each line, Awk sets the NF variable to number of fields found on that line. The above script,loops in reverse order starting from NF to 1 and outputs the fields one by one. It starts with field $NF, then $(NF-1),…, $1. After that it prints a newline character. Now let us see some other statements which can be used with looping statement.

Awk Break statement Break statement is used for jumping out of the innermost looping (while,do-while and for loop) that encloses it.

5. Awk Break Example: Awk Script to go through only 10 iteration 27 | P a g e

cosc362 - fall 2015

$ awk 'BEGIN{while(1) print "forever"}'

The above awk while loop prints the string “forever” forever, because the condition never get fails. Now if you want to stop the loop after first 10 iteration, see the below script. $ awk 'BEGIN{ x=1; while(1) { print "Iteration"; if ( x==10 ) break; x++; }}' Iteration Iteration Iteration Iteration Iteration Iteration Iteration Iteration Iteration Iteration

In the above script, it checks the value of the variable “x”, if it reaches 10 just jumps out of the loop using break statement.

Awk Continue statement Continue statement skips over the rest of the loop body causing the next cycle around the loop to begin immediately.

6. Awk Continue Example: Execute the loop except 5th iteration $ awk 'BEGIN{ x=1; while(x