Example 1: To Print the 2nd, 3rd and 5th Columns only

fruit_id,fruit_name,fruit_qty,unit_price,total_price
1,Mango,2,10,20
2,Apple,6,15,90
3,Banana,4,8,int
4,Watermelon,7,9,63

Solution:

awk ’BEGIN{
FS=",";
OFS="|";
}
{
print $2,$3,$5;
}’ fruits.txt

--------------------------------------------------------------------------------------

Example 2: To Find the total price of an apple (PIPELINING grep & awk)

fruit_id,fruit_name,fruit_qty,unit_price,total_price
1,Mango,2,10,20
2,Apple,6,15,90
3,Banana,4,8,int
4,Watermelon,7,9,63
5,apple,3,15,45

Solution:

grep -i "[Aa]pple" fruits.txt | awk ’BEGIN{FS=",";s=0;}

{
s = s+$5;
}
END{
print "The total price spent on apple fruit is " s;
}’

--------------------------------------------------------------------------------------

Example 3: Printing the Multiplication of Number 5

Solution:

awk ’BEGIN{
i=1;
while(i<=10)
{
print "5 * "i" = "i*5;
i++;
}
}’

--------------------------------------------------------------------------------------
Example 4: Find the number of fields in a record and total no.of records present in the text file

mango
watermelon
pine apple
custard apple
banana

Solution:

awk ’BEGIN{FS=" ";}

{
print "The Number of fields present in record " NR " is " NF".";
}
END{
print NR" records are present.";
}’ fruits2.txt

---------------------------------------------------------------------------------------------

Input Data:

29,Arun
26,Karthik
28,Kiran
52,Raju
78,Rachel

Example 5:

awk ’BEGIN{FS=",";}
{
if($1>50)
{
print "Value is greater than 50.";
}
else
{
print "Value is less than 50";
}
i=1;
while(i<=1)
{
print "Row " NR ", Column 2 (loop "i "): " $2;
i++;
}
}
END{
print match("End of Code", /of/);
print RSTART, RLENGTH;
}’ example.txt

Example 6: Average
 grep ’,A’ input3.txt | awk ’BEGIN{FS=",";total_age=0;}
{
total_age += $1;
count++;
}
END{
if(count>0)
{
avg_age = total_age/count;
printf "Average Age for Group A:%.2f\n", avg_age;
}
else
{
print "No Data Found";
}
}’

Example 7: Updating the text

Data

A|T|X|G
G|X|T|A
G|T|A|X

Solution:

awk ’BEGIN{OFS=FS="|";}
{
if($2=="T")
{
sub($1,"T");
print;
}
}END{

}
’ data.txt

EXAMPLE 8: Deleting the rows

Data:

500 500 500

1000 1000 1000
2000 2000 2000
3000 3000 3000

Solution:

awk ’{
for (i=1;i<=NF;i++)
{
if ($i<=1000){
 next;
}

};

print;

}’ < data.txt

--------------------------------------------------------------------------------------

ROYAL MAIL HOTEL (CASE STUDY)

employeeDetails.txt (DATA) :

Name,Age,Place,Experience,Salary
Anish,26,Chennai,2,10000
Jai,24,Chennai,2,10000
Kumar,29,Hyderabad,5,32000
John,32,Mumbai,2,11000
Neethu,21,Nagpur,3,13000
Satish,22,Ahmedabad,2,10000

Situation: To Print the complete data

awk ’BEGIN{FS=",";}
{
print;
}’ employeeDetails.txt

Situation: Manager wish to display the employee name and salary working in royal mail hotel

awk ’BEGIN{FS=",";}
{
print $1,$5;
}’ employeeDetails.txt

Situation: Manager wishes to print details of Kumar and Satish

awk ’/Kumar|Satish/’ employeeDetails.txt

Situation: Manager wishes to find the total expenses of hotel per month in the form of salary

awk -F"," ’BEGIN{

s=0;
}
{
s=s+$5;
}
END{
print "Total Exprenses per month in form of salary is " s;
}’ employeeDetails.txt

Situation: Manager wishes to find the total no.of employees earning 10000 per month
awk ’/10000/{
++count;
}
END{
print "No.of employees earning 10000 per month is " count;
}’ employeeDetails.txt

Situation: Manager wishes to find the employees as best performers who completed 2 years of
exprerience and earning more than 10000

awk ’BEGIN{
FS=",";
print "*********************Performance Report*********************";
}
{
if(NR!=1)
{
if($4>=2 && $5>10000)
{
print $1" is a good performer";
}
else if($4==2 && $5<=10000)
{
print $1" Needs to improve";
}
}

}
END{
print "*********************Performance Report*********************";
}’ employeeDetails.txt

-----------------------------------------------------------------------------------------

SORT COMMAND

input data file: list

1,John Cena,Title 758,Price $7.30

5,GoldBerg,Title 392,Price $1.98
2,Randy Orton,Title 739,Price $6.20
4,Triple H,Title 893,Price $6.42

input data file: list1

7,Shawn Michales,Title 620,Price $1.64

8,Roman Reigns,Title 920,Price $1.03
3,Brock Lesnar,Title 201,Price $6.71
6,Rey mysterio,Title 109,Price $12.4

Situation: Sort on the 2nd field of file named "list". File list is comma seperated value
Solution: sort -t"," -k 2 list.txt

Situation: Sort the input data file

Solution: sort list;

Situation: Sort can be applied on multiple files as well

Solution: sort -t"," -k 2 list.txt list1.txt

Situation: Sort in reverse order of first numeric column from multiple files

Solution: sort -nr list list1;

EXAMPLE SOLUTION:

grep ’Sangamithra’ | awk ’BEGIN{FS="|";average=0;OFS="|"}

{
if($5 >= 90 && $6 >= 90)
{
sum = $5 + $6;
average = sum / 2;
print $1, $2, $3, $4,average;
}
}
’ | sort -nr

---------------------------------------------------------------------------------------------------------

UNIQ COMMAND

input data file: list.txt

unix operating system

unix operating system
unix dedicated server
linux dedicated server

Situation: surpass the duplicate records

Solution: uniq list.txt

Situation: Pipelining command using (Sort and Uniq)

Solution: sort list.txt | uniq

Situation: Count the repeated lines

Solution: uniq -c list.txt

Situation: Display only the duplicate lines

Solution: uniq -d list.txt

Situation: Display all duplicate lines

Solution: uniq -D list.txt

Situation: Print only unique lines

Solution: uniq -u list.txt

------------------------------------------------------------------------------------------

GREP COMMAND
demo_file

THIS LINE IS THE 1ST UPPER CASE IN THIS FILE.

this line is the 1st lower case line in the file.
This Line Has All Its First Characters as Upper Case.

Two lines above this line is empty.

And this is the last line.

Situation: Search for a given string and also you can check a string in multiple files

Solution: grep "this" demo_file.txt

Situation: Search for a given string with caseinsensitive approach

Solution: grep -i "this" demo_file.txt

Situation: Search for a line starting with ’Two’ and ending with ’empty’

Solution: grep "Two.*empty" example.txt

Situation: To Search for a word and to avoid it to match the substrings -w option is used

Solution: grep -w "is" example.txt

Situation: If you want to display the lines which does not matches the given string/pattern then

Solution: grep -v "Two" example.txt

-----------------------------------------------------------------------------------------------------------

SED COMMAND

file_name

1.Linux - System Admin, Scripting etc.

2.Database - Oracle, mySQL, Linux etc.
3.Hardware
4.Security (Firewall, Network etc.)
5.Storage
7.Hello World
Operation in Sed

-n = suppress automatic printing of patternspace. It will not print any thing until explicit request to print is
found
-d = delete the line when pattren matches
-i = modifies the text in the input file

SYNTAX:

Replaces the occurrences of ’Linux’ with ’Unix’ in file_name.txt

sed ’s/Linux/Unix/’ file_name.txt

Replaces the 2nd occurrences of ’Linux’ with ’Unix’ in a line in file_name.txt

sed ’s/Linux/Unix/2’ file_name.txt

Replaces text in specific line in file_name.txt

sed ’2s/Linux/Unix/g’ file_name.txt

Delete the lines Matching a pattern

sed ’/Hardware/d’ file_name.txt

Delete the specific line

sed ’3d’ file_name.txt

Insert a line before a pattern

sed ’/Storage/i This is the new line’ file_name.txt

Append a line after a pattern

sed ’/Storage/a 6.Python’ file_name.txt

Print Specific Lines

sed -n ’2,5p’ file_name.txt

Replacing all digits with ’#’

sed ’s/[0-9]/#/’ file_name.txt

Save Changes to a New File

sed ’s/Old/New/g’ file_name.txt > file_name2.txt

Example: The search for "Unix" to replace with "Unix OS" should be case-insensitive.

INPUT:
If the input file contains the following lines
Unix is an multi-user,multi-tasking system.
It is a command based operating system.
We will learn unix in this module.

Solution: sed ’s/Unix/Unix OS/i’

------------------------------------------------------------------------------ Pipelining
------------------------------------------------------------------
Write a unix command to find the total price of assests belonging to finance department

Input:

AssetName,AssetType,Department,AssetPrice
lap1,laptop,finance,60000
des1,desktop,FinancE,40000
lap2,laptop,FINANCE,50000
des2,desktop,inance,80000
prn1,printer,finan,10000
prn2,printer,Finances,20000

Output:

Total Asset Price = 150000

Solution:

grep -i -w "finance" | awk ’BEGIN{FS=",";s=0}

{
s = s + $4
}
END{
if(s > 0)
{
print "Total Asset Price = " s;
}
else
{
print "No Asset Found"
}
}’

------------------------------------------------------------------------------AWK Command-------------------------------------

Regular Expressions

DOT

echo -e "cat\ncut\nfun\nfin\nfan" | awk ’/c.t/’

START OF THE LINE

echo -e "This\nThat\nThere\nTheir\nthese" | awk ’/^The/’

END OF THE LINE

echo -e "knife\nknow\nfun\nfin\nfan\nnine" | awk ’/n$/’

MATCH CHARACTER SET

echo -e "Call\nTall\nBall" | awk ’/[CT]all/’

EXCLUSIVE SET
echo -e "Call\nTall\nBall" | awk ’/[^CT]all/’

ALTERATION

echo -e "Call\nTBall\nSmall\nShall" | awk ’/Call|Ball/’

ZERO OR ONE OCCURRENCE

echo -e "Colour\nColor" | awk ’/Colou?r/’

ZERO OR MORE OCCURRENCE

echo -e "ca\ncat\ncatt" | awk ’/cat*/’

ONE OR MORE OCCURRENCE

echo -e "111\n22\n123\n234\n456\n222" | awk ’/2+/’

GROUPING

echo -e "Apple Juice\nApple Pie\nApple Tart\nApple Cake" | awk ’/Apple (Juice|Cake)/’

----------------------------------------------------------------------------------

USER DEFINED FUNCTIONS

# Returns minimum number
function find_min(num1, num2){
if (num1 < num2)
return num1
return num2
}

# Returns maximum number

function find_max(num1, num2){
if (num1 > num2)
return num1
return num2
}

# Main function
function main(num1, num2){
# Find minimum number
result = find_min(10, 20)
print "Minimum =", result

# Find maximum number

result = find_max(10, 20)
print "Maximum =", result
}

# Script execution starts here

BEGIN {
main(10, 20)
}
--------------------------------------------------------------------------------

CONTROL FLOW STATEMENTS

IF STATEMENT

awk ’BEGIN {num = 10; if (num % 2 == 0) printf "%d is even number.\n", num }’

IF-ELSE STATEMENT

awk ’BEGIN {
num = 11;
if (num % 2 == 0)
printf "%d is even number.\n", num;
else
printf "%d is odd number.\n", num;
}’

IF-ELSE-IF LADDER

awk ’BEGIN {
a = 30;

if (a==10)
print "a = 10";
else if (a == 20)
print "a = 20";
else if (a == 30)
print "a = 30";
}’

---------------------------------------------------------------------------

HACKER RANK

FOR LOOP to display only odd natural numbers from 1 to 99

awk ’BEGIN{
# Loop through numbers from 1 to 99
for(i=1;i<=99;i++)
{
# Check if the number is odd (remainder when divided by 2 is not equal to 0)
if(i%2 != 0)
{
print i;
}
}
}’

Script which accepts name as input and displays the greeting "Welcome(name)"

read name;
echo "Welcome $name";
Use a for loop to display the natural numbers from 1 to 50

awk ’BEGIN{
for(i=1;i<=50;i++)
{
if(i/2 != 0)
{
print i;
}
}
}’

Given two integers, X and Y , find their sum, difference, product, and quotient.

read a;
read b;

echo $(($a + $b));

echo $(($a - $b));
echo $(($a * $b));
echo $(($a / $b));

Given two integers, X and Y, identify whether X<Y or X>Y or X=Y.

Exactly one of the following lines:

- X is less than Y
- X is greater than Y
- X is equal to Y

awk ’BEGIN{FS=",";}
{
X=$1;Y=$2;

if(X > Y)
print "X is greater than Y";
else if(X < Y)
print "X is less than Y";
else
print "X is equal to Y";
}’ input.txt