Sed is a stream editor in a UNIX-like operating system that is used for filtering and transforming text. Sed is derived originally from the basic line editor ‘ed’, an editor you will find on every Unix system but one that is rarely used because of its difficult user interface.
How sed Works?
As sed is a stream editor, it does its work on a stream of data it receives from stdin, such as through a pipe, writing its results as a stream of data on stdout ( often desktop screen). We can redirect this output to a file . Sed doesn’t typically modify an original input file; instead, we can send contents of our file through a pipe to be processed by sed. This means we don’t need to have a file on the disk with the data you want to change, this is particularly useful if you have data coming from another process rather than already written in a file.
Syntax of sed
# sed [option] commands [input-file]
In this post we will discuss some of the practical examples of sed command, we will be doing a lot of sed operations on the file ‘passwd’, so first copy the file ‘/etc/passwd’ to the /tmp folder.
# cp /etc/passwd /tmp/
Example 1: Deleting all the Lines with sed
# cat /tmp/passwd | sed 'd'
Above Command sent the entire contents of the file /tmp/passwd through a pipe to sed. Keep in mind the file /tmp/passwd was not altered at all. Sed only read the contents of the file and we did not tell it to write to the file, only read from it. The results of editing commands on each line printed to standard output. In this case, nothing was printed on the screen because we have used option ‘d’ to delete every line.
Example 2: Invoking sed with ‘-e’ option (add the script to the commands to be executed)
Instead of invoking sed by sending a file to it through a pipe, we can instruct sed to read data from a file as shown in the below example.
# sed -e 'd' /tmp/passwd #
Invoking sed in this manner explicitly defines the editing command as a sed script to be executed on the input file /tmp/passwd. The script is simply a one-character editing command here but it could much larger.
We can also redirect the standard output from the sed command into a file.
# sed -e 'd' /tmp/passwd > /tmp/new-passwd
Example 3: Printing lines using sed (-n flag & p command)
The ‘-n’ flag disables the automatic printing so that sed will instead print lines only when it is explicitly told to do so with the ‘p’ command.
# cat /tmp/passwd | sed 'p' | head -5 root:x:0:0:root:/root:/bin/bash root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin bin:x:2:2:bin:/bin:/usr/sbin/nologin
As we see above if we specify ‘p’ editing command without ‘-n’ flag , sed will print duplicate lines. So to display unique lines use ‘-n’ flag with p command in sed. Examples is shown below:
# cat /tmp/passwd | sed -n 'p' | head -5 root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin bin:x:2:2:bin:/bin:/usr/sbin/nologin sys:x:3:3:sys:/dev:/usr/sbin/nologin sync:x:4:65534:sync:/bin:/bin/sync
Example 4: Editing the Source file by using ‘-i’ option
Sed command by default does not edit the orginal or source file for our safety but by using ‘-i’ option source file can be edited.
# sed -i '1d' /tmp/passwd
Above command will delete the first line of source file /tmp/passwd.
Example 5: Take backup of source file prior to editing
When we use ‘-i’ option in sed command then it becomes very risky because it will directly edit the source file, so it is better to take the backup of source file before editing, example is shown below.
# sed -i.bak '1d' /tmp/passwd
# ls -l /tmp/passwd* -rw-r--r-- 1 root root 2229 Nov 24 22:36 /tmp/passwd -rw-r--r-- 1 root root 2261 Nov 24 22:35 /tmp/passwd.bak
In the above sed command, 1st line of file /tmp/passwd will be deleted but before that sed command takes the backup of /tmp/passwd as /tmp/passwd.bak.
Example 6: Deleting the lines by specifying range.
Deleting first 5 lines of /tmp/passwd file.
# cat /tmp/passwd | sed '1,5d'
Example 7: Delete the empty lines of a file
# cat /tmp/detail.txt 245 678 linux unix suse
In details.txt file we have two empty lines, use below command to delete empty lines.
# sed '/^$/d' /tmp/detail.txt 245 678 linux unix suse
Example 8: Deleting the lines containing the strings
Suppose we want to delete line from the file /tmp/passwd which contains ‘games’ word.
# sed '/games/d' /tmp/passwd
Example 9: Search and Replace strings in the file
Suppose you want to replace the ‘root’ with ‘Admin’, example is shown below:
# sed 's/root/Admin/' /tmp/passwd
It is very important to note that sed substitutes only the first occurrence on a line. If the string ‘root’ occurs more than once on a line, only the first match will be replaced. To replace every string in the file with the new one instead of just the first occurrence, to make substitution globally add a letter ‘g’ to end of command as shown below:
# sed 's/root/Admin/g' /tmp/passwd
Example 10: Multiple substitution using -e option
Suppose we want to replace ‘root’ string with ‘Admin’ and ‘bash’ string with ‘sh’. Example is shown below:
# cat /tmp/passwd | sed -e 's/root/Admin/g' -e 's/bash/sh/g'