Unix Sed Tutorial: Find and Replace Text Inside a File Using RegEx

This article is part of on-going Unix Sed Tutorial series. In previous articles, we discussed about sed print operation and sed delete operation.

In this article let us review how to use sed substitute command “s”.

The `s’ command is probably the most important in `sed’ and has a lot of different options.

The `s’ command attempts to match the pattern space against the supplied REGEXP; if the match is successful, then that portion of the pattern space which was matched is replaced with REPLACEMENT.

Syntax:

#sed 'ADDRESSs/REGEXP/REPLACEMENT/FLAGS' filename
#sed 'PATTERNs/REGEXP/REPLACEMENT/FLAGS' filename

• s is substitute command
• / is a delimiter
• REGEXP is regular expression to match
• REPLACEMENT is a value to replace

FLAGS can be any of the following

• g Replace all the instance of REGEXP with REPLACEMENT
• n Could be any number,replace nth instance of the REGEXP with REPLACEMENT.
• p If substitution was made, then prints the new pattern space.
• i match REGEXP in a case-insensitive manner.
• w file If substitution was made, write out the result to the given file.
• We can use different delimiters ( one of @ % ; : ) instead of /

Let us first create thegeekstuff.txt file that will be used in all the examples mentioned below.

$ cat thegeekstuff.txt
# Instruction Guides
1. Linux Sysadmin, Linux Scripting etc.
2. Databases - Oracle, mySQL etc.
3. Security (Firewall, Network, Online Security etc)
4. Storage in Linux
5. Productivity (Too many technologies to explore, not much time available)
#  Additional FAQS
6. Windows- Sysadmin, reboot etc.

Let us review some interesting examples for substitution now.

1. Substitute Word “Linux” to “Linux-Unix” Using sed s//

In the example below, in the output line “1. Linux-Unix Sysadmin, Linux Scripting etc” only first Linux is replaced by Linux-Unix. If no flags are specified the first match of line is replaced.

$ sed 's/Linux/Linux-Unix/' thegeekstuff.txt
# Instruction Guides
1. Linux-Unix Sysadmin, Linux Scripting etc.
2. Databases - Oracle, mySQL etc.
3. Security (Firewall, Network, Online Security etc)
4. Storage in Linux-Unix
5. Productivity (Too many technologies to explore, not much time available)
#  Additional FAQS
6. Windows- Sysadmin, reboot etc.

2. Substitute all Appearances of a Word Using sed s//g

The below sed command replaces all occurrences of Linux to Linux-Unix using global substitution flag “g”.

$ sed 's/Linux/Linux-Unix/g' thegeekstuff.txt
# Instruction Guides
1. Linux-Unix Sysadmin, Linux-Unix Scripting etc.
2. Databases - Oracle, mySQL etc.
3. Security (Firewall, Network, Online Security etc)
4. Storage in Linux-Unix
5. Productivity (Too many technologies to explore, not much time available)
#  Additional FAQS
6. Windows- Sysadmin, reboot etc.

3. Substitute Only 2nd Occurrence of a Word Using sed s//2

In the example below, in the output line “1. Linux Sysadmin, Linux-Unix Scripting etc.” only 2nd occurance of Linux is replaced by Linux-Unix.

$ sed 's/Linux/Linux-Unix/2' thegeekstuff.txt
# Instruction Guides
1. Linux Sysadmin, Linux-Unix Scripting etc.
2. Databases - Oracle, mySQL etc.
3. Security (Firewall, Network, Online Security etc)
4. Storage in Linux
5. Productivity (Too many technologies to explore, not much time available)
#  Additional FAQS
6. Windows- Sysadmin, reboot etc.

4. Write Changes to a File and Print the Changes Using sed s//gpw

The example below has substitution with three flags. It substitutes all the occurance of Linux to Linux-Unix and prints the substituted output as well as written the same to the given the file.

$ sed -n 's/Linux/Linux-Unix/gpw output' thegeekstuff.txt
1. Linux-Unix Sysadmin, Linux-Unix Scripting etc.
4. Storage in Linux-Unix
$ cat output
1. Linux-Unix Sysadmin, Linux-Unix Scripting etc.
4. Storage in Linux-Unix

5. Substitute Only When the Line Matches with the Pattern Using sed

In this example, if the line matches with the pattern “-”, then it replaces all the characters from “-” with the empty.

$ sed '/\-/s/\-.*//g' thegeekstuff.txt
# Instruction Guides
1. Linux Sysadmin, Linux Scripting etc.
2. Databases
3. Security (Firewall, Network, Online Security etc)
4. Storage in Linux
5. Productivity (Too many technologies to explore, not much time available)
#  Additional FAQS
6. Windows

6. Delete Last X Number of Characters From Each Line Using sed

This sed example deletes last 3 characters from each line.

$ sed 's/...$//' thegeekstuff.txt
# Instruction Gui
1. Linux Sysadmin, Linux Scripting e
2. Databases - Oracle, mySQL e
3. Security (Firewall, Network, Online Security e
4. Storage in Li
5. Productivity (Too many technologies to explore, not much time availab
#  Additional F
6. Windows- Sysadmin, reboot e

7. Eliminate Comments Using sed

Delete all the comment lines from a file as shown below using sed command.

$  sed -e 's/#.*//' thegeekstuff.txt

1. Linux Sysadmin, Linux Scripting etc.
2. Databases - Oracle, mySQL etc.
3. Security (Firewall, Network, Online Security etc)
4. Storage in Linux
5. Productivity (Too many technologies to explore, not much time available)

6. Windows- Sysadmin, reboot etc.

8. Eliminate Comments and Empty Lines Using sed

In this example, there are two commands seperated by ‘;’

• First command replaces the lines starting with the # to the blank lines
• Second command deletes the empty lines.

$ sed -e 's/#.*//;/^$/d'  thegeekstuff.txt
1. Linux Sysadmin, Linux Scripting etc.
2. Databases - Oracle, mySQL etc.
3. Security (Firewall, Network, Online Security etc)
4. Storage in Linux
5. Productivity (Too many technologies to explore, not much time available)
6. Windows- Sysadmin, reboot etc.

9. Convert DOS newlines (CR/LF) to Unix format Using sed

Copy the DOS file to Unix, you could find \r\n in the end of each line.

This example converts the DOS file format to Unix file format using sed command.

$sed 's/.$//' filename

10. Eliminate HTML Tags from file Using sed

In this example, the regular expression given in the sed command matches the html tags and replaces with the empty.

$ sed -e 's/<[^>]*>//g'
This <b> is </b> an <i>example</i>.
This  is  an example.

If you enjoyed this article, you might also like..