≡ Menu

Unix Sed Tutorial: Find and Replace Text Inside a File Using RegEx

Sed Examples for Unix and Linux - Find and ReplaceThis article is part of on-going Unix Sed Tutorial series. In previous articles, we discussed about sed print operation and sed delete operation.

In this article let us review how to use sed substitute command “s”.

The `s’ command is probably the most important in `sed’ and has a lot of different options.

The `s’ command attempts to match the pattern space against the supplied REGEXP; if the match is successful, then that portion of the pattern space which was matched is replaced with REPLACEMENT.

Syntax:

#sed 'ADDRESSs/REGEXP/REPLACEMENT/FLAGS' filename
#sed 'PATTERNs/REGEXP/REPLACEMENT/FLAGS' filename
  • s is substitute command
  • / is a delimiter
  • REGEXP is regular expression to match
  • REPLACEMENT is a value to replace

FLAGS can be any of the following

  • g Replace all the instance of REGEXP with REPLACEMENT
  • n Could be any number,replace nth instance of the REGEXP with REPLACEMENT.
  • p If substitution was made, then prints the new pattern space.
  • i match REGEXP in a case-insensitive manner.
  • w file If substitution was made, write out the result to the given file.
  • We can use different delimiters ( one of @ % ; : ) instead of /

Let us first create thegeekstuff.txt file that will be used in all the examples mentioned below.

$ cat thegeekstuff.txt
# Instruction Guides
1. Linux Sysadmin, Linux Scripting etc.
2. Databases - Oracle, mySQL etc.
3. Security (Firewall, Network, Online Security etc)
4. Storage in Linux
5. Productivity (Too many technologies to explore, not much time available)
#  Additional FAQS
6. Windows- Sysadmin, reboot etc.

Let us review some interesting examples for substitution now.

1. Substitute Word “Linux” to “Linux-Unix” Using sed s//

In the example below, in the output line “1. Linux-Unix Sysadmin, Linux Scripting etc” only first Linux is replaced by Linux-Unix. If no flags are specified the first match of line is replaced.

$ sed 's/Linux/Linux-Unix/' thegeekstuff.txt
# Instruction Guides
1. Linux-Unix Sysadmin, Linux Scripting etc.
2. Databases - Oracle, mySQL etc.
3. Security (Firewall, Network, Online Security etc)
4. Storage in Linux-Unix
5. Productivity (Too many technologies to explore, not much time available)
#  Additional FAQS
6. Windows- Sysadmin, reboot etc.

2. Substitute all Appearances of a Word Using sed s//g

The below sed command replaces all occurrences of Linux to Linux-Unix using global substitution flag “g”.

$ sed 's/Linux/Linux-Unix/g' thegeekstuff.txt
# Instruction Guides
1. Linux-Unix Sysadmin, Linux-Unix Scripting etc.
2. Databases - Oracle, mySQL etc.
3. Security (Firewall, Network, Online Security etc)
4. Storage in Linux-Unix
5. Productivity (Too many technologies to explore, not much time available)
#  Additional FAQS
6. Windows- Sysadmin, reboot etc.

3. Substitute Only 2nd Occurrence of a Word Using sed s//2

In the example below, in the output line “1. Linux Sysadmin, Linux-Unix Scripting etc.” only 2nd occurance of Linux is replaced by Linux-Unix.

$ sed 's/Linux/Linux-Unix/2' thegeekstuff.txt
# Instruction Guides
1. Linux Sysadmin, Linux-Unix Scripting etc.
2. Databases - Oracle, mySQL etc.
3. Security (Firewall, Network, Online Security etc)
4. Storage in Linux
5. Productivity (Too many technologies to explore, not much time available)
#  Additional FAQS
6. Windows- Sysadmin, reboot etc.

4. Write Changes to a File and Print the Changes Using sed s//gpw

The example below has substitution with three flags. It substitutes all the occurance of Linux to Linux-Unix and prints the substituted output as well as written the same to the given the file.

$ sed -n 's/Linux/Linux-Unix/gpw output' thegeekstuff.txt
1. Linux-Unix Sysadmin, Linux-Unix Scripting etc.
4. Storage in Linux-Unix
$ cat output
1. Linux-Unix Sysadmin, Linux-Unix Scripting etc.
4. Storage in Linux-Unix

5. Substitute Only When the Line Matches with the Pattern Using sed

In this example, if the line matches with the pattern “-“, then it replaces all the characters from “-” with the empty.

$ sed '/\-/s/\-.*//g' thegeekstuff.txt
# Instruction Guides
1. Linux Sysadmin, Linux Scripting etc.
2. Databases
3. Security (Firewall, Network, Online Security etc)
4. Storage in Linux
5. Productivity (Too many technologies to explore, not much time available)
#  Additional FAQS
6. Windows

6. Delete Last X Number of Characters From Each Line Using sed

This sed example deletes last 3 characters from each line.

$ sed 's/...$//' thegeekstuff.txt
# Instruction Gui
1. Linux Sysadmin, Linux Scripting e
2. Databases - Oracle, mySQL e
3. Security (Firewall, Network, Online Security e
4. Storage in Li
5. Productivity (Too many technologies to explore, not much time availab
#  Additional F
6. Windows- Sysadmin, reboot e

7. Eliminate Comments Using sed

Delete all the comment lines from a file as shown below using sed command.

$  sed -e 's/#.*//' thegeekstuff.txt

1. Linux Sysadmin, Linux Scripting etc.
2. Databases - Oracle, mySQL etc.
3. Security (Firewall, Network, Online Security etc)
4. Storage in Linux
5. Productivity (Too many technologies to explore, not much time available)

6. Windows- Sysadmin, reboot etc.

8. Eliminate Comments and Empty Lines Using sed

In this example, there are two commands seperated by ‘;’

  • First command replaces the lines starting with the # to the blank lines
  • Second command deletes the empty lines.
$ sed -e 's/#.*//;/^$/d'  thegeekstuff.txt
1. Linux Sysadmin, Linux Scripting etc.
2. Databases - Oracle, mySQL etc.
3. Security (Firewall, Network, Online Security etc)
4. Storage in Linux
5. Productivity (Too many technologies to explore, not much time available)
6. Windows- Sysadmin, reboot etc.

9. Convert DOS newlines (CR/LF) to Unix format Using sed

Copy the DOS file to Unix, you could find \r\n in the end of each line.

This example converts the DOS file format to Unix file format using sed command.

$sed 's/.$//' filename

10. Eliminate HTML Tags from file Using sed

In this example, the regular expression given in the sed command matches the html tags and replaces with the empty.

$ sed -e 's/<[^>]*>//g'
This <b> is </b> an <i>example</i>.
This  is  an example.
Add your comment

If you enjoyed this article, you might also like..

  1. 50 Linux Sysadmin Tutorials
  2. 50 Most Frequently Used Linux Commands (With Examples)
  3. Top 25 Best Linux Performance Monitoring and Debugging Tools
  4. Mommy, I found it! – 15 Practical Linux Find Command Examples
  5. Linux 101 Hacks 2nd Edition eBook Linux 101 Hacks Book

Bash 101 Hacks Book Sed and Awk 101 Hacks Book Nagios Core 3 Book Vim 101 Hacks Book

Comments on this entry are closed.

  • Marcus Rhodes September 30, 2009, 7:47 am

    How would one rearrange the columns output by ls -al?

    On AIX, which lacks gnu/Linux’s excellent features, I need ls -al to output …

    The filename -rwxrwxrwx 1 root root 12345678 …

    … instead of …

    -rwxrwxrwx 1 root root 12345678 Jan 01 2009 The filename

    Thanks!

  • Paul M September 30, 2009, 11:15 am

    Ramesh,

    I am getting so much benefit from these articles. Thank you! Have you had any luck with making “Printer Friendly” versions of these pages? I would love to be able to print them out and write down my own notes on them.

    Paul

  • Jimi September 30, 2009, 1:25 pm

    Hi,

    there seems to be some typo in example 10. I am getting this output line, which obviously is not what you intented:

    This <b is </b an <iexample</i

    I could not come up with the correct version myself. Can you help?

  • Sasikala September 30, 2009, 11:04 pm

    @Marcus Rhodes,

    Hope this helps,
    $ ls -al The\ Geek\ Stuff | sed -e ‘s/\(\([^ ]* \+\)\{8\}\)\(.*\)$/\3 \1/g’
    The Geek Stuff -rw-r–r– 1 user group 0 Sep 30 21:50

    @Jimi,

    Thanks for catching the typo.
    Example: 10 command should be
    $ sed -e ‘s/<[^>]*>//g’
    Its for removing simple html tags.

  • amksep October 1, 2009, 1:38 am

    for example 10 i think the right syntax is
    sed -e ‘s/]*>//g’

  • amksep October 1, 2009, 1:39 am

    sorry you are right Jimi.

  • Nasrin October 1, 2009, 8:58 am

    @ Sasikala

    $ sed -e ’s/]*>//g’
    bash: [^: No such file or directory

    this also is not working 🙁

  • Ramesh Natarajan October 2, 2009, 10:19 am

    @Paul,

    Print Friendly option is implemented. You’ll see this at the bottom of all articles. Thanks.

  • Marcus Rhodes October 2, 2009, 1:31 pm

    Hmmm… works on Ubuntu, though the filenames need padding, but has no effect on AIX.

  • Jimi October 8, 2009, 1:48 pm

    I did some googling on Example 10, and found this solution, which seems to work:

    sed -n ‘/^$/!{s/]*>//g;p;}’

    It’s from this site:

    http://www.unix.com/linux/45584-how-remove-only-html-tags-inside-file.html

  • Srinivas December 27, 2010, 1:35 pm

    Excellent! Followed all articles, all are well presented.

  • Piyush January 18, 2011, 3:01 am

    Hi,

    Can i use two or more find and replace at a time using sed.

  • rishi January 25, 2011, 1:45 am

    Hi,
    I am trying to write a script.
    In /var/log/messages i have to delete lines older than 5days when compared to the present date. Could you people help me out in this script.
    Thanks

  • Mui June 26, 2011, 7:31 pm

    Hi,

    After i type in the command, eg sed ‘s/Linux/Linux-Unix/’ thegeekstuff.txt ,

    I type vi thegeekstuff.txt , it appears that the old file remains. Thus, do i still need to type in anything to save the changes in thegeekstuff.txt ??

    Thanks in advance.

  • Brad October 29, 2011, 3:15 am

    @Mui,

    You need to direct the output to another file; otherwise it is written to standard output, i.e. the terminal screen. Hence:

    sed ‘s/Linux/Linux-Unix/’ thegeekstuff.txt > new_file.txt

    Also want to thank the blogger for this site and its edifying content.

  • Ivan Neva February 8, 2012, 9:28 am

    Hi,

    This is a very useful tutorial. I’m actually facing a problem with the pattern search, and haven’t been able to figure it out. I have a file I need to load to a database, and for that I need to insert a special character when I find the following sequence of characters: double quotes – end of line – double quotes (“\n”). I need to insert a @ symbol between the first double quotes and the end-of-line (“@\n”). The \n is of course the end-of-line character. So far I’m only been able to insert the character using: (“$), sed ‘s/\”$/&@/g’ filename, but sometimes I need to make sure the next line starts with the double quotes character (“). Is there any way to do it with SED pattern matching? I have tried with different combinations, but without any luck.

    Thanks in advance, great blog by the way.

    Ivan Neva
    Oracle DBA

  • Joshua Hurst February 11, 2012, 12:34 pm

    Ivan,
    sed works line by line. There is no sed option that I know of to do what you are looking for.

  • chinna February 20, 2012, 3:11 pm

    chinna,

    i have file like
    raja
    ravi
    kiran

    i want to add space at the beginning of every line.help me

  • chinna February 22, 2012, 8:41 am

    chinna,

    example 10
    i have file like this

    try this using sed
    $ sed -e ‘s/\([]\)//g’
    oupput is
    linux
    redhat
    ubuntu
    thank you

  • chinna February 22, 2012, 8:45 am

    chinna,

    example 10
    i have file like this

    try this using sed
    $ sed -e ‘s/\([]\)//g’
    oupput is
    linux
    redhat
    ubuntu
    ******************************************** thank you

  • Mark March 22, 2012, 6:38 am

    Hello. I saw how I can replace a word’s first occurence in each line. But my question is: how can I replace a word only if it is the very first one in a line?
    for example: I want to replace ‘pine’ with ‘apple’, and I have the following text:

    pine cat pine car tag can
    cat pine tag car

    here, the word ‘pine’ from the second line should remain unchanged, as well as the second occurence in the first line. The result would be:

    apple cat pine car tag can
    cat pine tag car

  • Anonymous March 28, 2012, 7:21 am

    mark using the $ sign means at the beginning of a line, so

    sed ‘s/$pine/apple/’ should do it

  • Anonymous March 28, 2012, 2:16 pm

    Anonymous, the dollar sign means the end of a line and the carret means the beginning.

  • Priya April 13, 2012, 11:01 pm

    sed ‘s/^pine/apple/’ filename

  • Senthil April 24, 2012, 9:46 am

    Hi,
    I have an xml file with some properties and I want to replace a value based on the value from other tag. I tried with perl and its working. But I want to do it with ‘sed’. please help.
    example xml file:

    nameA
    valueA

    nameB
    valueB

    nameC
    valueC

    In the above example, I want to change the value of valueB to user input data based on nameB. The hint is “nameB” is default and will not change. But the valueB can be different even before user input. So, the sed command should find “nameB” property and then modify tag for it.

  • Mahesh May 30, 2012, 12:24 am

    I have file like this and there are many occurances of same parameters.

    8109 # CHI
    {
    DELAY_TIME 1;
    }
    Now i want to change that 1 to 15, but i just cannot find for DELAY_TIME and replace because DELAY_TIME occurs many times. Is there a way i can get this done?

  • Parthasarathi Dash June 14, 2012, 3:05 am

    Try with this command.this is very simple
    :%s/search_string/replacement_string/g

  • Erik H. June 26, 2012, 3:45 pm

    Hi

    I have this txt-file, containing;

    1418W CROMAX ADJUSTER;;588,01
    1420W CROLIDT TONEFARVE;;841
    1421W CIPAX WHATEVER TONESYSTEM;;462,57

    I need to strip it a bit.
    How do i remove anything between (and includinng) the first “space” and until the first “;”

    So i’m left with a file like;

    1418W;;588,01
    1420W;;841
    1421W;;462,57

  • Prem July 10, 2012, 7:25 am

    Hi,

    Can anyone pls let me know your idea for this scenario:
    Ex: <![CDUMP welcome to unix world $sorry#
    line 12
    ! Whatever we have is correct

    Req: I want to remove newline characters in between and

    Thanks in advance,
    Prem.

  • Prem July 10, 2012, 7:26 am

    Sorry i missed to specify at end of line.

    Ex: <![CDUMP welcome to unix world $sorry#
    line 12
    ! Whatever we have is correct

  • Monty Gaither August 3, 2012, 10:18 am

    I need to change BART that is in every .sh module in a directory to CART. Would the following work?

    $ sed -g -p ‘s/BART/CART/g’ *.*

  • Jay September 3, 2012, 2:10 am

    In the first example, on the 4th line it should be “4. Storage in Linux” and not “4. Storage in Linux-Unix” ?

  • Fernando November 8, 2012, 5:25 pm

    Hi,

    Could you help me please, I have the next problem. I receive a flat file with a lot of lines, each line has data delimited by pipes, for example:
    WD3783C6EA3978A777||26/10/2012 16:24:44||I||CONDUCTOR
    WDE56DAF1870A5CC57||26/10/2012 16:20:01||I||ROSA
    WCFBF376C70921F97B||26/10/2012 16:12:19||I||ERIKA
    WC024ABB31EACB6F1D||26/10/2012 15:58:08||I||JOSE DE JESUS

    The problem is that some rows have two or more CR in the middle of the data, for example:
    WD3783C6EA3978A777||26/10/2012 16:24:44||I||CONDUCTOR
    WDE56DAF1870A5CC57||26/10/2012 16:20:01|

    |I||ROSA
    WCFBF376C70921F97B|
    |26/10/2012 16:12:19||I||ERIKA
    WC024ABB31EACB6F1D||26/10/2012 15:58:08||I||JOSE DE JESUS

    I need replace the CR which appears in the middle of the data with a space ‘ ‘, but the last CR of each line no. This command echo `sed -e ‘s/$/\ |\ /g’ file.txt` replaces all CR with a space, but I dont know what else I have to aggregate for respect the last CR of each line.
    Thanks.

  • sushil November 20, 2012, 4:20 pm

    hi

    in first example ur output in 4th line is linux-unix.
    Is it printing mistake or i’m wrong

  • dude4linux February 23, 2013, 12:29 pm

    Found the answer to my own question. It was a bash shell escape and had nothing to do with sed. The sed expression had to use single quotes in order to preserve the indents and newlines. In bash, the proper way to escape a single quote within single quotes is ‘\”. I was close with the ”’ but no cigar. The working sed command is:
    sed -i -e ‘/ls –color=auto/a \
    alias ll=’\”ls –color=auto -lh’\” \
    alias la=’\”ls –color=auto -lAh’\” \
    alias l=’\”ls –color=auto -CF’\”
    ‘ ~/.bashrc

  • freemen February 26, 2013, 1:58 am

    how about wan to add word infront?

  • Felipe March 12, 2013, 9:31 pm

    Hi, how make this command work (syntax should be wrong):

    sed ‘s/if t > time.time() – 60:/if t > time.time() – 4:/’ /share/HDA_DATA/.qpkg/CouchPotato2/couchpotato/core/plugins/scanner/main.py

    I want change “sed ‘s/if t > time.time() – 60:” to “sed ‘s/if t > time.time() – 4:”

  • Niraj Kumar April 12, 2013, 7:23 am

    Hi,

    Pls help:

    I have data in file like this:
    123,ABC,84,TYPE hkahsd DIG 147,789,TYPE hkahsd DIG 147
    546,AJH,78,DIG 142 asdsdad,890,DIG 142 asdsdad

    Output should be like this:
    123,ABC,84,DIG 147,789,TYPE hkahsd DIG 147
    546,AJH,78,DIG 142,890,DIG 142 asdsdad

  • rusty June 25, 2013, 5:25 am

    what do PATTERNs and ADDRESs stand for in first example?

  • Subhendu Bera August 14, 2013, 12:02 am

    Its very useful. Thanx.

  • shrey August 17, 2013, 8:47 am

    Very helpful tutorials. I love it.
    Looks like a typo in example 1. Not only it replaces the 1st match on line 1 but also again on line 4. Should not be just only 1 replacement (on 1st match)?

    Shrey

  • mallik October 29, 2013, 1:55 am

    The last point u have mentioned is quite interesting. If suppose i want to remove “” tag for the prepositions like is,are, for, like that so how to do that using sed.

  • navin February 2, 2014, 10:12 am

    how to write ^$

  • Badhrish March 18, 2014, 4:44 am

    How to save the changes inline to the file after substitution made using sed

  • Eric June 23, 2014, 2:38 pm

    how about like edit php.ini
    you want to change the variable in there memory_limit
    if it is not large enough
    memory_limit = -1 or memory_limit = 128M and you want it to be
    memory_limit = 768M
    in a script

  • Connor May 28, 2015, 3:43 pm

    Shrey is correct; in addition to the substitution on line 1, there is an extra “Linux” -> “Linux-Unix” substitution on line 4 of example 1. IT should only replace the first occurrence on line 1, not the occurrence on line 4.

  • vijay July 28, 2015, 10:00 am

    hi can some one help me pls,

    i have requirement to change a key value pair in a file : like
    1.This is first=True
    2.This is second=True.

    i am trying to change only “1” value to false
    but the second “2” is also changing
    i am trying :
    sed -i -e ‘This is first s/true/false/’ myfilename.txt

    the value is changing for both ..how can i get value changed for one only

  • Anonymous October 23, 2015, 12:18 pm

    what to do when the file,the word to be searched in this file and that to be subsituted with in this file is taken from user within the script???

  • Cees Timmerman February 10, 2016, 3:40 am

    #9 looks wrong. “$sed ‘s/\r\n$/\n/’ filename” should be the safe alternative that doesn’t convert to Mac linebreaks.

  • marcus coleman April 9, 2016, 5:36 pm

    Hello this is a great article! I could not locate the answer to my problem

    # Never forward plain names (without a dot or domain part)
    #domain-needed
    # Never forward addresses in the non-routed address spaces.
    #bogus-priv

    sed ‘s/#domain-needed/domain-needed/g’ filename.conf
    sed s/#domain-needed/domain-needed/g filename.conf

    sed ‘s/#domain-needed/domain-needed/’ > filename.conf
    sed s/#domain-needed/domain-needed/g’ /path/to/filename.conf

    sed -n ‘s/#domain-needed/domain-needed/gpw output’ dnsmasq.conf
    bogus-priv
    but the file is still unchanged when I “CAT” it…

    what am I missing. Your help would be greatly appreciated.

    thanks in advance…

  • Karthick April 10, 2016, 10:22 am

    Hi Please help me here,

    I want to delete “/opt/app/bss/sieb/fs” character from a file at All occurrence and I need to replace with “/webmaster”. Need syntax

  • arun January 1, 2017, 7:59 am

    hi,

    am trying to delete all empty lines starting from a particular line number. but its not working as expected. Can anyone help me out.

    [root@localhost ~]# nl passwd
    1 root:x:0:0:root:/root:/bin/bash
    2 bin:x:1:1:bin:/bin:/sbin/nologin
    3 daemon:x:2:2:daemon:/sbin:/sbin/nologin

    4 adm:x:3:4:adm:/var/adm:/sbin/nologin

    5 lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

    [root@localhost ~]# sed -e ‘3,/^$/’d passwd
    root:x:0:0:root:/root:/bin/bash
    bin:x:1:1:bin:/bin:/sbin/nologin
    adm:x:3:4:adm:/var/adm:/sbin/nologin

    lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

    [root@localhost ~]# sed -e ‘3,$ /^$/’d passwd
    sed: -e expression #1, char 5: unknown command: `/’

  • John January 2, 2017, 10:35 am

    arun,
    Try sed -e ‘3,$ s/^\s*$//;/^$/d’ passwd

    The ‘d’ is part of the expression and must be inside the tick marks. Also you apparently can’t use both address (3,$) and match (/^$/) together. I modified the author’s example to search for and remove whitespace (\s) before deleting the line.
    Good Luck

  • Sasikala February 24, 2017, 5:16 am

    @arun,

    $ sed ‘4,${/^$/d}’ file

    I think this might help you.

  • DirectedSOul March 16, 2017, 3:09 pm

    I think the 10) solution to replace all the html tags are:

    sed “s/]\+>//g”

    Hope it helps.