≡ Menu

Unix Sed Tutorial : 7 Examples for Sed Hold and Pattern Buffer Operations

Linux Sed Command ExamplesThis article is part of the on-going Unix Sed Tips and Tricks series.

In our previous sed articles we learned — sed printing, sed deletion, sed substitute , sed file write, sed multiple commands, sed multi-line operation, and sed manipulate file lines.

In our first part of sed tutorial we learned that sed has two buffers — sed pattern buffer and sed hold buffer. So far we have talked only about pattern buffer.

In this article let us review how to use sed hold and pattern buffer using 7 practical sed examples.

As its name implies, sed hold buffer is used to save all or part of the sed pattern space for subsequent retrieval. The contents of the pattern space can be copied to the hold space, then back again. No operations are performed directly on the hold space. sed provides a set of hold and get functions to handle these movements.

Sed h function

The h (hold) function copies the contents of the pattern space into a holding area (also called as sed hold space), destroying any previous contents of the holding area.

Sed H function

The H function appends the contents of the pattern space to the contents of the holding area. The former and new contents are separated by a newline.

Sed g function

The g function copies the contents of the holding area into the pattern space, destroying the previous contents of the pattern space.

Sed G function

The G function appends the contents of the holding area to the contents of the pattern space. The former and new contents are separated by a newline. The maximum number of addresses is two.

Sed x function

The exchange function interchanges the contents of the pattern space and the holding area. The maximum number of addresses is two.

Now let us see some examples to learn about the above commands.

Let us first create thegeekstuff.txt file that will be used in the examples mentioned below.

$ cat thegeekstuff.txt
#Linux
        Administration
        Scripting
        Tips and Tricks

#Windows
        Administration

#Database
        Mysql
        Oracle
        Queries
        Procedures

1. Double Space a File Content Using Sed Command

$sed 'G' thegeekstuff.txt
#Linux

        Administration

        Scripting

        Tips and Tricks

#Windows

        Administration

#Database

        Mysql

        Oracle

        Queries

        Procedures 

$

In this example,

  1. Sed reads a line and places it in the pattern buffer.
  2. G command appends the hold buffer to the pattern buffer separated by \n. so one newline will be appended with the pattern space content.
  3. Similarly, If you want to triple space a file, append hold buffer content to the pattern buffer twice. (G;G)

2. Print File Content in Reverse Order Using Sed Command

Print the lines of a file in reverse order (similar to tac command that we discussed earlier).

$sed -n '1!G;h;$p' thegeekstuff.txt
        Procedures
        Queries
        Oracle
        Mysql
#Database

        Administration
#Windows

        Tips and Tricks
        Scripting
        Administration
#Linux

In this example,

  1. First line will be placed into the hold space as it is.
  2. From the 2nd line onwards, just append the hold space content with the pattern space. (Remember 2nd line is in pattern space, and 1st line is in hold space).
  3. Now 1st and 2nd line got reversed and move this to the hold space.
  4. Repeat the above steps till last line.
  5. Once the last line is reached, just append the hold space content with the pattern space and print the pattern space.

3. Print a Paragraph (Only if it contains given pattern) Using Sed Command

In thegeekstuff.txt print paragraph only if it contains the pattern “Administration”.

$ sed -e '/./{H;$!d;}' -e 'x;/Administration/!d' thegeekstuff.txt 

Linux
        Administration
        Scripting
        Tips and Tricks

Windows
        Administration

In this example,

  1. Till the empty line comes, keep appending the non empty lines into the hold space
  2. When empty line comes i.e paragraph ends, exchange the data between pattern and hold space. So that whole paragraph will be available in pattern space.
    Check if pattern “Administration” is available, if yes don’t delete it i.e print the pattern space

4. Print the line immediately before a pattern match using Sed Command

Print only the line immediately before,the pattern “Mysql”.

$ sed -n '/Mysql/{g;1!p;};h' thegeekstuff.txt
#Database

In this example,

  1. For each cycle, place the line into hold buffer, if it doesn’t match with the pattern “Mysql”.
  2. If the line matches with the pattern, get the data from the hold space(previous line) using g command and print it.
  3. In case, if the first line matches with the pattern “Mysql”,anyway hold space will be empty.(There is no previous line to the first line).So first line should not get printed(1!p)

5. Delete the last line of each paragraph using Sed Command

$ sed -n -e '/^$/{x;d}' -e '/./x;p' thegeekstuff.txt 

#Linux
        Administration
        Scripting

#Windows

#Database
        Mysql
        Oracle
        Queries

In this example,

  1. If the line is not empty,then exchange the line between pattern and hold space. So first line will be placed in the hold space.
  2. When next non empty line comes, exchange the pattern space and hold space, and print the pattern space. i.e first non empty line will be printed and 2nd line goes to hold. And in next cycle, 2nd non empty line is printed when 3rd line goes to hold and goes on like this.
  3. When empty line comes (previous line to the empty line will be available in hold buffer) just exchange pattern and hold space, and delete the line (last line of the paragraph) and start the next cycle.

6. For each line, append the previous line to the end of it using Sed Command

$ sed 'H;x;s/^\(.*\)\n\(.*\)/\2\1/' thegeekstuff.txt
#Linux
        Administration#Linux
        Scripting        Administration
        Tips and Tricks        Scripting
        Tips and Tricks
#Windows
        Administration#Windows
        Administration
#Database
        Mysql#Database
        Oracle        Mysql
        Queries Oracle
        Procedures                      Queries

In this example,

  1. Place the first line in Hold buffer.
  2. When the second line comes, append to Hold space (first line)
  3. Then exchange pattern and hold buffer. So now pattern space will have first and second line separated by \n, Hold space will have only second line.
  4. So interchange the lines in the pattern space.
  5. The above steps happens till the end of the file

7. Prepend tag of every block to every line of that block

$ sed '
/^#/{
h
d
}
G
s/^\(.*\)\n#\(.*\)/\2 \1/' thegeekstuff.txt
Linux         Administration
Linux         Scripting
Linux         Tips and Tricks
Linux
Windows         Administration
Windows
Database         Mysql
Database        Oracle
Database        Queries
Database        Procedures

In this example,

  1. When the first line of a block is met (beginning with #)
    • keep that line to the Hold Space via command `h’
    • Then delete using ‘d’ to start another cycle.
  2. For the rest lines of a block, Command `G’ appends the tag line from the Hold Space and substitute command interchanges tag and lines properly.
Add your comment

If you enjoyed this article, you might also like..

  1. 50 Linux Sysadmin Tutorials
  2. 50 Most Frequently Used Linux Commands (With Examples)
  3. Top 25 Best Linux Performance Monitoring and Debugging Tools
  4. Mommy, I found it! – 15 Practical Linux Find Command Examples
  5. Linux 101 Hacks 2nd Edition eBook Linux 101 Hacks Book

Bash 101 Hacks Book Sed and Awk 101 Hacks Book Nagios Core 3 Book Vim 101 Hacks Book

Comments on this entry are closed.

  • Ron December 9, 2009, 9:31 am

    Hi Sasikala.
    Thank You for excellent article.

  • Mahesh December 9, 2009, 10:34 am

    Thanks, those directives makes sed a very powerful tool.

  • Ramesh Natarajan December 10, 2009, 5:40 pm

    @Ron, @Mahesh,

    Thanks for your comments. Sed can be very powerful tool, when we understand and use all it’s features properly. I’m very glad you found this article helpful.

  • Ace April 30, 2010, 3:00 pm

    This is brilliant article thanks for the tip.

    I am looking for some tips on putting alpha character in sequetial order at the beginning of a line. Any help would be great.

  • rajendra prasad.d August 25, 2010, 2:11 am

    hai sir ,
    iam very happy about this article it is very useful in my teaching experience

  • Hi November 25, 2010, 3:02 am

    what does $!d mean in the 3th example ?

  • Gregor January 26, 2011, 7:33 am

    On the 3rd example it means don’t delete last line ($!d)
    $ – means last line

  • Harsh February 13, 2012, 4:33 am

    I want to use sed to delete part of code (paragraph) beginning with a pattern and ending with a semicolon (;)

    Example:

    Pattern .. ;
    or
    Pattern …
    … ;

    Now I came across an example to delete a paragraph separated by new lines
    sed -e ‘/./{H;$!d;}’ -e ‘x;/Pattern/!d’
    (which means put every non empty line in hold buffer, then if pattern exists delete it)

    I’m confused how to use semicolon not as a delimiter but as a part of pattern instead.

  • Satheesh Chandra April 29, 2012, 9:59 am

    Harsh,
    Assuming the file looks like this in the sample given below.

    $ cat crt_tabl.sql
    create table test_table
    (
    emp_id int not null primary key,
    emp_name varchar(30),
    gender char(1),
    dob date,
    address varchar(30)
    );
    alter table test_table add constraint ak_uq1 unique (emp_id, emp_name, gender);
    create unique index ui_idx1 on test_table (emp_id, emp_name, gender);

    If you want to delete only the “create table …. ;” part,

    $ sed -n ‘/create table/,/;/ !p’ crt_tabl.sql
    alter table test_table add constraint ak_uq1 unique (emp_id, emp_name, gender);
    create unique index ui_idx1 on test_table (emp_id, emp_name, gender);

    OR

    $ sed ‘/create table/,/;/d’ crt_tabl.sql
    alter table test_table add constraint ak_uq1 unique (emp_id, emp_name, gender);
    create unique index ui_idx1 on test_table (emp_id, emp_name, gender);

  • Haseeb May 21, 2012, 2:35 am

    Hi,
    I have a config file with may scripts eg:
    #script1
    ………………………
    #script2
    ………………………

    I need a script wich can comment or un comment the script with using the header #script1 or #script2

    Thanks,
    Haseeb

  • lancerex July 23, 2013, 8:11 pm

    thanks

  • Hi there December 31, 2013, 4:00 am

    I’m trying (and failing miserably) to do a variation of example 3. I want to delete a stanza that contains a string. each stanza starts with “name … {” and end with “}”
    for example to delete the stanza below that contains “string3”

    the structure of the file is similar to this

    name some_name {
    val1 string1
    val2 string2
    }
    name other_name {
    val1 string3
    val2 string4
    }
    name yet_another_name {
    val1 string5
    val2 string6
    }

  • Ajmal Hamza July 16, 2014, 1:44 am

    Re: 2. Print File Content in Reverse Order Using Sed Command

    In the flow,
    1. First line will be placed into the hold space as it is.

    Shouldn’t it be like “first line will be placed to PATTERN BUFFER ?
    This point confused me to the point of going mad, repeatedly reading for 2 days and trying to understand 🙂

  • Jalal Hajigholamali November 8, 2014, 12:31 am

    Hi,
    Very nice material

    Thanks a lot

  • Octo January 23, 2015, 2:02 am

    Hello,

    Thank you for this article. That’s great. I have a question about changing a line when another line matchs under a specific section of a file (block). Here is a basic example of my file. Extract of MyFic.txt :

    … Some tags …
    X1
    … oThers tags …
    X2
    … Some tags …

    … Some tags …

    … oThers tags …

    … Some tags …

    I would like to change the value of the tag by X3 when the tag Album value is X1 only into the section …

    The following command works fine : sed -i ‘/X1/,// s/.*/X3/’ MyFic.txt

    But this is not exactly what i really want. I would like to make something like that :
    sed -i ‘//,// {Containing X1} s/.*/X3/’ MyFic.txt

    I don’t know how to code using the sed command the following matching pattern : {Containing X1}

    Do you have some idea ?

    Thanks in advance.

    Octo

  • Octo January 23, 2015, 2:04 am

    Xml Tag don’t appear :((