Unix Sed Tutorial : 7 Examples for Sed Hold and Pattern Buffer Operations

by Sasikala on December 9, 2009

Linux Sed Command ExamplesThis article is part of the on-going Unix Sed Tips and Tricks series.

In our previous sed articles we learned — sed printing, sed deletion, sed substitute , sed file write, sed multiple commands, sed multi-line operation, and sed manipulate file lines.

In our first part of sed tutorial we learned that sed has two buffers — sed pattern buffer and sed hold buffer. So far we have talked only about pattern buffer.

In this article let us review how to use sed hold and pattern buffer using 7 practical sed examples.

As its name implies, sed hold buffer is used to save all or part of the sed pattern space for subsequent retrieval. The contents of the pattern space can be copied to the hold space, then back again. No operations are performed directly on the hold space. sed provides a set of hold and get functions to handle these movements.

Sed h function

The h (hold) function copies the contents of the pattern space into a holding area (also called as sed hold space), destroying any previous contents of the holding area.

Sed H function

The H function appends the contents of the pattern space to the contents of the holding area. The former and new contents are separated by a newline.

Sed g function

The g function copies the contents of the holding area into the pattern space, destroying the previous contents of the pattern space.

Sed G function

The G function appends the contents of the holding area to the contents of the pattern space. The former and new contents are separated by a newline. The maximum number of addresses is two.

Sed x function

The exchange function interchanges the contents of the pattern space and the holding area. The maximum number of addresses is two.

Now let us see some examples to learn about the above commands.

Let us first create thegeekstuff.txt file that will be used in the examples mentioned below.

$ cat thegeekstuff.txt
#Linux
        Administration
        Scripting
        Tips and Tricks

#Windows
        Administration

#Database
        Mysql
        Oracle
        Queries
        Procedures

1. Double Space a File Content Using Sed Command

$sed 'G' thegeekstuff.txt
#Linux

        Administration

        Scripting

        Tips and Tricks

#Windows

        Administration

#Database

        Mysql

        Oracle

        Queries

        Procedures 

$

In this example,

  1. Sed reads a line and places it in the pattern buffer.
  2. G command appends the hold buffer to the pattern buffer separated by \n. so one newline will be appended with the pattern space content.
  3. Similarly, If you want to triple space a file, append hold buffer content to the pattern buffer twice. (G;G)

2. Print File Content in Reverse Order Using Sed Command

Print the lines of a file in reverse order (similar to tac command that we discussed earlier).

$sed -n '1!G;h;$p' thegeekstuff.txt
        Procedures
        Queries
        Oracle
        Mysql
#Database

        Administration
#Windows

        Tips and Tricks
        Scripting
        Administration
#Linux

In this example,

  1. First line will be placed into the hold space as it is.
  2. From the 2nd line onwards, just append the hold space content with the pattern space. (Remember 2nd line is in pattern space, and 1st line is in hold space).
  3. Now 1st and 2nd line got reversed and move this to the hold space.
  4. Repeat the above steps till last line.
  5. Once the last line is reached, just append the hold space content with the pattern space and print the pattern space.

3. Print a Paragraph (Only if it contains given pattern) Using Sed Command

In thegeekstuff.txt print paragraph only if it contains the pattern “Administration”.

$ sed -e '/./{H;$!d;}' -e 'x;/Administration/!d' thegeekstuff.txt 

Linux
        Administration
        Scripting
        Tips and Tricks

Windows
        Administration

In this example,

  1. Till the empty line comes, keep appending the non empty lines into the hold space
  2. When empty line comes i.e paragraph ends, exchange the data between pattern and hold space. So that whole paragraph will be available in pattern space.
    Check if pattern “Administration” is available, if yes don’t delete it i.e print the pattern space

4. Print the line immediately before a pattern match using Sed Command

Print only the line immediately before,the pattern “Mysql”.

$ sed -n '/Mysql/{g;1!p;};h' thegeekstuff.txt
#Database

In this example,

  1. For each cycle, place the line into hold buffer, if it doesn’t match with the pattern “Mysql”.
  2. If the line matches with the pattern, get the data from the hold space(previous line) using g command and print it.
  3. In case, if the first line matches with the pattern “Mysql”,anyway hold space will be empty.(There is no previous line to the first line).So first line should not get printed(1!p)

5. Delete the last line of each paragraph using Sed Command

$ sed -n -e '/^$/{x;d}' -e '/./x;p' thegeekstuff.txt 

#Linux
        Administration
        Scripting

#Windows

#Database
        Mysql
        Oracle
        Queries

In this example,

  1. If the line is not empty,then exchange the line between pattern and hold space. So first line will be placed in the hold space.
  2. When next non empty line comes, exchange the pattern space and hold space, and print the pattern space. i.e first non empty line will be printed and 2nd line goes to hold. And in next cycle, 2nd non empty line is printed when 3rd line goes to hold and goes on like this.
  3. When empty line comes (previous line to the empty line will be available in hold buffer) just exchange pattern and hold space, and delete the line (last line of the paragraph) and start the next cycle.

6. For each line, append the previous line to the end of it using Sed Command

$ sed 'H;x;s/^\(.*\)\n\(.*\)/\2\1/' thegeekstuff.txt
#Linux
        Administration#Linux
        Scripting        Administration
        Tips and Tricks        Scripting
        Tips and Tricks
#Windows
        Administration#Windows
        Administration
#Database
        Mysql#Database
        Oracle        Mysql
        Queries Oracle
        Procedures                      Queries

In this example,

  1. Place the first line in Hold buffer.
  2. When the second line comes, append to Hold space (first line)
  3. Then exchange pattern and hold buffer. So now pattern space will have first and second line separated by \n, Hold space will have only second line.
  4. So interchange the lines in the pattern space.
  5. The above steps happens till the end of the file

7. Prepend tag of every block to every line of that block

$ sed '
/^#/{
h
d
}
G
s/^\(.*\)\n#\(.*\)/\2 \1/' thegeekstuff.txt
Linux         Administration
Linux         Scripting
Linux         Tips and Tricks
Linux
Windows         Administration
Windows
Database         Mysql
Database        Oracle
Database        Queries
Database        Procedures

In this example,

  1. When the first line of a block is met (beginning with #)
    • keep that line to the Hold Space via command `h’
    • Then delete using ‘d’ to start another cycle.
  2. For the rest lines of a block, Command `G’ appends the tag line from the Hold Space and substitute command interchanges tag and lines properly.

Linux Sysadmin Course Linux provides several powerful administrative tools and utilities which will help you to manage your systems effectively. If you don’t know what these tools are and how to use them, you could be spending lot of time trying to perform even the basic administrative tasks. The focus of this course is to help you understand system administration tools, which will help you to become an effective Linux system administrator.
Get the Linux Sysadmin Course Now!

If you enjoyed this article, you might also like..

  1. 50 Linux Sysadmin Tutorials
  2. 50 Most Frequently Used Linux Commands (With Examples)
  3. Top 25 Best Linux Performance Monitoring and Debugging Tools
  4. Mommy, I found it! – 15 Practical Linux Find Command Examples
  5. Linux 101 Hacks 2nd Edition eBook Linux 101 Hacks Book

Bash 101 Hacks Book Sed and Awk 101 Hacks Book Nagios Core 3 Book Vim 101 Hacks Book

{ 13 comments… read them below or add one }

1 Ron December 9, 2009 at 9:31 am

Hi Sasikala.
Thank You for excellent article.

2 Mahesh December 9, 2009 at 10:34 am

Thanks, those directives makes sed a very powerful tool.

3 Ramesh Natarajan December 10, 2009 at 5:40 pm

@Ron, @Mahesh,

Thanks for your comments. Sed can be very powerful tool, when we understand and use all it’s features properly. I’m very glad you found this article helpful.

4 Ace April 30, 2010 at 3:00 pm

This is brilliant article thanks for the tip.

I am looking for some tips on putting alpha character in sequetial order at the beginning of a line. Any help would be great.

5 rajendra prasad.d August 25, 2010 at 2:11 am

hai sir ,
iam very happy about this article it is very useful in my teaching experience

6 Hi November 25, 2010 at 3:02 am

what does $!d mean in the 3th example ?

7 Gregor January 26, 2011 at 7:33 am

On the 3rd example it means don’t delete last line ($!d)
$ – means last line

8 Harsh February 13, 2012 at 4:33 am

I want to use sed to delete part of code (paragraph) beginning with a pattern and ending with a semicolon (;)

Example:

Pattern .. ;
or
Pattern …
… ;

Now I came across an example to delete a paragraph separated by new lines
sed -e ‘/./{H;$!d;}’ -e ‘x;/Pattern/!d’
(which means put every non empty line in hold buffer, then if pattern exists delete it)

I’m confused how to use semicolon not as a delimiter but as a part of pattern instead.

9 Satheesh Chandra April 29, 2012 at 9:59 am

Harsh,
Assuming the file looks like this in the sample given below.

$ cat crt_tabl.sql
create table test_table
(
emp_id int not null primary key,
emp_name varchar(30),
gender char(1),
dob date,
address varchar(30)
);
alter table test_table add constraint ak_uq1 unique (emp_id, emp_name, gender);
create unique index ui_idx1 on test_table (emp_id, emp_name, gender);

If you want to delete only the “create table …. ;” part,

$ sed -n ‘/create table/,/;/ !p’ crt_tabl.sql
alter table test_table add constraint ak_uq1 unique (emp_id, emp_name, gender);
create unique index ui_idx1 on test_table (emp_id, emp_name, gender);

OR

$ sed ‘/create table/,/;/d’ crt_tabl.sql
alter table test_table add constraint ak_uq1 unique (emp_id, emp_name, gender);
create unique index ui_idx1 on test_table (emp_id, emp_name, gender);

10 Haseeb May 21, 2012 at 2:35 am

Hi,
I have a config file with may scripts eg:
#script1
………………………
#script2
………………………

I need a script wich can comment or un comment the script with using the header #script1 or #script2

Thanks,
Haseeb

11 lancerex July 23, 2013 at 8:11 pm

thanks

12 Hi there December 31, 2013 at 4:00 am

I’m trying (and failing miserably) to do a variation of example 3. I want to delete a stanza that contains a string. each stanza starts with “name … {” and end with “}”
for example to delete the stanza below that contains “string3″

the structure of the file is similar to this

name some_name {
val1 string1
val2 string2
}
name other_name {
val1 string3
val2 string4
}
name yet_another_name {
val1 string5
val2 string6
}

13 Ajmal Hamza July 16, 2014 at 1:44 am

Re: 2. Print File Content in Reverse Order Using Sed Command

In the flow,
1. First line will be placed into the hold space as it is.

Shouldn’t it be like “first line will be placed to PATTERN BUFFER ?
This point confused me to the point of going mad, repeatedly reading for 2 days and trying to understand :)

Leave a Comment

Previous post:

Next post: