Linux Basics – sed

How to use Linux sed ‘s’ Function

As Linux commands go, this one has a very confusing name. That is, until someone tells you that like many Linux commands, it is yet another abbreviation. This time, sed is short for stream editor. Many editors for text files operate in a sort of point-click-and-save fashion. They cannot edit a real-time stream of data such as a log file. sed, on the other hand, is able to modify text streaming through it in real-time. The sed command is a wonderful utility. The language is very simple, but the documentation is terrible. We asked our friendly Linux guru at HOSTAFRICA to explain it to us. Don’t forget to read our grep command in Linux.

The most important feature of sed

Used for daily tasks specifically, the most useful feature of sed is the ‘s‘ function. The ‘s‘ stands for substitution. The substitute command changes all occurrences of the regular expression into a new value. A simple example, using our sample.txt file from the previous Linux Basics blog (see below) is changing dog in the sample.txt file to mouse in the new.txt file.

First, our sample.txt file:

~$: cat sample.txt 
The quick
brown
fox
jumped over
the lazy
dog
.

Now to use the stream editor sed to change dog to mouse.

~$ cat sample.txt |sed s/dog/mouse/
The quick
brown
fox
jumped over
the lazy
mouse
.

Notice the command output content of the file sample.txt, but we pipe it into the sed command. We then instruct sed to substitute s dog with mouse, which looks as such s/dog/mouse/. We can also do this without the cat command as sed is clever enough to read files without assistance from a cat. Also, note that we have not yet modified anything.

~$ sed s/dog/mouse/ sample.txt 
The quick
brown
fox
jumped over
the lazy
mouse
.

Now we will redirect the output from sed to new.txt.

~$ sed s/dog/mouse/ sample.txt >new.txt
~$ cat new.txt 
The quick
brown
fox
jumped over
the lazy
mouse
.

Check content of sample.txt

~$ cat sample.txt 
The quick
brown
fox
jumped over
the lazy
dog
.

Now you can see, sed stood in the data stream between sample.txt and new.txt and changed dog to mouse. In general, it is better to quote your sed expression like this:

sed 's/dog/mouse/'

SAMPLE

~$ echo "dog"

Output

dog

~$ echo "dog" |sed 's/dog/mouse/'

Output

mouse

Let’s be creative

~$ echo "hounddog" |sed 's/dog/mouse/'

Output

houndmouse

Another important concept is that sed is line oriented. Suppose you have the input file:

one two three, one two three
four three two one
one hundred

and you used the command

~$ sed 's/one/ONE/' <file

The output would be:

ONE two three, one two three
four three two ONE
ONE hundred

Note that this changed one to ONE once on each line. The first line had one twice, but only the first occurrence was changed. That’s the default behaviour. If you want something different, you’ll have to use some of the options that are available.

Let’s return to the command: sed 's/one/ONE/'

There’re four parts to this substitute command:
s                  Substitute command
/../../    Delimiter
one             Regular Expression Pattern Search Pattern
ONE             Replacement string

The search pattern is on the left-hand side and the replacement string is on the right-hand side. The search pattern may be matched using REGULAR EXPRESSIONS, just as we did with the grep or egrep command.

The character after the s is the delimiter. It is conventionally a slash because this is what ed, more, and vi use. It can be anything you want. If you want to change a pathname that contains a slash – say /usr/local/bin to /common/bin – you could use the backslash to quote the slash as below:

sed 's/\/usr\/local\/bin/\/common\/bin/' 

but it is easier to read if you use an underline instead of a slash as a delimiter:

sed 's_/usr/local/bin_/common/bin_'

Use any delimiter you like, as long as it does not occur in the search string. Always remember that you need 3 delimiters.

There are many, many more options we won’t mention here

The sed command has many more options and is incredibly powerful. As this is a Linux Basics series, we will not delve into the magic of sed any further in this post.

However, if you’re looking for a deeper understanding, one of the best comprehensive guides to sed is by Bruce Barnet at grymoire.com.

Happy Hosting!