Sed

From LQWiki
Jump to: navigation, search

sed (stream editor) is an editor used, not interactively on text files (like vi or emacs), but on streams. This allows it to transform text input from a pipe or the command line or a file (if it is piped to sed or given as an argument to sed).

Examples

With sed, you can quickly replace strings in a stream:

$ cat test
hello world
$ cat test | sed "s/world/moon/g"
hello moon

Here the sed command is

sed "s/world/moon/g"

"s" stands for "substitute". world is the string that shall be replaced, moon is the string by which it shall be replaced. The "g" suffix causes sed to make more than one replacement per input line as is usually intended.

In-file replacement

To replace a text in an existing file, use the -i argument:

$ cat test
hello world
$ sed -i "s/world/moon/g" test
$ cat test
hello moon

In this example, the file test is modified to contain moon whereever it contained world before.

RegEx

You can also replace patterns instead of fixed strings. To define a pattern you use the regex syntax. To learn more about the examples below, see regEx.

The following example removes all tags from an html file:

$ sed 's/<.*>//g' foobar.html

An html tag is any string enclosed in brackets (defined as a regex: <.*>). This is replaced by nothing (what is between the two slashes //).

Backreferences

You regex can also contain a backreference. For example, the following command will transform strings from inputfile.txt to outputfile.txt. foo something will be replaced by foo("something") and bar 54321 something will be replaced by bar(54321, "something").

$ sed -e 's/foo \([^ ]*\)/foo("\1")/g' -e 's/bar \([0-9]*\) \([^ ]*\)/bar(\1, "\2")/' < inputfile.txt > outputfile.txt

Examining the above command in more detail: the -e flags specify a new expression, allowing more than one replacement rule in the same command. The escaped-parentheses \( and \) tell sed to remember the string that was enclosed. The \1 and \2 commands tell sed to insert a remembered string.

Sed scripts

If you want to use a lot of commands you can save them as a sed_script.txt:

# this is a sed script
# usage: $ sed -f sed_script.txt textfile.txt > new_textfile.txt

# start replacing all things that must be replaced
# replace / with _
s/\//_/g

Running the script on textfile.txt:

$ sed -f sed_script.txt textfile.txt > new_textfile.txt

Another option is to save it as an executable script script.sh:

#!/usr/bin/sed -f
# save your commands in this script.sh; chmod u+x script.sh
# call it like so: $ ./script.sh textfile.txt > new_textfile.txt

# start replacing all things that must be replaced
# replace / with _
s/\//_/g

To execute the script on your textfile.txt:

$ chmod u+x script.sh
$ ./script.sh textfile.txt > new_textfile.txt

See also