linux: how to replace text in file from command line

Posted on by barkeep

Linux has several different utilities that can perform string or text manipulation. Depending on your environment and the tools that are available, you should be able to easily parse, find and replace any string in a text file. Almost every single text editor comes with a find and replace feature with a UI, so we are going to only explore how it can be done from the command line.

Some of the commonly used utilities are sed, awk, gawk, perl and vim. We will look at several examples of how you could use them in different scenarios. All of these also come with several different command line options.

using sed

sed or stream editor is probably the most well known of all the text manipulation utilities. It provides the most flexibility and ease when it comes to replacing text in large texts. The most simple example of using sed to replace text is:

bash$ sed -i 's/oldtext/newtext/g' textfile.txt

the -i command line option specifies that the substitution should be done in place. This will modify the text and save back to the original file. The last command line argument is the name of the file.

sed follows a regular expression pattern for substitution that is used by many programs and utilities. In the example above, the s in the command string specifies that it is a substitute command expression. The first string after the forward slash is the string that will be replaced with the new text.

  • s : the substitute command
  • oldtext : a regular expression that match the text to be replaced
  • newtext : the string that will replace the matched string
  • g: global or replace all matched instances of the expression.

The sed command can be piped as well. That means you pipe the output of sed through other command or pipe the output of other command through to sed.

using awk

awk and gawk are powerful text processing utilities that almost are programming languages in its own way. We will just use it for simple text replacement but the program is capable of much much more. Being a much more powerful tool, the syntax of awk can be a little challenging than that of sed, but then that is subjective.

Without going into how awk works, we will look at how two functions in awk can perform this substitution. The two functions are sub and gsub. There are both identical for most part but sub will only replace the first occurrence of a string while gsub will replace occurrences of the string. We will use gsub for the reminder of this post.

bash$ awk '{gsub(/oldtext/, "newtext"); print}'

awk can take input files as arguments as well. If you have the latest versions of awk then you can do in-place editing just like sed. Otherwise you will need to direct the output to another file.

bash$ awk '{gsub(/oldtext/, "newtext")}' inputfile1.txt inputfile2.txt

using perl

Perl is a programming language that can be run from the command line interface and can be used for text processing. However, Perl is a general purpose language rather than a text processing language. But simple text substitutions can be done in Perl from the command line or from within scripts.

bash$ perl -pi -e 's/oldtext/newtext' inputfile1.txt
  • -p : this specifies that it needs to loop over the input files
  • -e : this allows you to write the one line of script
  • -i : this specifies that it is an inline edit

Perl is much more powerful than this, but we are only dealing with simple command line substitutions in this post.

using ex mode of vim

Almost all Linux distros should have the vi editor installed. This means you will also have the ex mode that will allow you to perform text substitution just as you can from within the vi editor.

bash$ ex -s -c '%s/oldtext/newtext/g|x' inputfile1.txt
  • -s : suppress any interactive user feedback
  • -c : the command follows this
  • % : perform command on all lines
  • s : substitution command
  • g : replace all occurences
  • x : write changes back to file

We have not dealt with regular expressions that can be used to match several strings at the same time. In all of the examples above, you can very well use regular expressions instead of the string oldtext to match against multiple strings that can be substituted.

These are just some of the ways you can perform a text replacement from command line in Linux. There are several others which involve piping the output using different commands. It all depends on what exactly you want to search for and replace it with.