957

How do I find and replace every occurrence of:

subdomainA.example.com

with

subdomainB.example.com

in every text file under the /home/www/ directory tree recursively?

8
  • 116
    Tip: Don't do the below in an svn checkout tree... it will overwrite magic .svn folder files. Commented Nov 8, 2010 at 19:42
  • 9
    oh my god this is exactly what I just did. But it worked and doesn't seem to have done any harm. Whats the worst that could happen? Commented Feb 6, 2013 at 17:56
  • 5
    @J.Katzwinkel: at the very least, it may corrupt checksums, which may corrupt your repository. Commented May 14, 2013 at 13:36
  • 4
    Quick tip for all the people using sed: It will add trailing newlines to your files. If you don't want them, first do a find-replace that won't match anything, and commit that to git. Then do the real one. Then rebase interactively and delete the first one. Commented Oct 3, 2014 at 19:09
  • 6
    You can exclude a directory, such as git, from the results by using -path ./.git -prune -o in find . -path ./.git -prune -o -type f -name '*matchThisText*' -print0 before piping to xargs Commented Aug 24, 2016 at 19:58

37 Answers 37

1008
find /home/www \( -type d -name .git -prune \) -o -type f -print0 | xargs -0 sed -i 's/subdomainA\.example\.com/subdomainB.example.com/g'

-print0 tells find to print each of the results separated by a null character, rather than a new line. In the unlikely event that your directory has files with newlines in the names, this still lets xargs work on the correct filenames.

\( -type d -name .git -prune \) is an expression which completely skips over all directories named .git. You could easily expand it, if you use SVN or have other folders you want to preserve -- just match against more names. It's roughly equivalent to -not -path .git, but more efficient, because rather than checking every file in the directory, it skips it entirely. The -o after it is required because of how -prune actually works.

For more information, see man find.

Sign up to request clarification or add additional context in comments.

6 Comments

This worked for me, and my case was find/replacing IP address values. Question for the gallery, though: Why are the dots escaped for the first subdomainA\.example\.com value but not for the second sudomainB.example.com value? I executed it in the suggested format, and it seemed to do the job perfectly, but I'm curious why the escaping is only presented for the first string pattern.
This script will stop without reaching the end with the error Permission denied if one of the files has immutable flag. Better to use -exec sed -i ... {} \; instead of pipe.
I often use find . -type f -print0 | xargs -0 sed -i -e 's/\r$//' to replace all CRLFs with LFs in files recursively in a specific directory.
using MACOS and frustrated why it is not working -> try -> find . \( ! -regex '.*/\..*' \) -type f | LC_ALL=C xargs sed -i '' 's/foo/bar/g'
@elrobis (12 years later, but for the record) the first URL used escaped dots because it was in the regex match text and is special, but the second URL was in the replacement text and dots are not special in that context.
|
563

The simplest way for me is

grep -rlZ oldtext . | xargs -0 sed -i 's/oldtext/newtext/g'

13 Comments

This works especially well, when you need to exclude directories, like with .svn. For example: grep -rl oldtext . --exclude-dir=.svn | xargs sed -i 's/oldtext/newtext/g'
On macOS, sed -i causes sed: 1: "file_path": invalid command code .. This is because -i is a different flag on macOS. I found grep -rl old . | xargs sed -i "" -e 's/old/new/g' works. I found this useful
If you are using a compiled language and want to avoid checking binaries, you can pass the I flag like grep -Irl oldtext . | xargs sed -i 's/oldtext/newtext/g'
In a git project, be sure and use git grep -rl oldtext . | xargs sed -i 's/oldtext/newtext/g' to avoid searching the dependencies (which are probably ignored via .gitignore) :) Great solution! @phyatt this is a better way to do that.
I found that you can add -Z to grep, and -0 to xargs to capture filenames with spaces: grep -rlZ oldtext . | xargs -0 sed -i 's/oldtext/newtext/g': stackoverflow.com/questions/17296525/…
|
321

Note: Do not run this command on a folder including a git repo - changes to .git could corrupt your git index.

find /home/www/ -type f -exec \
    sed -i 's/subdomainA\.example\.com/subdomainB.example.com/g' {} +

Compared to other answers here, this is simpler than most and uses sed instead of perl, which is what the original question asked for.

3 Comments

Note that if you're using BSD sed (including on Mac OS X) you'll need to give an explicit empty string arg to sed's -i option. ie: sed -i '' 's/original/replacement/g'
How can I modify it to exclude .git subfolder?
@reducingactivity Hi! You can use this : grep -rl placeholder . | grep -Ev ".git" | xargs sed -i s/placeholder/lol/g (grep -Ev excludes patterns) - TIP: before actually running it to replace it, use it first without the -i like a dry-run.
116

All the tricks are almost the same, but I like this one:

find <mydir> -type f -exec sed -i 's/<string1>/<string2>/g' {} +
  • find <mydir>: look up in the directory.

  • -type f:

    File is of type: regular file

  • -exec command {} +:

    This variant of the -exec action runs the specified command on the selected files, but the command line is built by appending each selected file name at the end; the total number of invocations of the command will be much less than the number of matched files. The command line is built in much the same way that xargs builds its command lines. Only one instance of `{}' is allowed within the command. The command is executed in the starting directory.

Comments

58

For me the easiest solution to remember is https://stackoverflow.com/a/2113224/565525, i.e.:

sed -i '' -e 's/subdomainA/subdomainB/g' $(find /home/www/ -type f)

NOTE: -i '' solves OSX problem sed: 1: "...": invalid command code .

NOTE: If there are too many files to process you'll get Argument list too long. The workaround - use find -exec or xargs solution described above.

4 Comments

On Cygwin it produces sed: can't read : No such file or directory. Why and how to fix?
@pmor sounds like your "$(find ...)" command is returning no files, did you misspell the directory? try running just find /home/www/ -type f on its own
@guest4308 I didn't misspell the directory. The reason is the ''. On Cygwin there is no need for ''.
@pmor oh, I see. it should work if you remove the space between -i and ''. since adding a suffix is not required, it's interpreting the two as separate options, and when there's an expression provided all other options are interpreted as files to run through
42
cd /home/www && find . -type f -print0 |
      xargs -0 perl -i.bak -pe 's/subdomainA\.example\.com/subdomainB.example.com/g'

1 Comment

Some explanation would be in order, especially as it doesn't use any of the asked-for tools (the question is also tagged with them). E.g., what is the idea/gist? Please respond by editing your answer, not here in comments (without "Edit:", "Update:", or similar - the answer should appear as if it was written today).
37

For anyone using silver searcher (ag)

ag SearchString -l0 | xargs -0 sed -i 's/SearchString/Replacement/g'

Since ag ignores git/hg/svn file/folders by default, this is safe to run inside a repository.

3 Comments

Thanks for a working solution! I will need to find equivalent with ripgrep.
@reducingactivity Check out github.com/chmln/sd :) I'm a happy user
Replacing ag with rg for ripgrep works perfectly fine, too.
27

This one is compatible with git repositories, and a bit simpler:

Linux:

git grep -z -l 'original_text' | xargs -0 sed -i 's/original_text/new_text/g'

Mac:

git grep -z -l 'original_text' | xargs -0 sed -i '' -e 's/original_text/new_text/g'

(Thanks to http://blog.jasonmeridth.com/posts/use-git-grep-to-replace-strings-in-files-in-your-git-repository/)

4 Comments

Wiser to use git-grep's -z option together with xargs -0.
git grep obviously only makes sense in a git repo. The general replacement would be grep -r.
@gniourf_gniourf Can you explain?
@PetrPeller: with -z, git-grep will separate the output fields by null bytes instead of newlines; and with -0, xargs will read the input separated by null bytes, instead of blanks (and not do weird stuff with quotes). So if you don't want the command to break if the filenames contain spaces, quotes or other funny characters, the command is: git grep -z -l 'original_text' | xargs -0 sed ....
20

A straight forward method if you need to exclude directories (--exclude-dir=..folder) and also might have file names with spaces (solved by using 0Byte for both grep -Z and xargs -0)

grep -rlZ oldtext . --exclude-dir=.folder | xargs -0 sed -i 's/oldtext/newtext/g'

1 Comment

all other 7+ answers i've seen just ignore the white spaces!
20

To cut down on files to recursively sed through, you could grep for your string instance:

grep -rl <oldstring> /path/to/folder | xargs sed -i s^<oldstring>^<newstring>^g

If you run man grep you'll notice you can also define an --exlude-dir=.git flag if you want to omit searching through .git directories, avoiding git index issues as others have politely pointed out.

Leading you to:

grep -rl --exclude-dir=.git <oldstring> /path/to/folder | xargs sed -i s^<oldstring>^<newstring>^g

Comments

18

An one nice oneliner as an extra. Using git grep.

git grep -lz 'subdomainA.example.com' | xargs -0 perl -i'' -pE "s/subdomainA.example.com/subdomainB.example.com/g"

2 Comments

Good idea if working inside a git repo as you don't risk overwriting .git/ contents (as reported in the comments to another answer).
Thanks, I use it as a bash function refactor() { echo "Replacing $1 by $2 in all files in this git repository." git grep -lz $1| xargs -0 perl -i'' -pE "s/$1/$2/g" } Usage, for example to replace 'word' with 'sword': refactor word sword then verify what it did with git diff.
18

Simplest way to replace (all files, directory, recursive)

find . -type f -not -path '*/\.*' -exec sed -i 's/foo/bar/g' {} +

Note: Sometimes you might need to ignore some hidden files i.e. .git, you can use above command.

If you want to include hidden files use,

find . -type f  -exec sed -i 's/foo/bar/g' {} +

In both case the string foo will be replaced with new string bar

2 Comments

kind of slow, but worked for me me great!
another drawback is that this will update the timestamps of all found files, even if there is no text replacement. For my application this caused an issue, as we depend on timestamps.
14
find /home/www/ -type f -exec perl -i.bak -pe 's/subdomainA\.example\.com/subdomainB.example.com/g' {} +

find /home/www/ -type f will list all files in /home/www/ (and its subdirectories). The "-exec" flag tells find to run the following command on each file found.

perl -i.bak -pe 's/subdomainA\.example\.com/subdomainB.example.com/g' {} +

is the command run on the files (many at a time). The {} gets replaced by file names. The + at the end of the command tells find to build one command for many filenames.

Per the find man page: "The command line is built in much the same way that xargs builds its command lines."

Thus it's possible to achieve your goal (and handle filenames containing spaces) without using xargs -0, or -print0.

Comments

13

or use the blazing fast GNU Parallel:

grep -rl oldtext . | parallel sed -i 's/oldtext/newtext/g' {}

Beware that if you run this in the root of a git repository, you may end up corrupting your git index. To avoid this, you can use ripgrep instead of grep, like so:

rg -l oldtext | parallel sed -i 's/oldtext/newtext/g' {}

2 Comments

how does one installs GNU Parallel?
try to find the parallel package. arch: sudo pacman -S parallel; ubuntu/debian: sudo apt-get install parallel; fedora: dnf install parallel; I use arch btw
9

I just needed this and was not happy with the speed of the available examples. So I came up with my own:

cd /var/www && ack-grep -l --print0 subdomainA.example.com | xargs -0 perl -i.bak -pe 's/subdomainA\.example\.com/subdomainB.example.com/g'

Ack-grep is very efficient on finding relevant files. This command replaced ~145 000 files with a breeze whereas others took so long I couldn't wait until they finish.

5 Comments

Nice, but grep -ril 'subdomainA' * is nowhere near as fast as grep -Hr 'subdomainA' * | cut -d: -f1.
@Henno : just one question : how I can exclude binary files (executables files)?
ack-grep does that automatically for you.
@Henno : Does it include shell scripts?
Yes. Here is a complete list of file types it supports: beyondgrep.com/documentation
7

Try this:

sed -i 's/subdomainA/subdomainB/g' `grep -ril 'subdomainA' *`

1 Comment

Hi @RikHic, nice tip - was thinking about something like this; unfortunately that formatting above didn't quite turn out right :) So I'll try with a pre tag (doesn't work) - so with escaping backticks then: sed -i 's/subdomainA/subdomainB/g' ` grep -ril 'subdomainA' /home/www/* ` - this still doesn't look all too good, but should survive copypaste :) Cheers!
7

grep -lr 'subdomainA.example.com' | while read file; do sed -i "s/subdomainA.example.com/subdomainB.example.com/g" "$file"; done

I guess most people don't know that they can pipe something into a "while read file" and it avoids those nasty -print0 args, while presevering spaces in filenames.

Further adding an echo before the sed allows you to see what files will change before actually doing it.

2 Comments

The reason -print0 is useful is that it handles cases which while read simply cannot handle -- a newline is a valid character in a Unix file name, so for your code to be completely robust, it needs to cope with such file names, too. (Also, you want read -r to avoid some pesky POSIX legacy behavior in read.)
Also, the sed is a no-op if there are no matches, so the grep isn't really necessary; though it is a useful optimization for avoiding to rewrite files which do not contain any matches, if you have lots of those, or want to avoid updating date stamps on files needlessly.
7

According to this blog post:

find . -type f | xargs perl -pi -e 's/oldtext/newtext/g;'

3 Comments

How do you escape slashes / ?. For example, I want to replace IP addresses: xxx.xxx.xxx.xxx for xxx.xxx.xxx.xxx/folder
You can escape the / with \ . For example : find . -type f | xargs perl -pi -e 's/xxx.xxx.xxx.xxx\/folder/newtext/g;'
Perl, like sed, allows you to use any character as the delimiter after s; so, for example, try s%foo/bar%baz/quux% to replace foo/bar with baz/quux
5
#!/usr/local/bin/bash -x

find * /home/www -type f | while read files
do

sedtest=$(sed -n '/^/,/$/p' "${files}" | sed -n '/subdomainA/p')

    if [ "${sedtest}" ]
    then
    sed s'/subdomainA/subdomainB/'g "${files}" > "${files}".tmp
    mv "${files}".tmp "${files}"
    fi

done

Comments

5

You can use awk to solve this as below,

for file in `find /home/www -type f`
do
   awk '{gsub(/subdomainA.example.com/,"subdomainB.example.com"); print $0;}' $file > ./tempFile && mv ./tempFile $file;
done

hope this will help you !!!

3 Comments

Works on MacOs wihtout any problems! All sed based commands failed when binaries were included even with the osx specific settings.
Careful...this will blow up if any of the files find returns have a space in their names! It's much safer to use while read: stackoverflow.com/a/9612560/1938956
this won't work for files whose names contain spaces or new lines
4

If you do not mind using vim together with grep or find tools, you could follow up the answer given by user Gert in this link --> How to do a text replacement in a big folder hierarchy?.

Here's the deal:

  • recursively grep for the string that you want to replace in a certain path, and take only the complete path of the matching file. (that would be the $(grep 'string' 'pathname' -Rl).

  • (optional) if you want to make a pre-backup of those files on centralized directory maybe you can use this also: cp -iv $(grep 'string' 'pathname' -Rl) 'centralized-directory-pathname'

  • after that you can edit/replace at will in vim following a scheme similar to the one provided on the link given:

    • :bufdo %s#string#replacement#gc | update

Comments

4

For replace all occurrences in a git repository you can use:

git ls-files -z | xargs -0 sed -i 's/subdomainA\.example\.com/subdomainB.example.com/g'

See List files in local git repo? for other options to list all files in a repository. The -z options tells git to separate the file names with a zero byte, which assures that xargs (with the option -0) can separate filenames, even if they contain spaces or whatnot.

Comments

3

If you wanted to use this without completely destroying your SVN repository, you can tell 'find' to ignore all hidden files by doing:

find . \( ! -regex '.*/\..*' \) -type f -print0 | xargs -0 sed -i 's/subdomainA.example.com/subdomainB.example.com/g'

1 Comment

The parentheses appear to be superfluous. This previously had a formatting error which made it unusable (the Markdown rendering would eat some characters from the regex).
3

A bit old school but this worked on OS X.

There are few trickeries:

• Will only edit files with extension .sls under the current directory

. must be escaped to ensure sed does not evaluate them as "any character"

, is used as the sed delimiter instead of the usual /

Also note this is to edit a Jinja template to pass a variable in the path of an import (but this is off topic).

First, verify your sed command does what you want (this will only print the changes to stdout, it will not change the files):

for file in $(find . -name *.sls -type f); do echo -e "\n$file: "; sed 's,foo\.bar,foo/bar/\"+baz+\"/,g' $file; done

Edit the sed command as needed, once you are ready to make changes:

for file in $(find . -name *.sls -type f); do echo -e "\n$file: "; sed -i '' 's,foo\.bar,foo/bar/\"+baz+\"/,g' $file; done

Note the -i '' in the sed command, I did not want to create a backup of the original files (as explained in In-place edits with sed on OS X or in Robert Lujo's comment in this page).

Happy seding folks!

1 Comment

Using $(find ...) to drive a for loop could fail; see mywiki.wooledge.org/BashFAQ/020 and iki.fi/era/unix/award.html#backticks
3

just to avoid to change also

  • NearlysubdomainA.example.com
  • subdomainA.example.comp.other

but still

  • subdomainA.example.com.IsIt.good

(maybe not good in the idea behind domain root)

find /home/www/ -type f -exec sed -i 's/\bsubdomainA\.example\.com\b/\1subdomainB.example.com\2/g' {} \;

Comments

3

I just use tops:

find . -name '*.[c|cc|cp|cpp|m|mm|h]' -print0 |  xargs -0 tops -verbose  replace "verify_noerr(<b args>)" with "__Verify_noErr(<args>)" \
replace "check(<b args>)" with "__Check(<args>)" 

2 Comments

plus one for ` '*.[c|cc|cp|cpp|m|mm|h]' `
That's a bug actually; it searches for files containing a single c or | or p or m or h after the dot. The square brackets create a character class which matches a single character out of the ones between the brackets; repeating a character is allowed but useless.
3

I'm surprised I've not seen the simple answer using file globbing, which I just used to scan/update ONLY packge.json files with **/package.json

this was macos specific under zsh

cd /home/www
sed -i '' -e 's/subdomainA.example.com/subdomainA.example.com/g' **/*

1 Comment

I never knew you could do a glob with sed - thanks for posting this.
3

Here's a version that should be more general than most; it doesn't require find (using du instead), for instance. It does require xargs, which are only found in some versions of Plan 9 (like 9front).

du -a | awk -F' '  '{ print $2 }' | xargs sed -i -e 's/subdomainA\.example\.com/subdomainB.example.com/g'

If you want to add filters like file extensions use grep:

du -a | grep "\.scala$" | awk -F' '  '{ print $2 }' | xargs sed -i -e 's/subdomainA\.example\.com/subdomainB.example.com/g'

Comments

2

For Qshell (qsh) on IBMi, not bash as tagged by OP.

Limitations of qsh commands:

  • find does not have the -print0 option
  • xargs does not have -0 option
  • sed does not have -i option

Thus the solution in qsh:

    PATH='your/path/here'
    SEARCH=\'subdomainA.example.com\'
    REPLACE=\'subdomainB.example.com\'

    for file in $( find ${PATH} -P -type f ); do

            TEMP_FILE=${file}.${RANDOM}.temp_file

            if [ ! -e ${TEMP_FILE} ]; then
                    touch -C 819 ${TEMP_FILE}

                    sed -e 's/'$SEARCH'/'$REPLACE'/g' \
                    < ${file} > ${TEMP_FILE}

                    mv ${TEMP_FILE} ${file}
            fi
    done

Caveats:

  • Solution excludes error handling
  • Not Bash as tagged by OP

1 Comment

This has some pesky issues with quoting as well as reading lines with for.
2

Using combination of grep and sed

for pp in $(grep -Rl looking_for_string)
do
    sed -i 's/looking_for_string/something_other/g' "${pp}"
done

4 Comments

@tripleee I modified this a bit. In this case output for command grep -Rl pattern generated list of files where the pattern is. Files are not read in for loop.
Huh? You still have a for loop; if any returned file name contains whitespace, it will not work correctly, because the shell tokenizes the for argument list. But then you use the file name variable without quotes inside the loop, so it would break there instead if you fixed this. Correcting these remaining bugs would make yours identical to @MadMan2064's answer.
@tripleee yes, that is true, I missed this.
this won't work for files whose names contain spaces or new lines

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.