using variables in regex?

Question

Part of a shell script that I am creating takes a plain text list of files...

 11111.jpg
 22222.jpg
 33333.jpg

...and appends a user-defined prefix that is stored in a variable to create a list of paths that looks like this:

 user/defined/prefix/11111.jpg
 user/defined/prefix/22222.jpg
 user/defined/prefix/33333.jpg

I am attempting to use sed to add the prefix in this manner:

 sed -e 's/^/prefix/' oldFile > newFile.new

The variable is getting assigned correctly:

 echo $selectedPrefix
 user/defined/prefix

Put no combinations of single quotes, double quotes of whatever seem to get sed to use the ACTUAL value of the variable instead of just the variable name.

 sed -e 's/^/$selectedPrefix/' oldFile > newFile.new

Yields:

 $selectedPrefix11111.jpg
 $selectedPrefix22222.jpg
 $selectedPrefix33333.jpg

Help! I'm sure the solution is simple but I feel like I've tried everything....

Thanks guys! So is this basically a case where, because my variable contains '/' I need to use something else as a seperator for sed? In which case I could use '~' as per your example, or something else? — Andrew Hall
– Andrew Hall, Commented Dec 1, 2016 at 20:17
You can use any other character also as delimiter like | or # or @ — anubhava
– anubhava, Commented Dec 1, 2016 at 20:30
You can try sed -e "s/^/${selectedPrefix//\//\/}/" oldFile > newFile.new. Then you won't have to worry what other delimiter to use. — alvits
– alvits, Commented Dec 2, 2016 at 3:14

fedorqui · Accepted Answer · 2016-12-02 09:01:23Z

1

As mentionned by Cyrus, you need to used " (double quote) instead ' (single quote) if you want the variable replacement because single quoted string are interpreted literally so it doesn't see $selectedPrefix as a variable but as the string value of $selectedPrefic hence what you saw.

Since you are working with paths in you sed, you are correct in assuming that you should use a different separator for your sed comment. I usually prefer using | but ~ would also work.

so basically you could have:

sed -e "s~^~$selectedPrefix~" oldFile > newFile.new

edited Dec 2, 2016 at 9:01

fedorqui

293k113 gold badges592 silver badges640 bronze badges

answered Dec 1, 2016 at 20:28

Mike

895 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Jamil Said Over a year ago

Unfortunately this answer is not entirely "safe", as this code would fail in case that the path on "selectedPrefix" included the sed delimiter "~". As I assume that the "selectedPrefix" is subject to change, only escaping the sed delimiters on the variable as I did on my answer would work no matter what the path on "selectedPrefix" includes.

Mike Over a year ago

You are right that it's not foolproof vs your suggested answer. I just thought that it might seem more friendly to use and will work most time than not but you are absolutely right that it will fail for any delimiter that is contained in the variable

Jamil Said Over a year ago

It is easy to oversee this, I've done it myself. One time I remember I used different delimiters for sed in a code and during the night, when I was already in bed, it came to me (from nowhere) that this exception could happen. I left bed and changed the code in the middle of the night. LOL

Jamil Said · Accepted Answer · 2016-12-01 20:40:31Z

0

This code would solve your problem:

selectedPrefixEscaped="$(echo "$selectedPrefix" | sed 's/\//\\\//g')" && sed -e "s/^/$selectedPrefixEscaped/" oldFile > newFile.new

Just using a different delimiter on sed would leave you open to problems when (if) the path contains the new delimiter (ex.: /folder/folder#5/file.txt would be problematic if using # as sed delimiter).

edited Dec 1, 2016 at 20:40

answered Dec 1, 2016 at 20:23

Jamil Said

2,1433 gold badges18 silver badges18 bronze badges

14 Comments

alvits Over a year ago

That can be abbreviated to sed -e "s/^/${selectedPrefix//\//\/}/" oldFile > newFile.new.

Jamil Said Over a year ago

Alvits, I tried your abbreviation but it does not work on my computer. Error message: sed: -e expression #1, char 10: unknown option to `s'.

alvits Over a year ago

You are probably not running bash. The syntax I gave above only works on bash.

Jamil Said Over a year ago

I am running bash (GNU bash, version 4.3.30(1)). Are you aware that "$selectedPrefix" expanded contains forward slashes (selectedPrefix=user/defined/prefix), which are also the delimiter of sed in this case? Try your code with the expansion, it does not work (at least here it doesn't, I tried again).

alvits Over a year ago

You can't set selectedPrefix while referencing it at the same line because the shell will expand an empty expression. Running this line echo 11111.jpg | sed -e "s/^/${selectedPrefix//\//\/}/" when selectedPrefix is already set, produces user/defined/prefix11111.jpg.

|

Collectives™ on Stack Overflow

using variables in regex?

2 Answers 2

3 Comments

14 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

14 Comments

Your Answer

Sign up or log in

Post as a guest

Related