1

I am having trouble with a short bash script. It seems like all forward slashes needs to be escaped. How can required characters in expanded (environment) variables be escaped before perl reads them? Or some other method that perl understands.

This is what I am trying to do, but this will not work properly.

eval "perl -pi -e 's/$HOME\/_TV_rips\///g'" '*$videoID.info.json'

That is part of a longer script where videoID=$1. (And for some reason perl expands variables both within single and double quotes.)

This simple workaround with no forward slash in the expanded environment variable $USER works. But I would like to not have /Users/ hard coded:

eval "perl -pi -e 's/\/Users\/$USER\/_TV_rips\///g'" '*$videoID.info.json'

This is probably solvable in some better way fetching home dir for files or something else. The goal is to remove the folder name in youtube-dl's json data. I am using perl just because it can handle extended regex. But perl is not required. Any better substitute for extended regex on macOS is welcome.

Improve this question
10
  • 1
    The XY problem is asking about your attempted solution rather than your actual problem. Commented Oct 31, 2021 at 17:36
  • Not really. I am describing my current method that I am having problem with and also describe my overall goal in the last paragraph. And also stating that I am open to other methods as well, but also curious what goes wrong with perl and to solve that - even if there is another better solution that I might be inspired by the in answers. Commented Oct 31, 2021 at 17:38
  • 1
    I don't get it why you would use eval here Commented Oct 31, 2021 at 17:48
  • 1
    You can use s{...}{...} instead of s/.../.../ to avoid escaping / (since it is no longer the separator in the statement. You could also use $ENV{HOME} to access the environment variable from Perl instead of using the one from shell. Note that depending on the quoting you might need to escape $ then. "And for some reason perl expands variables both within single and double quotes." - Perl isn't expanding anything here but the shell does since you use double quotes around the perl statement, i.e. eval "....shell will expand things here ..." Commented Oct 31, 2021 at 17:58
  • 1
    Anytime you are using eval you are probably doing something wrong. You don't need to use eval to get environment variables into Perl. They are already there, in the %ENV hash. If you want to escape characters inside a regex, you can just use \Q to quote meta characters. If you add a simple test case, this would probably be an incredibly simple Perl task. Commented Oct 31, 2021 at 18:18

3 Answers 3

Reset to default
3

You are building the following Perl program:

s//home/username\/_TV_rips\///g

That's quite wrong.

You shouldn't be attempting to build Perl code from the shell in the first place. There are a few ways you could pass values to the Perl code instead of generating Perl code. Since the value is conveniently in the environment, we can use

perl -i -pe's/\Q$ENV{HOME}\E\/_TV_rips\///' *"$videoID.info.json"

or better yet

perl -i -pe's{\Q$ENV{HOME}\E/_TV_rips/}{}' *"$videoID.info.json"

(Also note the lack of eval and the fixed quoting on the glob.)

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Superb, thanks! The syntax in the second version is much cleaner. What tool do you recommend instead of perl for extended regex editing of file content?
@Kasam, 1) You didn't identify the reasons for ruling out perl, so there's no way to know what other tools are out of considerations 2) Tool recommendations are off-topic
1

Just assembling the ideas in comments, this should achieve what you expected :

perl -pi -e 's{$ENV{HOME}/_TV_rips/}{}g' *$videoID.info.json

@ikegami thanks for your comment! It is indeed safer with \Q...\E, in case $HOME contains characters like $.

Improve this answer

3 Comments

That didn't work on my system (perl 5, version 18). Using \Q as in @ikegami's reply did the trick.
@Kasam. You are mistaken. This works fine. The \Q is only supposed to help with special characters in $ENV{HOME}, of which there should never be any. If you are relying on \Q, you are doing something wrong (like using "...$HOME..." instead of '...$ENV{HOME}...' and not using what we posted.
@ikegami I tried it again, it worked fine. Thanks for the reminder. Even simpler syntax.
0

All RegEx delimiters must of cource be escaped in input String.

But as Stefen stated, you can use other delimiters in perl, like %, §.

Special characters

  1. # Perl comment - don't use this
  2. ?,[], {}, $, ^, . Regex control chars - must be escaped in Regex. That makes it easier if you have many slashes in your string.

You should always write a comment to make clear you are using different delimiters, because this makes your regex hard to read for inexperienced users.

Try out your RegEx here: https://regex101.com/r/cIWk1o/1

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.