Using variables with sed (UNIX)

Question

I am trying to read in a log file and create a new file with only the entries after the given date and time. For example I am looking for only after the time 07:00:00.000 in the below log:

2015-01-16 00:00:00.001 DATA
2015-01-16 07:05:00.121 DATA
2015-01-16 07:10:15:543 DATA

Would go to:

2015-01-16 07:05:00.121 DATA
2015-01-16 07:10:15:543 DATA

I can already get this to work with:

sed -n '/^2015-01-16 07/,$p' oglog.log > newlog.log

But when I try to add the date via a variable:

date=$(date -d -1hour +'%Y-%m-%d %H')

I cannot get it to work. I know you need to get the shell to expand variable via either double or single quotes (https://askubuntu.com/questions/76808/how-to-use-variables-in-sed-command), but with either of these methods it causes problems with $p in the replacement part of the sed command. How can I get the shell the expand one variable but not the other and use a command similar to below?

sed -n '/^$date/,$p' oglog.log > newlog.log

Right I have tried this method: result=$(sed -n "/^$testDate/,$p" logtest.log > newlogtest.log) returns: sed: -e expression #1, char 20: unexpected ',' and result=$(sed -n '/^"$testDate"/,$p' logtest.log > newlogtest.log) returns an empty file. — whla
– whla, Commented Jan 16, 2015 at 19:30
Note that if the first event after 07:00:00.000 is at 08:00:00.000, nothing will be printed. You might need to use awk which can do >= comparisons — Jonathan Leffler
– Jonathan Leffler, Commented Jan 16, 2015 at 19:31
@JonathanLeffler good to know! This log has 10's of entries every minute so I think it will be alright. — whla
– whla, Commented Jan 16, 2015 at 19:38
possible duplicate of Why is my sed command failing when using variables? — tripleee
– tripleee, Commented Jan 16, 2015 at 21:00

Jonathan Leffler · Accepted Answer · 2015-01-16 19:26:38Z

3

sed -n "/^$date/"',$p' oglog.log > newlog.log

You want the $date expanded by the shell, so that needs to be in double quotes. You don't want the $p expanded by the shell, so that needs to be in single quotes. You could also use a backslash to prevent the shell expanding that $p, but that quickly gets nasty if you have other meaningful backslashes in the regex:

sed -n "/^$date/,\$p" oglog.log > newlog.log

answered Jan 16, 2015 at 19:26

Jonathan Leffler

759k145 gold badges961 silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Gilles Quénot · Accepted Answer · 2015-01-16 19:39:36Z

1

Use

sed -n "/^$date/,\$p" oglog.log > newlog.log

variables are not expanded in single quotes.

"Double quote" every literal that contains spaces/metacharacters and every expansion: "$var", "$(command "$var")", "${array[@]}", "a & b". Use 'single quotes' for code or literal $'s: 'Costs $5 US', ssh host 'echo "$HOSTNAME"'. See
http://mywiki.wooledge.org/Quotes
http://mywiki.wooledge.org/Arguments
http://wiki.bash-hackers.org/syntax/words

edited Jan 16, 2015 at 19:39

answered Jan 16, 2015 at 19:26

Gilles Quénot

188k43 gold badges232 silver badges229 bronze badges

Collectives™ on Stack Overflow

Using variables with sed (UNIX)

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related