2

I'm finding really hard to use sed command, plus I can't seem to find well written tutorials.

Let me say that I worked with regular expression in other languages (Python, JavaScript, Java), so that shouldn't be the problem.

So, here are my questions (a "theoretical" one and a more practical one):

  1. are regular expressions used in sed exactly the same as the ones used by Python/JS/Java? I read about BREs and EREs, but how much are they different? Shouldn't ERE be an extension of BRE?

  2. if I want to, say, just extract something from a piped output, what's the sed syntax to do that?

Details about the second question: say I have the output of uptime piped with sed:

uptime | sed ...

Given an example output from uptime: 18:13 up 5:12, 2 users, load averages: 0,45 0,37 0,40, I want to parse the single uptime in the form of two separated numbers (hours and minutes), and then I want to display them in the form of xxhyym (xx are the hours, yy the minutes).

And to finish, here's what I'd do in Python:

hh, mm = re.match(r'\s+ up \s+(\d{1,2}):(\d{1,2})').groups()
print '%sh%sm' % (hh, mm)
Improve this question
1
  • 2
    Just a heads up: both your python and @vonbrand's sed will fail if the uptime is longer than a day (unless you both really mean to ignore days). Commented Mar 26, 2013 at 17:37

2 Answers 2

Reset to default
4

Traditional unix tools support either BRE or ERE (basic or extended regular expressions). POSIX codifies both. Wikipedia explains them. Most modern tools extend ERE, often with additional features first introduced in Perl (which is known as PCRE).

ERE extends the functionality of BRE, but doesn't extend the syntax. In BRE, only the characters \[.*^$ have a special meaning, and some operators such as grouping \(…\) use backslashes. In ERE, +?|() are also special, and backslash followed by a non-alphanumeric character is never special.

BRE doesn't have Python/PCRE's \d and \s. You can express these character sets with the traditional set constructs and character classes: \d is [[:digit:]] and \s is [[:space:]]. Note the double brackets: one to indicate a character set and one to indicate a character class; for example “letters, dashes or underscores” can be written [-_[:alpha:]].

BRE's don't have a + operator (some sed implementations support \+ as an extension to the BRE syntax); X+ is the same as XX*. Groups and match counts need additional backslashes.

The BRE equivalent of Python's \s+ up \s+(\d{1,2}):(\d{1,2}) is thus [[:space:]][[:space:]]* up [[:space:]][[:space:]]*\([[:digit:]]\{1,2\}\):\([[:digit:]]\{1,2\}\). Note that you're matching too much whitespace: \s+ and a space means at least two whitespace characters.

You'll need to match the whole line, as sed's s command rewrites the line. There isn't a separate command to write out a string assembled from saved groups. Correcting for the extra whitespace, the analog of your Python snippet is:

uptime | sed 's/^.*[[:space:]][[:space:]]*up[[:space:]][[:space:]]*\([[:digit:]]\{1,2\}\):\([[:digit:]]\{1,2\}\).*$/\1h\2m/'

Unlike the Python snippet, this extracts the first match rather than the last match, but it doesn't matter here.

The output of uptime sticks to space characters and ASCII digits, so you can simplify the regex:

uptime | sed 's/^.* up  *\([0-9]\{1,2\}\):\([0-9]\{1,2\}\).*$/\1h\2m/'

This will only match the output of uptime if the machine has been up for less than 1 day. I'll leave matching the number of days as an exercise. (Hint: write two expressions: sed -e s/AS ABOVE/\1h\2m/ -e 's/EXERCISE/\1d\2h\3m/')

Improve this answer
1
  • Note that ERE's + can be written \{1,\} in BREs. Commented Mar 27, 2013 at 0:28
1

Each tool uses (mostly) their own RE library. Even among different sed versions you will find differences here. Two popular standards are POSIX standard regular expressions, many tools accept those (at least with some options), another popular set is the Perl Compatible Regular Expression library (PCRE). But the last ones are quite a bit different from "vanilla" REs...

In your case:

uptime | sed -e 's/^ \([0-9][0-9]\):\([0-9][0-9]\).*$/\1h\2m/'

(Tried on Fedora 18, sed-4.2.1-10.fc18.x86_64, GNU sed).

Update: What is wrong with the copious documentation on GNU sed's homepage? Or this tutorial? The info documentation for GNU sed is a bit long-winded, but complete.

Improve this answer
5
  • Mmm ok, that's worse than I imagined it. I hoped I could use some syntax like \d{1,2} which I really find useful. Anyway, on OS X the command you wrote returns the whole output of uptime... LOL. Anyway, thank you for the answer, I'm gonna study sed in a deeper way. Commented Mar 26, 2013 at 17:43
  • If you want that kind of syntax (not universal), you might want to consider perl, it has very rich patterns and is quite one-liner friendly. And no, I haven't ever claimed to be a sed expert. Commented Mar 26, 2013 at 17:58
  • No I wasn't saying that, I was just saying that sed interests me so I'm gonna study it deeply probably :). Commented Mar 26, 2013 at 18:10
  • Oh sure you can use something like \d{1,2},namely [0-9]\{1,2\}. The syntax of sed can be a bit off, and the regexps are not as powerful, but I find it very useful for many tasks on the command line Commented Mar 26, 2013 at 20:05
  • 1
    @juanpa, for this problem that is more typing (and more thinking ;-) than my solution. Commented Mar 26, 2013 at 20:17

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.