8

I'm trying to print lines that have a 4th field value of 1001 or 1003 from a file called mypasswd.  I can only use grep or egrep with regular expressions.  Here is the file:

daemon:x:2:2:Daemon 1001:/sbin:/bin/bash
ftp:x:40:49:FTP export account:/srv/ftp:/bin/bash
daemonuser:x:50:59:nouser/bin/false:/home/nouser:/bin/bash
gdm:x:106:111:Gnome Display Mgr daemon:/var/lib/gdm:/bin/false
haldaemon:x:101:102:User for haldaemon:/var/run/hald:/bin/false
lp:x:4:7:Printing daemon:/var/spool/lpd:/bin/bash
mail:x:8:12:Mailer daemon:/var/spool/clientmqueue:/bin/false
root:x:0:0:root:/root:/bin/bash
sshd:x:71:65:SSH daemon:/var/lib/sshd:/bin/false
olivert:x:1001:1005:Tom Oliver:/home/olivert:/bin/csh
smiths:x:1049:1000:Sue Williams:/export/home/smiths:/bin/csh
northj:x:1003:1003:Jim jones-North:/home/northj:/bin/csh
denniss:x:1005:1003:Sue Dennis:/home/denniss:/bin/bash
smitha:x:1050:1001:Amy Smith:/export/home/smitha:/bin/bash
jonesc:x:1053:1001:Cathy Jones:/export/home/jonesc:/bin/ksh
smithd:x:1055:1001:Dan Smith Jr:/export/home/smithd:/bin/csh

So the output should be

northj:x:1003:1003:Jim jones-North:/home/northj:/bin/csh
denniss:x:1005:1003:Sue Dennis:/home/denniss:/bin/bash
smitha:x:1050:1001:Amy Smith:/export/home/smitha:/bin/bash
jonesc:x:1053:1001:Cathy Jones:/export/home/jonesc:/bin/ksh
smithd:x:1055:1001:Dan Smith Jr:/export/home/smithd:/bin/csh

I can easily just run egrep '1001|1003' mypasswd, but that also gives me "daemon" (fifth field contains "1001") and "olivert" (third field is "1001").  I'm just needing the 4th field values (values that are after three colons) that match those two numbers using egrep/grep regular expressions. Any answers are greatly appreciated, as they will help me out in the long run with this.

Improve this question
2
  • 2
    grep isn't the best solution here; a "robust" solution with grep would require matching & skipping the first three fields. Are you open to a more obvious solution with other tools, such as awk, or perl? Commented Nov 13, 2019 at 16:44
  • 1
    Per your comment on one of the answers, if you're required to use grep (perhaps because you're learning about grep and/or regular expressions in a class), please say so. Commented Nov 13, 2019 at 16:51

8 Answers 8

Reset to default
16

It would be more direct, in my opinion, to use a tool like awk that can:

  • split fields for you
  • test exactly the fields you want for the values you want

For example:

awk -F: '$4 == 1001 || $4 == 1003' mypasswd

... tells awk to:

  • split the incoming lines into fields based on colons, with -F:
  • uses an "or" expression to test whether field 4 has the value 1001 or 1003
  • if the condition above is true, then print the line (the default action)

Awk can take a little bit to learn; one of the major things to understand about it is that it uses paired "pattern" and "action" statements. The "pattern" section(s) determine which "action" statements get executed.

You could rewrite the above awk to make it more explicit; by doing so, we can explicitly print whatever we want (such as the 5th field):

awk -F: '$4 == 1001 || $4 == 1003 { print $5 }'

... or to have an empty "pattern" section -- meaning, execute the "action" for every line, and then test inside the action pattern for the values:

awk -F: '{ if ($4 == 1001 || $4 == 1003)  print $5 }'

To force grep into action, you could do:

grep -E '^([^:]*:){3}(1001|1003):' mypasswd | cut -d: -f5

To tell it to look, from the beginning of the line, for the group "anything-that-isn't-a-colon any number of times, followed a colon" three times, followed by either 1001 or 1003, then followed by a colon; print the whole matching line, but then pass it to cut to print just the 5th field.

Improve this answer
7
  • Thank you! The grep command worked really well! Wish I had enough rep to give you an upvote (lol). Commented Nov 13, 2019 at 17:04
  • Glad it helped; if you keep asking good questions, the rep will come naturally! Please do take note to say in your questions whether you have an artificial limitation of some sort; otherwise, people (like me) are prone to answering with tools that have fewer sharp edges. Give the question some time, and review the answers when you're ready. If any of them solved your question, there should be a checkmark next to it for you to click. Thanks, and welcome to U&L! Commented Nov 13, 2019 at 17:07
  • @JakeM, I think you do now :D Commented Nov 14, 2019 at 16:44
  • Or even awk -F: '$4 ~ /1001|1003/ ... Commented Nov 14, 2019 at 17:27
  • @D.BenKnoble except, as ilkkachu pointed out, using /1001|1003/ would falsely match on e.g. 10010 or 10030. Commented Nov 14, 2019 at 17:29
4

I might do this with sed

sed -n '/^.*:.*:.*:\(1001\|1003\):/p' mypasswd

The -n supresses the lines and the p at the end prints the ones that match.

you could also do it with grep

grep '^.*:.*:.*:1002\|1003:.*:.*:' mypasswd
Improve this answer
5
  • The .* can match an additional colon, so the sed solution here will also match lines where the gecos/comment field is 1001 or 1003, but at least you have the trailing : to stop 10010 from matching. The grep solution seems fine since you have all the colons spelled out, ...that is unless someone has a passwd file with more fields than usual (I don't know if that would fly with other utilities though) Commented Nov 14, 2019 at 16:32
  • @ilkkachu yes, I should have used non-greedy globs, but my regex skills are not that good to know which programs have greedy operators. Commented Nov 14, 2019 at 16:34
  • Non-greedy globs are a Perl thing, they don't work in the standard tools. [^:]* should work here since the delimiter is just one character. Commented Nov 14, 2019 at 16:36
  • 1
    Note that it assumes GNU sed or compatible for \|. Using -E and EREs may be more portable these days (it will be in the next major POSIX version). Same for grep (and grep -E has been standard for decades, egrep before that). Commented Nov 14, 2019 at 17:22
  • … or, for this particular case, you could use 100[13]. That works only because 1001 and 1003 are the same except for one character; you wouldn’t be able to use that trick for 999\|1001 or even 1009\|1010. Commented Nov 14, 2019 at 19:51
4

As @JeffSchaller says, awk is the tool for the job and since OP wants regex we can just combine the two

awk -F: '$4 ~ /^100[13]$/' mypasswd

and that allows a little golf putt on the grep version

grep -E "^(.*:){3}100[13]:" mypasswd
Improve this answer
2
  • Same comment as for others: the grep solution can match lines where the fourth field is 10010 (trailing digits), or where the fifth field starts with 1001 (.* can match additional colons). The awk one can also match in the middle of the field, so 10010 and 11001 would match. Commented Nov 14, 2019 at 16:34
  • Fair comment, so the 4th field match needs anchoring at both ends in both cases. Commented Nov 14, 2019 at 17:19
2

Obligatory Perl solution(s):

Interpret the field as a number:

perl -F: -ane 'print if $F[3] == 1001 || $F[3] == 1003' mypasswd

or use a regex

perl -F: -ane 'print if $F[3] =~ /^(1001|1003)$/' mypasswd

or a somewhat shorter regex:

perl -F: -ane 'print if $F[3] =~ /^100[13]$/' mypasswd

or do some tricks with arithmetic (a bit tongue-in-cheek this one):

perl -F: -ane 'print if abs($F[3] - 1002) == 1' mypasswd

The options are: -a = autosplit to array @F (indexing starts at zero), -F: = use : as field separator, -ne run the script here for each line

The anchors to start/end of line (^ and $) are necessary so that numbers like 11001 or 10010 don't match.

Improve this answer
1
  • The regex variants here would miss lines where the number starts with a leading zero, but I hope nobody does that... (it would be in danger of being interpreted as octal, and that would be bad). I just hope I didn't leave any other mistakes. Commented Nov 14, 2019 at 16:49
2

"I'm just needing the 4th field values (values that are after three colons) that match those two numbers using egrep/grep regular expressions"

If you just need the 4th field, use cut with grep like so:

$ cat mypasswd | cut -d: -f4 | grep -E '^(1001|1003)$'
1001
1003

The grep pattern needs to be anchored to the start and end of the line, otherwise it might match a number like 10010. An alternative way to do that would be grep -x -E '1001|1003'.

Improve this answer
3
  • hmm, shouldn't that be cut -d: -f 4 for the fourth field? It's one-based in cut. Commented Nov 14, 2019 at 16:40
  • @ilkkachu Thank for adding missing anchors. Regarding the 4th element, you're right. Regarding the regexp, there are many types to choose from, like: cat mypasswd | cut -d: -f4 | grep '^100[1,3]$' and similar. Also there's no need to escape the colon sign as it has no special meaning here. Commented Nov 14, 2019 at 17:59
  • Oh sure, there's also alternatives for the regex. I just mentioned -x since it occurred to me that it might look nicer than ^(..|..)$. Commented Nov 14, 2019 at 18:02
1

You said in a comment (now deleted)

I need to print the whole line for this to work, but the 4th field value should be highlighted.

The only was grep can “highlight” a “field value” is to color it.  If you want to do that, we can extend Stéphane Chazelas’s refinement on ilkkachu’s refinement on StrongBad’s answer:

grep -E '^[^:]*:[^:]*:[^:]*:(1001|1003):[^:]*:[^:]*:' mypasswd

and (assuming that this is GNU grep, which is normal on Linux) add some PCRE magic:

grep --color -P '^[^:]*:[^:]*:[^:]*:\K(1001|1003)(?=:[^:]*:[^:]*:)' mypasswd

The \K is the “keep out” (a.k.a. “kill”) syntax that says that the text matched by the regular expression so far should be excluded from the regex match.  The (?=regex) is a (positive) lookahead; it says that the 1001 or 1003 should be matched only if it is followed by a colon and three more fields.

Given the input file in the question, the above command produces the following output:

northj:x:1003:1003:Jim jones-North:/home/northj:/bin/csh
denniss:x:1005:1003:Sue Dennis:/home/denniss:/bin/bash
smitha:x:1050:1001:Amy Smith:/export/home/smitha:/bin/bash
jonesc:x:1053:1001:Cathy Jones:/export/home/jonesc:/bin/ksh
smithd:x:1055:1001:Dan Smith Jr:/export/home/smithd:/bin/csh

where the bold italic text is colored.

Improve this answer
1

Here is another version that should work for passwd files:

 grep -E --color=auto ':[[:alnum:]]+:[[:digit:]]+:(1001|1003):' mypasswd

Explanation:

  • grep -E enables extended regular expressions, so you can use the OR syntax in (1001|1003). (Another consequence of adding -E is that you don't need to escape + and some other special characters.)
  • --color=auto is entirely optional; it merely adds colouring to the matched strings.
  • ':[[:alnum:]]+:[[:digit:]]+:(1001|1003):' matches a string consisting of a colon, followed by one or more alphanumeric characters, followed by a colon, followed by one or more digits, followed by a colon, followed by either 1001 or 1003, followed by another colon. (In a context where the the second field is always "x", you can replace [[:graph:]]+ with x.)
  • I am not making any assumptions about what is in the first column, so there is nothing before the first colon.

For the sample data in the question, this gives you the following output (with colouring removed):

 northj:x:1003:1003:Jim jones-North:/home/northj:/bin/csh
 denniss:x:1005:1003:Sue Dennis:/home/denniss:/bin/bash
 smitha:x:1050:1001:Amy Smith:/export/home/smitha:/bin/bash
 jonesc:x:1053:1001:Cathy Jones:/export/home/jonesc:/bin/ksh
 smithd:x:1055:1001:Dan Smith Jr:/export/home/smithd:/bin/csh
Improve this answer
1
  • 2
    Here, too, [[:graph:]]+ can eat through an additional colon, and the numbers might match on the comment field. You could use [^:]+ instead, but then the first colon might match the second colon in the line. Also the trailing (1001|1003) would match 10010, you'd need a : in the end to stop that. Using grep for stuff like this is tricky. Commented Nov 14, 2019 at 16:22
0

Hop 3 times and perch at the 3rd colon. Then perform a search from where we left off and look for 1001 or 1003 before hitting a colon. If search is a success, print the line.

$ perl -ne ' /:/g;//g;//g;/\G100[13]:/&&print' file

Gnu sed

$ sed -e 'h;s/:/\n/4;/:100[13]\n/!d;g' file
Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.