1

I've scoured various message boards to understand why I can't use a variable as input to a command in certain scenarios. Is it a STDIN issue/limitation? Why does using echo and here strings fix the problem?

For example,

~$ myvar=$(ls -l)
~$ grep Jan "$myvar"
grep: total 9
-rwxr-xr-x 1 jvp jvp 561 Feb  2 23:59 directoryscript.sh
-rw-rw-rw- 1 jvp jvp   0 Jan 15 10:30 example1
drwxrwxrwx 2 jvp jvp   0 Jan 19 21:54 linuxtutorialwork
-rw-rw-rw- 1 jvp jvp   0 Jan 15 13:08 littlefile
-rw-rw-rw- 1 jvp jvp   0 Jan 19 21:54 man
drwxrwxrwx 2 jvp jvp   0 Feb  2 20:33 projectbackups
-rwxr-xr-x 1 jvp jvp 614 Feb  2 20:41 projectbackup.sh
drwxrwxrwx 2 jvp jvp   0 Feb  2 20:32 projects
-rw-rw-rw- 1 jvp jvp   0 Jan 19 21:54 test1
-rw-rw-rw- 1 jvp jvp   0 Jan 19 21:54 test2
-rw-rw-rw- 1 jvp jvp   0 Jan 19 21:54 test3: File name too long

As you can see I get the error... 'File name too long'

Now, I am able to get this to work by using either:

  echo "$myvar" | grep Jan  

  grep Jan <<< "$myvar"

However, I'm really after a better understanding of why this is the way it is. Perhaps I am missing something about basics of command substitution or what is an acceptable form of STDIN.

Improve this question
4
  • I think there is no issue here. You seem to be having a file with the name test3: File name too long in your directory. BTW, using ls output programmatically is never considered a good practice. If you are interested in finding files that have been modified since certain time, there are other commands like find. Commented Feb 5, 2017 at 16:02
  • @codeforester No, there is no such file in the directory, it is just that the format of the error message correctly raised is confusing. Commented Feb 5, 2017 at 16:06
  • If it were an error, it wouldn't have ended up OP's variable since stderr isn't being assigned to the myvar. Commented Feb 5, 2017 at 16:08
  • 2
    What you see when you get that error message is not the grep output, but only an error message which indeed here looks like a file listing. But it is not. grep interprets the whole input (the output of your ls command) as a single file name argument to grep through. That file name is so long that an error is raised. No magic here. The issue is that the error message contains the proposed file name again. Which looks like a file listing, so a grep output. But as said, it is not. Check yourself: run your grep command with an arbitrary string that exceeds say 20 lines. Commented Feb 5, 2017 at 16:09

3 Answers 3

Reset to default
2

The grep utility can operate...

  1. On files the names of which are provided on the command line, after the regular expression used for matching
  2. On a stream supplied on its standard input.

You are doing this :

myvar=$(ls -l)
grep Jan "$myvar"

This provides the content of variable myvar as an argument to the grep command, and since it is not a file name, it does not work.

There are many ways to achieve your goal. Here are a few examples.

Use the content of the variable as a stream connected to the standard input of grep, with one of the following methods (all providing the same output) :

grep Jan <<<"$myvar" 
echo "$myvar" | grep Jan
grep Jan < <(echo "$myvar")

Avoid the variable to start with, and send the output of ls directly to grep :

ls -l | grep Jan
grep Jan < <(ls -l)

Provide grep with an expression that actually is a file name :

grep Jan <(ls -l)

The <(ls -l) expression is syntax that causes a FIFO (first-in first-out) special file to be created. The ls -l command sends its output to FIFO. The expression is converted by Bash to an actual file name that can be used for reading.

To clear any confusion, the two statements below (already shown above) look similar, but are fundamentally very different :

grep Jan <(ls -l)
grep Jan < <(ls -l)

In the first one, grep receives a file name as an argument and reads this files. In the second case, the additional < (whitespace between the two < is important) creates a redirection that reads the FIFO and feeds its output to the standard input of grep. There is a FIFO in both cases, but it is presented in a totally different way to the command.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Thanks, I now can say I have a better understanding of these fundamental concepts. It was not clear to me that the variable was providing a 'STRING' which is neither a FILE' nor is it provided via 'STDIN' stream. I imagine this is the case for most commands? Also, could you give an example or clarify the two different versions of FIFO, why use one vs the other?
1

I think there's a fundamental misunderstanding of how Unix tools/Bash operates here.

It appears what you're trying to do here is store the output of ls in a variable (which is something you shouldn't do for other reasons) and trying to grep across the string stored inside that variable using grep.

This is not how grep works. If you look at the man page for grep, it says:

SYNOPSIS

   grep [OPTIONS] PATTERN [FILE...]
   grep [OPTIONS] [-e PATTERN | -f FILE] [FILE...]

DESCRIPTION

   grep searches the named input FILEs for lines containing a match to
   the given PATTERN.  If no files are specified, or if the file “-” is
   given, grep searches standard input.  By default, grep prints the
   matching lines.

Note that it specifically says "grep searches the named input FILEs". Then it goes on to say "If no files are specified [...] grep searches standard input".

In other words, by definition grep does not search over strings. It searches files. Therefore you can not pass grep a string, via a bash variable.

When you type

grep Jan "$myvar"

Based on the syntax, grep thinks "Jan" is the PATTERN and the entire string in "$myvar" is a FILEname. Hence the error File name too long.

When you write 

echo "$myvar" | grep Jan

What you're now doing is making bash output the contents of "$myvar" to standard output. The | (pipe operator) in bash, connects the stdout (standard output) of the echo command, to the stdin (standard input) of the grep command. As noted above, when you omit the FILEname parameter to grep, it searches for a string in it's stdin by default, which is why this works.

Improve this answer

Comments

0

Grep takes as command line parameters files, not direct strings. You do indeed need the echo to make grep search in your variable.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.