73

I have some set of bash functions which output some information:

  • find-modelname-in-epson-ppds
  • find-modelname-in-samsung-ppds
  • find-modelname-in-hp-ppds
  • etc ...

I've been writing functions which read output and filter it:

function filter-epson {
    find-modelname-in-epson-ppds | sed <bla-blah-blah>
}

function filter-hp {
    find-modelname-in-hp-ppds | sed <the same bla-blah-blah>
}
etc ...

But the I thought that it would be better do something like this:

function filter-general {
    (somehow get input) | sed <bla-blah-blah>
}

and then call in another high-level functions:

function high-level-func {
    # outputs filtered information
    find-modelname-in-hp/epson/...-ppds | filter-general 
}

How can I achieve that with the best bash practices?

9
  • 2
    What's the difference? Only the brevity? But I think definition with function looks more expressive, isn't it? Commented Dec 22, 2012 at 17:22
  • 2
    What do you mean by more expressive? In this link you'll see that it's obsolete and deprecated (and not POSIX). Commented Dec 22, 2012 at 17:25
  • 1
    What do your functions find-modelname-... really look like and do? Maybe you should tell us more so that we can advice the best option. You're clearly trying to factor some pieces of code, but we need to know what it is exactly. Commented Dec 22, 2012 at 17:34
  • 1
    @gniourf_gniourf the link you reference says function fname { ... } is fine. What's deprecated is function fname() {...} Commented Oct 3, 2013 at 15:54
  • 1
    @gniourf_gniourf Yes, but it's not deprecated in bash as you indicated. Anyway the paren syntax is uglier and doesn't make sense; They aren't used to call the function and one doesn't pass parameters inside them. Commented Oct 3, 2013 at 17:09

6 Answers 6

94

If the question is How do I pass stdin to a bash function?, then the answer is:

Shellscript functions take stdin the ordinary way, as if they were commands or programs. :)

input.txt:

HELLO WORLD
HELLO BOB
NO MATCH

test.sh:

#!/bin/sh

myfunction() {
    grep HELLO
}

cat input.txt | myfunction

Output:

hobbes@metalbaby:~/scratch$ ./test.sh 
 HELLO WORLD 
 HELLO BOB 

Note that command line arguments are ALSO handled in the ordinary way, like this:

test2.sh:

#!/bin/sh

myfunction() {
    grep "$1"
}

cat input.txt | myfunction BOB

Output:

hobbes@metalbaby:~/scratch/$ ./test2.sh 
 HELLO BOB 
Sign up to request clarification or add additional context in comments.

7 Comments

This does not work on bash 4.1.0. I get the output: " HELLO WORLD \n HELLO BOB \n NO MATCH"
Ok, it's because of my usage of \n in the input string. I've edited the answer; it now just cats an input file, for the sake of simplicity. This question is about accessing stdin from a function--and that part works even in bash 2.05b. :)
Ah, yes... because echo does not support escape sequences. Another way to handle that is define a variable like NL="<hit Enter here>" and replace "\n" with "$NL". Working now. Thanks!
jpmc26, hi, Welcome to Stack Overflow. I hope this answer isn't really 'virtually useless'. The question is about the simple case, so the answer is about the simple case, too. I've never tried to pipe stdin to a function that contains more than one command. I'll play with that idea in my shell later. Meanwhile, if you have time, please post a new question about that.
You probably want to avoid the useless use of cat
|
38

To be painfully explicit that I'm piping from stdin, I sometimes write

cat - | ...

4 Comments

In my case, indeed I needed to be "painfully explicit", otherwise it didn't work.
@Mahdi What do you mean? Are there shells out there which require the use of cat - in this case/context? Which? When? How?
I have a script that runs as a mapper in a Hadoop map-reduce job, and the input contents are given to me via stdin. I needed to read the whole input into a file for further processing, and this solution worked for me.
This should be an accepted answer. It is short, simple and most importantly, straight to the point. Basically, man page for cat recommends doing it just like that. Or just cat | , but that is less explicit.
37

A very simple means to get stdin into a variable is to use read. By default, it reads file descriptor "0", i.e. stdin i.e., /dev/stdin.

Example Function:

input(){ local in; read in; echo you said $in; }                    

Example implementation:

echo "Hello World" | input               

Result:

you said Hello World

Additional info

You don't need to declare a variable as being local, of course. I just included that for the sake of good form. Plain old read in does what you need.

So you understand how read works, by default it reads data off the given file descriptor (or implicit stdin) and blocks until it encounters a newline. Much of the time, you'll find that will implicitly be attached to your input, even if you weren't aware of it. If you have a function that seems to hang with this mechanism just keep this detail in mind (there are other ways of using read to deal with that...).

More robust solutions

Adding on to the basic example, here's a variation that lets you pass the input via a stdin OR an argument:

input()
{ 
    local in=$1; if [ -z "$in" ]; then read in; fi
    echo you said $in
}

With that tweak, you could ALSO call the function like:

input "Hello World"

How about handling an stdin option plus other arguments? Many standard nix utilities, especially those which typically work with stdin/stdout adhere to the common practice of treating a dash - to mean "default", which contextually means either stdin or stdout, so you can follow the convention, and treat an argument specified as - to mean "stdin":

input()
{ 
    local a=$1; if [ "$a" == "-" ]; then read a; fi
    local b=$2
    echo you said $a $b
}

Call this like:

input "Hello" "World"

or

echo "Hello" | input - "World"

Going even further, there is actually no reason to only limit stdin to being an option for only the first argument! You might create a super flexible function that could use it for any of them...

input()
{ 
    local a=$1; if [ "$a" == "-" ]; then read a; fi
    local b=$2; if [ "$b" == "-" ]; then read b; fi
    echo you said $a $b
}

Why would you want that? Because you could formulate, and pipe in, whatever argument you might need...

myFunc | input "Hello" -

In this case, I pipe in the 2nd argument using the results of myFunc rather than the only having the option for the first.

6 Comments

Coming to this answer from the future - your example will only read the first line of multi-line input streamed to the function - is there a better way to grab 'all 'of stdin and use it in a single variable?
Hi future reader! Use the -d option for read to specify a different delimiter instead of the default newline. See: linuxcommand.org/lc3_man_pages/readh.html
I'm not sure what you're attempting to do exactly, but if reading a file it would probably make sense to use cat instead of read. If gathering directly typed input, you could prompt the user to end their text in some special character to denote the end of input.
Just came from the future too to say that though it doesn't collect a multiline string, it fitted the best for my case and it's much quicker than cat when I want to have it with [d]ash instead of bash. Running an external command such as cat is useless in my case because my function is only reading stdin to turn a string into an uppercase/lowercase, and your solution was great. By the way, you're my first bookmark on SE sites ever, so thank you very much!
|
8

Call sed directly. That's it.

function filter-general {
    sed <bla-blah-blah>
}

Comments

1

The first command that takes STDIN but no command is piped to it, receives the STDIN from the function/script itself, but it happens only once and the next command without pipe, receives empty STDIN:

function stdTest {
  echo 1:; echo "Local STDIN" | cat -n;
  echo 2:; cat -n;
  echo 3:; cat -n;
}

$ echo -e "Hey\nYou" | stdTest
1:
     1  Local STDIN
2:
     1  Hey
     2  You
3:

Comments

0

If you are asking how to avoid repeating the same sed snippet in each of your functions, a simple solution is to put that in a function.

blah_blah () {
    sed blah blah
}

filter-epson () {
    find-modelname-in-epson-ppds | blah_blah
}

filter-hp () {
    find-modelname-in-hp-ppds | blah_blah
}

In some situations, you might want to create a wrapper which postprocesses the output from whatever commands you pass in:

wrapper () {
    "$@" | sed blah blah
}

filter-epson () {
    wrapper find-modelname-in-epson-ppds
}

filter-hp () {
    wrapper find-modelname-in-hp-ppds --some options --perchance
}

Tangentially notice also the preference for POSIX function syntax over the Bash-only function keyword, which really adds no value here.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.