How send parameters through a pipe in a python script using argparse?

Question

I have to create a python script that can be used with Linux pipes

I want to run an script where some parameters can be send with a pipe or in the same line

Some examples of the use of my script with the expected output:

echo "a" > list.txt
echo "b" >> list.txt

./run.py p1 p2   # ['p1', 'p2'] expected output
cat list.txt | ./run.py  # ['a', 'b'] expected output
cat list.txt | ./run.py p1 p2 # ['p1', 'p2', 'a', 'b'] expected output

I tried:

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('args', nargs=argparse.REMAINDER)
args = parser.parse_args().args
print args

It works only with the parameters in the same line:

./run.py p1 p2  #['p1', 'p2'] OK
cat list.txt | ./run.py  # []  Not OK
cat list.txt | ./run.py p1 p2 # ['p1', 'p2'] expected output

Directly piping arguments doesn't work here since there isn't a stdin handler in run.py, without modifying the original python script, tools like xargs can help build and execute the correct command line with arguments piped from stdin — etopylight
– etopylight, Commented Oct 24, 2017 at 4:06
I mean if you want argparse to read from stdin, you can use sys.stdin, for example by adding parser.add_argument('infile', nargs='?', type=argparse.FileType('r'), default=sys.stdin) you can retrieve the input from parser.parse_args().infile too — etopylight
– etopylight, Commented Oct 24, 2017 at 9:33

etopylight · Accepted Answer · 2017-10-24 11:31:37Z

11

A solution by using only argparse

import argparse
import sys

parser = argparse.ArgumentParser()
parser.add_argument('args', nargs=argparse.REMAINDER)
parser.add_argument('stdin', nargs='?', type=argparse.FileType('r'), default=sys.stdin)
args = parser.parse_args().args

if not sys.stdin.isatty():
    stdin = parser.parse_args().stdin.read().splitlines()
else:
    stdin = []

print(args + stdin)

nargs='?' makes stdin optional and sys.stdin.isatty() checks if sys.stdin is empty

edited Oct 24, 2017 at 11:31

answered Oct 24, 2017 at 9:50

etopylight

1,3391 gold badge10 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

ShadowRanger Over a year ago

This solution makes no sense. Having the first argument use nargs=argparse.REMAINDER means all the arguments are collected into args, always, while stdin is always the default sys.stdin, it never comes from the command line. So all of this code essentially boils down to args = sys.argv[1:] (because that's what collecting all the arguments as args does), if not sys.stdin.isatty(): stdin = sys.stdin.read().splitlines() (because parser.parse_args().stdin will always be sys.stdin, since it's impossible for any argument to replace the default), else: stdin = [].

user707650 · Accepted Answer · 2017-10-24 03:01:06Z

4

I find xargs useful in such a case.

I haven't tried myself, but perhaps

cat list.txt | xargs ./run.py p1 p2

works for you?

In case you need to be specific where the arguments go, you can use the xargs placeholder option -J:

cat list.txt | xargs -J{} ./run.py p1 {} p2

would put "a b" between "p1" and "p2".

answered Oct 24, 2017 at 3:01

user707650

1 Comment

Troncador Over a year ago

It solves my problem, but it will be nice to solve uniquely with python. If no one responds, I will approve your answer.

mattmc3 · Accepted Answer · 2022-08-07 00:20:04Z

I recommend not not adding an arg for stdin because it just jacks up your argparse help. Instead, add a regular positional argument, and then if stdin was provided, simply read it and assign to that argument.

#!/usr/bin/env python3

import sys
import argparse


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("-f", "--flag", action="store_true", help="set a flag")
    parser.add_argument("PARAMS", nargs="*")
    args = parser.parse_args()
    if not sys.stdin.isatty():
        #print("stdin detected!")
        args.PARAMS.extend(sys.stdin.read().splitlines())
    print(repr(args))


if __name__ == "__main__":
    main()

This method gives you good help:

$ ./run.py -h          
usage: run.py [-h] [-f] [PARAMS ...]

positional arguments:
  PARAMS

optional arguments:
  -h, --help  show this help message and exit
  -f, --flag  set a flag

And it does what you're asking for in the question:

./run.py p1 p2                   # Namespace(flag=False, PARAMS=['p1', 'p2'])
printf '%s\n' a b | ./run.py -f  # Namespace(flag=True, PARAMS=['a', 'b'])
cat list.txt | ./run.py a b      # Namespace(flag=False, PARAMS=['a', 'b', 'x', 'y', 'z'])

mechsin · Accepted Answer · 2022-08-15 00:05:53Z

So when I first ran across this post I did something similar to mattmc3 answer. However later on I pondered if we couldn't do this with an custom action, below is the result. It is a lot more code than the other answers but is easier to wrap into a module and reuse in multiple places. Because of the way I use super this only works in Python 3 but could be easily modified to work in Python 2 if you still need it.

#!/usr/bin/env python3

import argparse
import sys

class ExtendFromPipe(argparse._StoreAction):

    def __init__(self, *pargs, **kwargs):
        super().__init__(*pargs, **kwargs)
        # Values from STDIN will extend a list so forcing nargs to '*' will
        # ensure this argument always creates a list.
        self.nargs = '*'

    def __call__(self, parser, namespace, values, option_string=None):
        # Calling super here ensures that there will be a default list
        # After we check to see if the STDIN is coming from a TTY interface
        # if we are being piped information this will be False. We then give
        # a default type conversion if there wasn't one provide and split
        # the input lines from the STDIN and convert them using the type
        # We then get the current value from the name space extend it with
        # the STDIN values and then update the namespace with the new values.
        super().__call__(parser, namespace, values, option_string)
        if not sys.stdin.isatty():
            typecon = self.type if self.type else str
            fromstdin = [typecon(k) for k in sys.stdin.read().splitlines()]
            temp = getattr(namespace, self.dest)
            temp.extend(fromstdin)
            setattr(namespace, self.dest, temp)

if __name__ == "__main__":

    desc = 'Implements Action class that reads from STDIN'
    parser = argparse.ArgumentParser(description=desc)

    parser.add_argument('input', action=ExtendFromPipe)

    cli_args = parser.parse_args()

    print(cli_args.input)

This returns the outputs exactly as requested it will even take care type conversions if passed and all inside the confines of the argparser framework.

Collectives™ on Stack Overflow

How send parameters through a pipe in a python script using argparse?

4 Answers 4

1 Comment

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related