command line interface for background python process

Question

How can I create a command line interface that persists as a background process and only executes commands when specific commands are entered? The following is pseudo code:

def startup():
   # costly startup that loads objects into memory once in a background process

def tear_down()
   # shut down; generally not needed, as main is acting like a service
   
def main():
    startup()

    # pseudo code that checks shell input for a match; after execution releases back to shell
    while True:
        usr_in = get_current_bash_command()
        if usr_in == "option a":
            # blocking action that release control back to shell when complete
        if usr_in == "option b":
            # something else with handling like option a
        if usr_in == "quit":
            # shuts down background process; will not be used frequently
            tear_down()
            break
   print("service has been shut down.  Bye!")

if __name__ == "__main__":
    # run non-blocking in background, only executing code if usr_in matches commands:
    main()

Note what this is not:

typical example of argparse or click, that runs a (blocking) python process until all commands are completed
a series of one-off scripts for each command; they utilize objects in memory instantiated in the background process with startup()
a completely different shell, like ipython; I'd like to integrate this with a standard shell, e.g. bash.

I am familiar with click, argparse, subprocess, etc., and as far as I can tell they accept a single command and block until completion. I specifically seek something that interacts with a background process so that expensive startup (loading objects into memory) is handled only once. I have looked at both python daemon and service packages, but I'm unsure if these are the right tool for the job also.

How can I accomplish this? I feel as though I don't know what to google to get the answer I need...

daemon is a great tool for the job. The command line runs, tries to communicate with the daemon and creates one if needed. The communication path is usually a socket but at a higher level interface with something like xmlrpc, http/rest, zeromq, and a list of a hundred other options. I didn't down vote you, but SO is more for specific programming problems, not a broad discussion of options. — tdelaney
– tdelaney, Commented Feb 16, 2020 at 5:19
Here are some references: python-daemon (doesn't work on Windows) can start a daemon. Its documentation is lousy, so use the source. Use its pidfile support to decide when a daemon should be started. daemoniker claims crossplatform support but I've never used it. As a first cut at client / server communciation, The XLMRPC client server example is a good first start for communicating. — tdelaney
– tdelaney, Commented Feb 16, 2020 at 6:20
can you comment on any tradeoffs with the flask app approach below? — anon01
– anon01, Commented Feb 16, 2020 at 6:23
flask is a great tool. It usually runs on web servers like apache or nginx but I think the idea here is to use its internal mini-web server. I don't use flask and it seems a bit heavy weight - but just my totally uninformed opinion here! My personal experience is with daemons and services using zeromq which works really well but has a bit of a learning curve. But when I throw something together quick and dirty XMLRPC is a good choice. Just IMHO. — tdelaney
– tdelaney, Commented Feb 16, 2020 at 6:37
You could research "microservices" or kubernetes because running a bunch of smaller distributed services seems to be the wave of the future (well, current also) — tdelaney
– tdelaney, Commented Feb 16, 2020 at 6:39

Joran Beasley · Accepted Answer · 2020-02-16 05:14:09Z

2

this is just one way you might do this... (there are other ways as well, this is just a pretty simple method)

you could use a server as your long running process (you would then turn it into a service using init systemV or upstart)

hardworker.py

import flask

app = flask.Flask("__main__")

@app.route("/command1")
def do_something():
    time.sleep(20)
    return json.dumps({"result":"OK"})

@app.route("/command2")
def do_something_else():
    time.sleep(26)
    return json.dumps({"result":"OK","reason":"Updated"})


if __name__ == "__main__":
   app.run(port=5000)

then your client could just make simple http requests against your "service"

thin_client.sh

if [[ $1 == "command1" ]]; then
   curl http://localhost:5000/command1
else if [[ $1 == "command2" ]]; then
   curl http://localhost:5000/command2
fi;

edited Feb 16, 2020 at 5:14

answered Feb 16, 2020 at 5:08

Joran Beasley

114k13 gold badges167 silver badges187 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Joran Beasley Over a year ago

any method to communicate between processes will require serialization

Joran Beasley Over a year ago

although i dont understand the problem ... people are not going to be looking at the terminal output for volumes of data ... so just do your work on the webpage (save an output file if you want) print a summary to the console (summaries are cheap to serialize) (remember the server is running on the same pc ... so it has full access to FS (and paths that might be arguments to the cli)

Collectives™ on Stack Overflow

command line interface for background python process

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related