Running Python Script Using Nginx and WSGI - Stuck

Question

I'm new to python and have been put on a task of building out a spreadsheet parser. I've created a python script that reads an xlsx file and parses the data. I have an Nginx server set up that this will be hosted on. I need this script to be an API endpoint so I can pass the parsed data back as JSON. I have been reading about WSGI for production server and have tried to follow the route of building that out. I am able to serve a path on the server and have it output the wsgi python script. The script has the following:

def application(environ, start_response):
status = '200 OK'
html = '<html>\n' \
       '<body>\n' \
       ' Hooray, mod_wsgi is working\n' \
       '</body>\n' \
       '</html>\n'
response_header = [('Content-type','text/html')]
start_response(status, response_header)
return [html]

I'm a little confused as to how to receive a request and send back json with my excel parser class? Thanks and I hope I'm being clear. I do have a flask server that works, but I do not know how to have it constantly running to serve my endpoint:

app = Flask(__name__)

@app.route('/parser/direct_energy', methods=['GET'])

def get_data(): return jsonify(commissions_data)

if name == 'main': app.run(host='0.0.0.0')

AKX · Accepted Answer · 2020-07-22 13:25:20Z

1

You don't want to use raw WSGI for this.

Use a package such as FastAPI (or Flask) to make everything easier for you.

For instance, using FastAPI, an app with an endpoint to receive a binary (Excel) file and return a JSON response is approximately

from fastapi import FastAPI, File, UploadFile

app = FastAPI()


@app.post("/process")
def process_file(file: UploadFile = File()):
    response = my_data_processing_function(data)
    return {"response": response}

See:

To get going: https://fastapi.tiangolo.com/tutorial/
To process files: https://fastapi.tiangolo.com/tutorial/request-files/
To deploy your service (behind Nginx): https://fastapi.tiangolo.com/deployment/

answered Jul 22, 2020 at 13:25

AKX

171k16 gold badges146 silver badges228 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

user3845941 Over a year ago

Thank you so much, I've also tried using flask and don't know how to constantly run the flask server so I can serve the route. I have this in my actual api script: app = Flask(name) @app.route('/parser/direct_energy', methods=['GET']) def get_data(): return jsonify(commissions_data) if name == 'main': app.run(host='0.0.0.0') However this doesn't work with my IP address

AKX Over a year ago

Flask also has instructions on how to run the server: flask.palletsprojects.com/en/1.1.x/deploying/wsgi-standalone/…

user3845941 Over a year ago

AKX - I tried gunicorn api:app and the server starts fine. It says it's Listening at: 127.0.0.1:8000, however how do I hit that endpoint on the server? My ip address with :8000 attached does not work

James Stevens · Accepted Answer · 2020-07-22 13:28:42Z

0

I use python/flask for development & gunicorn for production.

To get it to accept HTTP requests, I use function decorators. Its the most common way.

@application.route('/epp/api/v1.0/request', methods=['POST'])
def eppJSON():
    if flask.request.json is None:
        return abort(400, "No JSON data was POSTed")
    return jsonRequest(flask.request.json, flask.request.remote_addr)

So here, the url /epp/api/v1.0/request accepts POSTed JSON and returns JSON

When you run flask in dev mode it listens on http://127.0.0.1:5000

https://github.com/james-stevens/epp-restapi/blob/master/epprest.py https://github.com/james-stevens/dnsflsk/blob/master/dnsflsk.py

These are both python/flask projects of mine. Feel free to copy. They each run multiple instances of the python code in a single container load-balanced by nginx - pretty neat combination.

answered Jul 22, 2020 at 13:28

James Stevens

4393 silver badges8 bronze badges

1 Comment

user3845941 Over a year ago

Is there a reason why all my paths in flask besides '/' go to 404 not found with my NGinx server?

user3845941 · Accepted Answer · 2020-07-23 19:22:32Z

UPDATE

I got things working throug NGinx, flask, and GUnicorn. However, my flask app is only working when I go to '/'. If I go to a route such as /parser/de/v1 I get a 404 Not Found.
Here is my setup for NGinx:

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    # SSL configuration
    #
    # listen 443 ssl default_server;
    # listen [::]:443 ssl default_server;
    #
    # Note: You should disable gzip for SSL traffic.
    # See: https://bugs.debian.org/773332
    #
    # Read up on ssl_ciphers to ensure a secure configuration.
    # See: https://bugs.debian.org/765782
    #
    # Self signed certs generated by the ssl-cert package
    # Don't use them in a production server!
    #
    # include snippets/snakeoil.conf;

    root /var/www/html/excel_parser;

    # Add index.php to the list if you are using PHP
    index index.html index.htm index.nginx-debian.html index.php;

    server_name 208.97.141.147;

    location / {
            # First attempt to serve request as file, then
            # as directory, then fall back to displaying a 404.
            proxy_pass http://127.0.0.1:5000;
            proxy_connect_timeout 75s;
            proxy_read_timeout 300s;
            try_files $uri $uri/ =404;
    }

    # pass PHP scripts to FastCGI server
    #
    #location ~ \.php$ {
    #       include snippets/fastcgi-php.conf;
    #
    #       # With php-fpm (or other unix sockets):
    #       fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
    #       # With php-cgi (or other tcp sockets):
    #       fastcgi_pass 127.0.0.1:9000;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #       deny all;
    #}

James Stevens · Accepted Answer · 2020-07-24 13:56:55Z

my nginx.conf looks slightly different, partly becuase I am running multiple WSGI instances, then getting nginx to load-balance over them

worker_processes  3;
events {
    worker_connections  1024;
}
user daemon;
http {
    access_log      off;
    error_log       stderr error;
    include         mime.types;
    default_type    application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    upstream dns_servers {
        server unix:/ram/dnsflsk_1.sock;
        server unix:/ram/dnsflsk_2.sock;
        server unix:/ram/dnsflsk_3.sock;
        }

    server {
        listen 800 ssl;
        server_name localhost;
        ssl_certificate      certkey.pem;
        ssl_certificate_key  certkey.pem;
        ssl_session_cache    shared:SSL:1m;
        ssl_session_timeout  5m;
        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers  on;
        location / {
            proxy_pass http://dns_servers;
        }
    }
}

But with this, all the URLs are passed to the python/wsgi

Collectives™ on Stack Overflow

Running Python Script Using Nginx and WSGI - Stuck

4 Answers 4

3 Comments

1 Comment

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

3 Comments

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related