Open csv file in Flask sent as binary data with curl

Question

I am working on a challenge in which I have to create a Flask app to train a Machine Learning model using a csv file. This csv file is sent to my Flask app using the following curl command:

curl \
        --silent \
        --fail \
        --data-binary @train.csv \
        --header "Content-Type: text/csv"

This curl command is pre-defined and so I can't change it to suit my needs. Also I don't have a front-end component with a form, only the Flask app.

I am trying to open the train.csv file in Flask but I have been unsucessful. I have tried many different ways but nothing works. How can you open this file in Flask using this command? Specifically a csv file sent as binary data with no variable name like so [email protected]

Here is a snippet of some code I have tried to open the file:

# Imports
import os
from flask import Flask, jsonify, request, make_response, session
import numpy as np
import pandas as pd

app = Flask(__name__)

@app.route('/genres/train', methods=['POST'])
def train_model():
    # Check if the POST request has the file part
    #if 'file' not in request.files:
        #return jsonify({'error': 'No file part'})
  
    # Get uploaded file
    #file = request.files['file']
    file = request.get_data()

    # If user does not select file, raise error
    if file.filename == '':
        return jsonify({'error': 'No selected file'})

    # If a file has been selected and it has csv extension then it can proceed
    if file and file.filename.split('.', 1)[1].lower() == 'csv':
        #with open('test.war', 'rb') as f:
        data = pd.read_csv(file, delimiter=',')

Note that request.files['file'] was used by me to test my Flask app using Postman to check that it worked, which it did.

cizario · Accepted Answer · 2020-07-01 21:34:55Z

2

if i understand your question, for testing purposes you want to send/post/upload a .csv file via curl command to this endpoint /genres/train to process the file with pandas, if so i would recommend you this tutorial Upload files with CURL

To upload files with CURL, many people make mistakes that thinking to use -X POST as regular form data; in facts, that way will cause errors.

The proper way to upload files with CURL is to use -F ( — form) option, which will add enctype=”multipart/form-data” to the request.

$ curl -F "NAME_FILEINPUT=@path/to/local/file" UPLOAD_ADDRESS

in your case, you need first to change the code little bit

@app.route('/genres/train', methods=['POST'])
def train_model():

[..]

    # i assume you already have a form with a file input with "file" as parameter name
    file = request.files['file']
    # file = request.get_data()

[..]

and then try this command:

$ curl -F "file=@path/to/local/file.csv" http://localhost:5000/genres/train

answered Jul 1, 2020 at 21:34

cizario

4,2843 gold badges16 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Luis Barroso Over a year ago

Thank you for your answer! I am working on this challenge and the curl command is not mine, it is pre-defined so I can't change it. I do not have a form in html, I only have the Flask app and I used Postman to test it by myself first and that is why I used the 'request.files ['file']' which worked. But in reality the data is sent to my Flask App using that curl command which sends it as binary data with no variable name like 'file=filename'. This is where I run into a problem because I do not know how to access the data if it does not have a variable name and it is also stored as binary data.

Luis Barroso · Accepted Answer · 2020-07-08 07:05:15Z

1

I managed to find a solution on my own. Here is the code I used to read the csv file sent as binary data:

@app.route("/genres/train", methods=["POST"])
def train_model():
    # Decode the request
    data = request.data.decode("utf-8")

    # Write data from the request in a local csv file
    train_csv = "train_local.csv"
    f = open(train_csv, "w", encoding="utf-8")
    f.write(data)
    f.close()

    # Load the train csv file as a DataFrame
    train_df = pd.read_csv(train_csv)

answered Jul 8, 2020 at 7:05

Luis Barroso

139 bronze badges

Collectives™ on Stack Overflow

Open csv file in Flask sent as binary data with curl

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related