How to use Python to execute a cURL command?

Question

I want to execute a curl command in Python.

Usually, I just need to enter the command in the terminal and press the return key. However, I don't know how it works in Python.

The command shows below:

curl -d @request.json --header "Content-Type: application/json" https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere

There is a request.json file to be sent to get a response.

I searched a lot and got confused. I tried to write a piece of code, although I could not fully understand it and it didn't work.

import pycurl
import StringIO

response = StringIO.StringIO()
c = pycurl.Curl()
c.setopt(c.URL, 'https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere')
c.setopt(c.WRITEFUNCTION, response.write)
c.setopt(c.HTTPHEADER, ['Content-Type: application/json','Accept-Charset: UTF-8'])
c.setopt(c.POSTFIELDS, '@request.json')
c.perform()
c.close()
print response.getvalue()
response.close()

The error message is Parse Error. How to get a response from the server correctly?

Does this help? stackoverflow.com/questions/2667509/curl-alternative-in-python — br3w5
– br3w5, Commented Aug 25, 2014 at 17:25
FWIW, have you considered using pycurl the "Python binding to cURL" ? Depending your needs, it might be more efficient/convenient than invoking the command line utility behind the scene. — Sylvain Leroux
– Sylvain Leroux, Commented Aug 25, 2014 at 17:25
Do you need to use cURL? Have you considered Requests? Might be simpler, especially if you're new to python, which tends to be unforgiving. — vch
– vch, Commented Aug 25, 2014 at 17:26
I just wanted to also point you to this great answer on how to execute a shell command in Python: stackoverflow.com/a/92395/778533 — tommy.carstensen
– tommy.carstensen, Commented Apr 2, 2018 at 3:40

otorrillas · Accepted Answer · 2022-09-07 07:51:24Z

345

For the sake of simplicity, you should consider using the Requests library.

An example with JSON response content would be something like:

import requests
r = requests.get('https://github.com/timeline.json')
r.json()

If you look for further information, in the Quickstart section, they have lots of working examples.

For your specific curl translation:

import requests

url = 'https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere'
payload = open("request.json")
headers = {'content-type': 'application/json', 'Accept-Charset': 'UTF-8'}
r = requests.post(url, data=payload, headers=headers)

edited Sep 7, 2022 at 7:51

user3064538

answered Aug 25, 2014 at 17:33

otorrillas

4,8351 gold badge24 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

otorrillas Over a year ago

Please @tricknology, try to search for the bug and, if you do not happen to find a proper solution, post a new question.

tricknology Over a year ago

Should anyone else happen to see this, the reason why that was happening to me is that I was giving a string as my payload instead of a dictionary object.

Stephen Lead Over a year ago

It seems there's a small typo in the headers, which should read 'Accept-Charset': 'UTF-8'

geekQ Over a year ago

Requests is an extra dependency you need to install and manage. For a simple solution using just standard library, see stackoverflow.com/a/13921930/111995

blamblambunny Over a year ago

This is all very nice, but doesn't actually answer the question the OP posed?

|

Kyle Bridenstine · Accepted Answer · 2022-09-07 08:14:20Z

206

Use curlconverter.com. It'll convert almost any curl command into Python, Node.js, PHP, R, Go and more.

Example:

curl -X POST -H 'Content-type: application/json' --data '{"text":"Hello, World!"}' https://hooks.slack.com/services/asdfasdfasdf

becomes this in Python

import requests

json_data = {
    'text': 'Hello, World!',
}

response = requests.post('https://hooks.slack.com/services/asdfasdfasdf', json=json_data)

edited Sep 7, 2022 at 8:14

user3064538

answered Jun 5, 2018 at 16:02

Kyle Bridenstine

6,41312 gold badges67 silver badges106 bronze badges

2 Comments

w. Patrick Gale Over a year ago

Curl converter does not seem to work well with fields, such as curl -F name=John -F shoesize=11 https://example.com/. It treats them as files.

Benas Over a year ago

curlconverter.com was very useful!

Parishilan Rayamajhi · Accepted Answer · 2022-10-01 08:02:59Z

40

curl -d @request.json --header "Content-Type: application/json" https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere

its Python implementation looks like this:

import requests

headers = {
    'Content-Type': 'application/json',
}

params = {
    'key': 'mykeyhere',
}

with open('request.json') as f:
    data = f.read().replace('\n', '')

response = requests.post('https://www.googleapis.com/qpxExpress/v1/trips/search', params=params, headers=headers, data=data)

Check this link, it will help convert cURL commands to Python, PHP and Node.js

edited Oct 1, 2022 at 8:02

user3064538

answered Dec 28, 2017 at 10:38

Parishilan Rayamajhi

3,09727 silver badges24 bronze badges

Comments

Joran Beasley · Accepted Answer · 2016-06-14 16:01:19Z

28

import requests
url = "https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere"
data = requests.get(url).json

maybe?

if you are trying to send a file

files = {'request_file': open('request.json', 'rb')}
r = requests.post(url, files=files)
print r.text, print r.json

ahh thanks @LukasGraf now i better understand what his original code is doing

import requests,json
url = "https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere"
my_json_data = json.load(open("request.json"))
req = requests.post(url,data=my_json_data)
print req.text
print 
print req.json # maybe?

edited Jun 14, 2016 at 16:01

answered Aug 25, 2014 at 17:28

Joran Beasley

114k13 gold badges167 silver badges187 bronze badges

5 Comments

Lukas Graf Over a year ago

That doesn't include data from the requests.json file though, and doesn't set the Content-Type: application/json header - also, this will send a GET request, not a POST.

Lukas Graf Over a year ago

curl -d @<file> will read the fields to post from <file> - that's not the same as file upload.

Joran Beasley Over a year ago

@LukasGraf thanks :) ... I dont use curl much (read: almost never)

dpg5000 Over a year ago

One small note, data = requests.get(url).json should be data = requests.get(url).json()

Joran Beasley Over a year ago

in 2014 it was a property now its a function :) good call though

nikoskip · Accepted Answer · 2018-07-10 02:28:23Z

16

My answer is WRT python 2.6.2.

import commands

status, output = commands.getstatusoutput("curl -H \"Content-Type:application/json\" -k -u (few other parameters required) -X GET https://example.org -s")

print output

I apologize for not providing the required parameters 'coz it's confidential.

edited Jul 10, 2018 at 2:28

nikoskip

1,9201 gold badge22 silver badges38 bronze badges

answered Mar 15, 2018 at 16:41

apoorva_bhat

1791 silver badge5 bronze badges

3 Comments

nikoskip Over a year ago

If you need to use some specials options from curl like --resolve this is the way. Thank you.

Grant Gubatan Over a year ago

how can i only get the returned json without the tabular stat

Kristen Waite Over a year ago

In Python 3, commands has been replaced with subprocess - the above works using subprocess.getstatusoutput().

blamblambunny · Accepted Answer · 2022-09-07 07:59:09Z

13

I had this exact question because I had to do something to retrieve content, but all I had available was an old version of Python with inadequate SSL support. If you're on an older MacBook, you know what I'm talking about. In any case, curl runs fine from a shell (I suspect it has modern SSL support linked in) so sometimes you want to do this without using requests or urllib.request.

You can use the subprocess module to execute curl and get at the retrieved content:

import subprocess

# 'response' contains a []byte with the retrieved content.
# use '-s' to keep curl quiet while it does its job, but
# it's useful to omit that while you're still writing code
# so you know if curl is working
response = subprocess.check_output(['curl', '-s', baseURL % page_num])

Python 3's subprocess module also contains .run() with a number of useful options.

edited Sep 7, 2022 at 7:59

user3064538

answered Jul 7, 2019 at 20:58

blamblambunny

7559 silver badges20 bronze badges

1 Comment

Ângelo Polotto Over a year ago

I having the same issues with the company MacBook with proxy security analysis. I'll write my wapper around CURL to process some requests.

Mike 'Pomax' Kamermans · Accepted Answer · 2024-05-05 19:14:11Z

4

I use the built-in os module:

import os

os.system("sh script.sh")

script.sh literally only contains the curl.

edited May 5, 2024 at 19:14

Mike 'Pomax' Kamermans

54.3k17 gold badges133 silver badges182 bronze badges

answered Mar 28, 2021 at 6:37

Jacob Mead

651 silver badge2 bronze badges

1 Comment

Mike 'Pomax' Kamermans Over a year ago

Note that this won't be able to capture the output though.

chainstair · Accepted Answer · 2021-05-06 16:46:38Z

PYTHON 3

Only works within UNIX (Linux / Mac) (!)

Executing a cURL with Python 3 and parsing its JSON data.

import shlex
import json
import subprocess

# Make sure that cURL has Silent mode (--silent) activated
# otherwise we receive progress data inside err message later
cURL = r"""curl -X --silent POST http://www.test.testtestest/ -d 'username=test'"""

lCmd = shlex.split(cURL) # Splits cURL into an array

p = subprocess.Popen(lCmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate() # Get the output and the err message

json_data = json.loads(out.decode("utf-8"))

print(json_data) # Display now the data

Sometimes you also need to install these dependencies on UNIX if you experience strange errors:

# Dependencies  
sudo apt install libcurl4-openssl-dev libssl-dev
sudo apt install curl

TENLO CRAFT · Accepted Answer · 2021-12-17 13:36:26Z

1

if you os supporting curl you can do something like this:

import os

os.system("curl -d @request.json --header "Content-Type: application/json" https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere")

I'm using this way... And I think you can use this too!

by the way.. the module "os" is auto-installing when you install python. soo, you don't need to install packages ;)

answered Dec 17, 2021 at 13:36

TENLO CRAFT

191 bronze badge

Comments

BARIS KURT · Accepted Answer · 2022-09-13 13:43:37Z

1

use requests lib.. this code is :

curl -LH "Accept: text/x-bibliography; style=apa" https://doi.org/10.5438/0000-0C2G

equal to this:

import requests

headers = {
    'Accept': 'text/x-bibliography; style=apa',
}

r = requests.get('https://doi.org/10.5438/0000-0C2G', headers=headers)


print(r.text)

edited Sep 13, 2022 at 13:43

answered Mar 31, 2022 at 21:57

BARIS KURT

5385 silver badges16 bronze badges

Comments

MisterCat · Accepted Answer · 2023-12-13 08:31:09Z

To summarize all the options:

execute the command in Python using a subprocess.

For example:

import subprocess

# The command you would type in the terminal
command = ["curl", "https://www.example.com"]

# Run the command
result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)

# Get the output and error message (if any)
output = result.stdout
error = result.stderr

# Check if it was successful
if result.returncode == 0:
    print("Success:")
    print(output)
else:
    print("Error:")
    print(error)

Using pycurl library: An interface to libcurl
Use curl alternative to create an HTTP request. For example using requests library in Python.

Here is the full explanation link: Using cURL with Python and it's alternative.

liu alan · Accepted Answer · 2024-12-19 11:59:29Z

0

First of all , you should use "WRITEDATA" other than "WRITEFUNCTION". The later is for self defined function. Besides , python 3 is recommended. You can read the reference on the official website about the difference between WRITEDATA and WRITEFUNCTION.

https://curl.se/libcurl/c/CURLOPT_WRITEDATA.html
https://curl.se/libcurl/c/CURLOPT_WRITEFUNCTION.html

It works when I use WRITEDATA and python3 instead.

from io import BytesIO  
import pycurl  


response = BytesIO()  
c = pycurl.Curl()  
c.setopt(c.URL, 'https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere')  
c.setopt(c.WRITEDATA, response)  
c.setopt(c.HTTPHEADER, ['Content-Type: application/json','Accept-Charset: UTF-8'])  
c.setopt(c.POSTFIELDS, '@request.json')  
c.setopt(c.VERBOSE, 1)  
c.perform()  
c.close()  
res = response.getvalue().decode('utf-8')  
print(res)  
response.close()

edited Dec 19, 2024 at 11:59

answered Dec 18, 2024 at 14:45

liu alan

13 bronze badges

2 Comments

Anonymous Dec 18, 2024 at 20:06

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.

liu alan Dec 19, 2024 at 11:56

Much Thanks for your recommendation

Elisha Habinsky · Accepted Answer · 2022-08-30 06:50:57Z

-3

This is one approach:

Import os
import requests 
Data = os.execute(curl URL)
R= Data.json()

edited Aug 30, 2022 at 6:50

Elisha Habinsky

13110 bronze badges

answered May 9, 2018 at 20:49

AliRazaRizwe

331 bronze badge

1 Comment

SeanFromIT Over a year ago

os.system instead of os.execute, and requests seems unnecessary in this case

Collectives™ on Stack Overflow

How to use Python to execute a cURL command?

13 Answers 13

9 Comments

2 Comments

Comments

5 Comments

3 Comments

1 Comment

1 Comment

Comments

Comments

Comments

Comments

2 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

13 Answers 13

9 Comments

2 Comments

Comments

5 Comments

3 Comments

1 Comment

1 Comment

Comments

Comments

Comments

Comments

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related