How to encode strings to API requests in Python?

Question

I'm trying to filter data from an API request to get only registers that were created or updated yesterday.

There are some notes listed in the documentation, some are shown below.

2. The query must be URL encoded
4. Query string must be enclosed between a pair of double quotes and can have up to 512 characters
5. Logical operators AND, OR along with parentheses () can be used to group conditions
6. Relational operators greater than or equal to :> and less than or equal to :< can be used along with date fields and numeric fields

The format of the request is: .../api/v2/search/tickets?query="created_at:>'2017-01-01' OR updated_at:>'2017-01-01'"

Url encoded it becomes: .../api/v2/search/tickets?query="created_at:>%272017-01-01%27%20OR%20updated_at:>%272017-01-01%27"

I wrote the following code:

import datetime as dt
import urllib.parse
import requests

url = f".../api/v2/search/tickets"
params={}

yesterday = dt.date.today() - dt.timedelta(days = 1)
query = urllib.parse.quote(f"created_at:>{yesterday} OR updated_at:>{yesterday}")
params["query"] = query

response = requests.get(url, auth=(api_key, "X"), params=params)

I'm using urllib.parse.quote to URL encode the string but I'm getting a different format if compared to the example provided from the API documentation.

.../tickets?query=created_at%253A%253E%25272022-10-30%2527%2520OR%2520updated_at%253A%253E%25272022-10-30%2527

Error raised: requests.exceptions.HTTPError: 400 Client Error: Bad Request for url

I can't find a way to keep the double quotes, I tried to enclose the double quotes in single quotes but it doesn't work.

I can't find a way to encode the filter string as the example shown in the API Documentation.

Any help?

There are no double quotes in your string. Remember that requests is perfectly capable of URL-encoding your data. That's what has happened here -- it's getting double-encoded. Note the %25, which is a % sign. — Tim Roberts
– Tim Roberts, Commented Oct 31, 2022 at 22:46
Also note that > is a character that gets translated during URL encoding. There should not be > signs in your final URL. — Tim Roberts
– Tim Roberts, Commented Oct 31, 2022 at 22:51

Tim Roberts · Accepted Answer · 2022-11-01 16:58:29Z

2

Here's the result if you let requests do the encoding. This is proper URL encoding:

import datetime as dt
import urllib.parse
import requests

url = "http://timr.example.com/api/v2/search/tickets"
api_key = "7"
params={}

yesterday = dt.date.today() - dt.timedelta(days = 1)
query = f'"created_at:>{yesterday} OR updated_at:>{yesterday}"'
params["query"] = query

response = requests.get(url, auth=(api_key, "X"), params=params)
print(response.url)

Output:

http://timr.example.com/api/v2/search/tickets?query=%22created_at%3A%3E2022-10-30+OR+updated_at%3A%3E2022-10-30%22

FOLLOWUP

OK, there's a way you can do this, by interrupting requests automated processing. You have it prepare almost everything, then you insert your fake encoded query at the end:

import datetime
import requests

def fakeencode(s):
    return s.replace("'","%27").replace(' ','%20')

query = "created_at:>'2017-01-01' OR updated_at:>'2017-01-01'"

url = "http://timr.example.com/api/v2/search/tickets"
api_key = "7"

yesterday = datetime.date.today() - datetime.timedelta(days = 1)
query = f"created_at:>'{yesterday}' OR updated_at:>'{yesterday}'"
query = '?query="'+fakeencode(query)+'"'

sess = requests.Session()
req = requests.Request('GET', url, auth=(api_key,"X"))
p = req.prepare()
p.url += query
response = sess.send(p)
print(response.url)

Output:

http://timr.example.com/api/v2/search/ticketsquery="created_at:>%272022-10-31%27%20OR%20updated_at:>%272022-10-31%27"

edited Nov 1, 2022 at 16:58

answered Oct 31, 2022 at 22:52

Tim Roberts

55.3k4 gold badges28 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Luiz Over a year ago

Hi Tim, I keep getting the same error. You didn't use any library to encode the string, you said that the requests library is capable of URL-encoding the data but in your example the encoded url is different from the expected format: query="created_at:>%272017-01-01%27%20OR%20updated_at:>%272017-01-01%27".

Luiz Over a year ago

Here is the link of the documentation: developers.freshdesk.com/api/#filter_tickets:~:text=%27https%3A/…

Tim Roberts Over a year ago

That's not proper URL encoding. You may have to do the translation yourself and build up a URL, instead of using the convenient params handling in requests.

Collectives™ on Stack Overflow

How to encode strings to API requests in Python?

1 Answer 1

FOLLOWUP

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

FOLLOWUP

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related