1

Im using get request to get data from a site.

captured reqeuest details :

Query String Parameters

s: vrnf8

following is my code . But i get 404 not found status code.

import requests

url = "https://www.fleetmon.com/search/?s=vrnf8"

pload = {"s":"vrnf8"}

resp = requests.get(url, params=pload)

print(resp.text)
print(resp.status_code)

please help me with this !

request capture file : https://filebin.net/ilx668e5yrdmbu7p

response must be this :

[[{"lastreport_class": "position-age-live", "course": 213, "callsign": "VRNF8", "lastreport_verbose": "2020-07-17 12:28 UTC", "speed": 15.6, "destination": "SGSIN PEBGA", "vessel_url": "/vessels/seaspan-amazon_9630391_8896228/", "imo": "9630391", "location": "South Kuroshio, JP", "latitude": 28.732660, "vesselid": "8896228", "mmsi": "477390400", "lastreport_timestamp": 1594985298, "lastreport_short": "21 min", "name_clean": "SEASPAN AMAZON", "vessel_type": "Container ship", "master_image_id": 2278121, "flag_id": "HK", "icon": "cargo", "is_moving": true, "name": "SEASPAN AMAZON", "longitude": 28.732660, "length": "337", "flag_name": "Hong Kong SAR of China"}], 1, [], 0]
Improve this question
3
  • 1
    https://www.fleetmon.com/search/?s=vrnf8 is not working Commented Jul 17, 2020 at 12:35
  • what does this link suggest? filebin.net/ilx668e5yrdmbu7p Commented Jul 17, 2020 at 12:50
  • @VishalSingh its the cptured request details file Commented Jul 18, 2020 at 11:55

2 Answers 2

Reset to default
1

Try this code below:

import requests

url = "https://www.fleetmon.com/search/?s=vrnf8"

headers = {
    'X-Requested-With': 'XMLHttpRequest'
}

response = requests.get(url, headers=headers)

print(response.json())

Result:

[[{'lastreport_class': 'position-age-live', 'course': 212, 'callsign': 'VRNF8', 'lastreport_verbose': '2020-07-17 13:22 UTC', 'speed': 15.9, 'destination': 'SGSIN PEBGA', 'vessel_url': '/vessels/seaspan-amazon_9630391_8896228/', 'imo': '9630391', 'location': 'South Kuroshio, JP', 'latitude': 28.540143, 'vesselid': '8896228', 'mmsi': '477390400', 'lastreport_timestamp': 1594988550, 'lastreport_short': '1 min', 'name_clean': 'SEASPAN AMAZON', 'vessel_type': 'Container ship', 'master_image_id': 2278121, 'flag_id': 'HK', 'icon': 'cargo', 'is_moving': True, 'name': 'SEASPAN AMAZON', 'longitude': 28.540143, 'length': '337', 'flag_name': 'Hong Kong SAR of China'}], 1, [], 0]
Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

how did you find the header for the request ?
@GHOSTH4CK3R It is the request header of ajax request mostly.
i meanthe headers = {x-requestedwith: } part. howd you find exactly whats missing header ?
How do I find it?By test one by one, by experience.You don't know the source code of the backend,and pages would always take some measures to prevent web-scraping.There are no uniform rules.
0 
import requests

url = "https://www.fleetmon.com/search/?s=vrnf8"

pload = {"s":"vrnf8"}

resp = requests.get(url, params=pload)

print(resp.text)
print(resp.status_code)

NOTE: The url you have typed doesn't exist

404 page not found

Correct Code

import requests

url = "https://www.fleetmon.com/search/"

pload = {"s":"vrnf8"}

resp = requests.get(url, params=pload)

print(resp.text)
print(resp.status_code)

But Now also you will get an error

Explanation

let's understand what's happening in your code

import requests

url = "https://www.fleetmon.com/search/?s=vrnf8"

pload = {"s":"vrnf8"}

resp = requests.get(url, params=pload)

'''
resp is equivalent to  request.get("https://www.fleetmon.com/search/?s=vrnf8/?s=vrnf8")
when we make get request to url, it appends /?query=value to it's requested url
and this url doesn't exist so you will get 404
'''

print(resp.text)
print(resp.status_code)

code that I sent


```python
import requests

url = "https://www.fleetmon.com/search"

pload = {"s":"vrnf8"}

resp = requests.get(url, params=pload)

'''
resp is equivalent to  request.get("https://www.fleetmon.com/search/?s=vrnf8")
when we make get request to url, it again appends /?query=value to it's requested url
and even this url also doesn't exist so you will again get 404
'''

print(resp.text)
print(resp.status_code)

NOTE: You can check weather a url exist or not by typing it in google search bar

Improve this answer

1 Comment

because i have less than 15 reputation i cant upvote

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.