I am trying to get the movies and series on the website https://www.jiocinema.com/search/avengers I have extracted the movies using selenium but I got to know about the xhr requests. I am new to the concept and don't know whether I can use the api or not ?
The API link is: https://prod.media.jio.com/apis/common/v3.1/search/search
The xhr response looks like 
Is there any way I can get the data from the above xhr response ?
You actually don't need selenium for this. You are calling a REST-API here.
Simply do something like this:
import requests
import traceback
def searchApi(query):
endpoint = "http://prod.media.jio.com/apis/common/v3.1/search/auto"
data = {
"q": query
}
try:
response = requests.post(endpoint, data=data)
if(response.status_code == 200):
for msg in response:
print(msg)
except Exception:
print(traceback.format_exc())
Usage:
searchApi("avengers")
Raw output:
{
"code": 200,
"message": "success",
"data": {
"items": [
{
"name": "avengers grimm",
"type": "Movies"
},
{
"name":"avengers endgame official trailer hindi ",
"type":"Videos"
},
{
"name":"avengers endgame official trailer",
"type":"Videos"
},
{
"name":"avengers endgame special look",
"type":"Videos"
}
.... continues
]
}
}
Alternatively, if you want to access the data-response directly.
import json
def searchApi(query):
endpoint = "http://prod.media.jio.com/apis/common/v3.1/search/auto"
data = {
"q": query
}
try:
response = requests.post(endpoint, data=data)
if(response.status_code == 200):
response = response.json()
for msg in response["data"]["items"]:
print("name: ", msg["name"], "type: ", msg["type"])
except Exception:
print(traceback.format_exc())
Formatted output msg["name"] and msg["type"]:
name: avengers grimm type: Movies
name: avengers endgame official trailer type: Videos
name: avengers endgame special look type: Videos
name: avengers endgame official trailer hindi type: Videos
name: the avengers earth s mightiest heroes type: TV Shows
name: marvel's avengers age of ultron type: Movies
name: marvel's avengers assemble type: TV Shows
name: marvel's avengers age of ultron official trailer hindi type: Videos
name: marvel's avengers age of ultron official trailer type: Videos
name: marvel's the avengers type: Movies
name: marvel's the avengers official trailer type: Videos
name: marvel's the avengers official trailer hindi type: Videos
name: making of south indian avengers type: Videos
You can use the requests library to make post requests like so...
import requests
headers = {'User-Agent':'Some user agent'}
data = requests.post('https://prod.media.jio.com/apis/common/v3.1/search/search',headers=headers).text
You might need headers to make the request...
