Parse text response from http request in Python

Question

I am trying to fetch data from an APU but as response I am getting the plain text. I want to read all text line by line.

This is the url variable: http://www.amfiindia.com/spages/NAVAll.txt?t=23052017073640

First snippet:

from pymongo import MongoClient
import requests
from bs4 import BeautifulSoup as bs
url = "https://www.amfiindia.com/spages/NAVAll.txt?t=23052017073640"
request = requests.get(url)
soup = bs(request.text,"lxml")
for line in soup:
    print line
    break

Result: It prints out the entire text

Second snippet:

request = requests.get(url)
for line in request.text():
    print line
    break

Result: It prints out 1 character

request = requests.get(url)
requestText = request.text()
allMf = requestText.splitlines()

Result: Exception: 'unicode' object is not callable

I have tried few more cases but not able to read text line by line.

Are you sure that in the second snippet you loop over request.text() and not over request.text? And if yes, it should throw an exception because request.text is a property, not a method. Therefore, you don't need () — illright
– illright, Commented Aug 30, 2017 at 8:03

Danil Speransky · Accepted Answer · 2017-08-30 08:07:31Z

16

request.text is a property and not a method, request.text returns a unicode string, request.text() throws the error 'unicode' object is not callable.

for line in request.text.splitlines():
    print line

edited Aug 30, 2017 at 8:07

answered Aug 30, 2017 at 8:02

Danil Speransky

30.5k6 gold badges70 silver badges78 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

SajidSalim · Accepted Answer · 2017-08-30 09:42:34Z

1

import requests
from bs4 import BeautifulSoup as bs
url = "https://www.amfiindia.com/spages/NAVAll.txt?t=23052017073640"
request = requests.get(url)
soup = bs(request.text,"lxml")

# soup.text is to get the returned text
# split function, splits the entire text into different lines (using '\n') and stores in a list. You can define your own splitter.
# each line is stored as an element in the allLines list.
allLines = soup.text.split('\n') 

for line in allLines: # you iterate through the list, and print the single lines
    print(line)
    break # to just print the first line, to show this works

edited Aug 30, 2017 at 9:42

answered Aug 30, 2017 at 8:13

SajidSalim

3596 silver badges19 bronze badges

2 Comments

Zabuzard Over a year ago

Can you explain your code and why it solves the issue? Code only answers are often not that helpful.

SajidSalim Over a year ago

Added the explanation.

Beomi · Accepted Answer · 2017-08-30 08:08:04Z

0

Try this:

from pymongo import MongoClient
import requests
from bs4 import BeautifulSoup as bs
url = "https://www.amfiindia.com/spages/NAVAll.txt?t=23052017073640"
request = requests.get(url)
soup = bs(request.text,"lxml")
for line in soup:
    print line.text
    break

answered Aug 30, 2017 at 8:08

Beomi

1,73716 silver badges18 bronze badges

Collectives™ on Stack Overflow

Parse text response from http request in Python

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related