How save an string output after a for loop in python with pandas and csv modules?

Question

I have the following for loop:

    for titles in titles:
        title = titles.xpath("a/text()").extract()
        link = titles.xpath("a/@href").extract()
        print(title, link)

How can I dump the title and link into a formmated .csv file?

I am pretty sure that this a well-hashed out issue that you can Google pretty much as you typed it. — Mad Physicist
– Mad Physicist, Commented Oct 4, 2016 at 16:25

Community · Accepted Answer · 2017-05-23 11:46:39Z

3

You should use the python CSV module. Look here for more information: Writing List of Strings to Excel CSV File in Python.

Here is an example for your problem:

import csv
results = []

# your code... also add this to your for loop.
    results.append([title, link])

csv_file = open("csv_file.csv",'wb')
wr = csv.writer(csv_file)
for row in results:
    wr.writerow(row)

edited May 23, 2017 at 11:46

CommunityBot

11 silver badge

answered Oct 4, 2016 at 16:31

Preston Hager

1,6491 gold badge20 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

john doe Over a year ago

Thanks for the help preston!... when I tried this approach the csv file is blank... I got the following exception:

File "/Users/user/PycharmProjects/scrapy_tests/ttest/spiders/test.py", line 28, in parse     wr.writerow(row) TypeError: a bytes-like object is required, not 'str

Dimitris Fasarakis Hilliard Over a year ago

Change 'wb' to 'w', can't see why you opened in binary mode.

Preston Hager Over a year ago

You can use either I think, the examples I found used binary writing so that's what I used.

berna1111 · Accepted Answer · 2016-10-12 12:19:46Z

2

Something like this (documentation):

...
doc_path = "path/to/doc.csv" # must exist, first folder on the place
                             # where you are running the script!
with open(doc_path, 'a') as doc:
    for entry in titles:
        title = entry.xpath("a/text()").extract()
        link  = entry.xpath("a/@href").extract()
        print(title, link)
        doc.write(str(title)+','+str(link)+'\n')

edited Oct 12, 2016 at 12:19

answered Oct 4, 2016 at 16:31

berna1111

1,8611 gold badge18 silver badges24 bronze badges

2 Comments

john doe Over a year ago

Thanks berna!... what about something with pandas?

berna1111 Over a year ago

You could use something like pandas.DataFrame.to_csv, but even people using pandas use traditional write (example).

Tim D · Accepted Answer · 2016-10-04 16:25:00Z

1

Use a csv writer.

Basic usage:

import csv
with open('file.csv') as f:
    writer = csv.writer(f)
    writer.writerow(["title", "link"])

answered Oct 4, 2016 at 16:25

Tim D

5084 silver badges4 bronze badges

Comments

python必须死 · Accepted Answer · 2016-10-04 16:36:35Z

1

result = []
for titles in titles:
    title = titles.xpath("a/text()").extract()
    link = titles.xpath("a/@href").extract()
    print(title, link)
    result += [title + ',' + link + '\n']

with open("result.csv" "w+") as f:
    f.writelines(result)

answered Oct 4, 2016 at 16:36

python必须死

1,08911 silver badges12 bronze badges

1 Comment

john doe Over a year ago

Thanks!... I got this exception: ` line 24, in parse result += [title + ',' + link + '\n'] TypeError: can only concatenate list (not "str") to list`

Collectives™ on Stack Overflow

How save an string output after a for loop in python with pandas and csv modules?

4 Answers 4

3 Comments

2 Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

3 Comments

2 Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related