Writing python list values to a csv file

Question

I'm trying to export the list generated to a csv file where each row in the website table corresponds to a new row in the file and each value is in an individual cell, such as:

NAME.....ICO DATE....ICO PRICE....CURR. PRICE....24 HR ROI Stratis.....06/20/16.......$0.007...........$7.480................+38.80%

The current output looks like this:

['Patientory\n05/31/17\n$0.104\n$0.274\n+46.11%\n+25.54%\nN/A']

import csv
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait as wait

csvrows = []

def get_css_sel(selector):
    posts = browser.find_elements_by_css_selector(selector)
    for post in posts:
        print(post.text)
        csvrows.append([post.text])

browser = webdriver.Chrome(executable_path=r'C:\Scrapers\chromedriver.exe')
browser.get("https://icostats.com")
wait(browser, 20).until(EC.presence_of_element_located((By.CSS_SELECTOR, "#app > div > div.container-0-16 > div.table-0-20 > div.tbody-0-21 > div:nth-child(2) > div:nth-child(8)")))

get_css_sel("#app > div > div.container-0-16 > div.table-0-20 > div.tableheader-0-50")              #fetch header of table
get_css_sel("#app > div > div.container-0-16 > div.table-0-20 > div.tbody-0-21 > div")              #fetch rows of table

def create_csv(thelist):
    with open('ICO.csv', 'w') as myfile:
        for i in thelist:
            wr = csv.writer(myfile, quoting=csv.QUOTE_ALL)
            wr.writerow([i])

create_csv(csvrows)

aneroid · Accepted Answer · 2017-07-03 03:06:43Z

2

In get_css_sel(), each post.text contains the row text separated by newlines \n - same as your example of the output. So appending [post.text] appends a list with a single item for the full row. Change that to:

csvrows.append(post.text.split('\n'))  # remove the extra list brackets
                                       # since split returns a list.

Ex:

>>> y = 'Patientory\n05/31/17\n$0.104\n$0.274\n+46.11%\n+25.54%\nN/A'
>>> y.split('\n')
['Patientory', '05/31/17', '$0.104', '$0.274', '+46.11%', '+25.54%', 'N/A']

Additionally, in your writing loop, you shouldn't re-create the csv.writer for every row, just do it once before looping over thelist.

And since you have all the rows you want in csvrows, you can use csvwriter.writerows directly.

def create_csv(thelist):
    with open('ICO.csv', 'w') as myfile:
        wr = csv.writer(myfile, quoting=csv.QUOTE_ALL)
        wr.writerows(thelist)

edited Jul 3, 2017 at 3:06

answered Jul 3, 2017 at 3:01

aneroid

16.7k3 gold badges42 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

tklein Over a year ago

That's it! Also, how can I remove the quotes? I can't call remove() on it: AttributeError: 'NoneType' object has no attribute 'remove'

aneroid Over a year ago

Which quotes are you trying to remove? In the CSV, you've put quoting=csv.QUOTE_ALL. Remove that if you don't want unnecessary quotes. Also, the default dialect is 'excel' which is usually sufficient.

aneroid Over a year ago

If you're getting empty rows and no text for post.text then put if post.text: before csvrows.append....

whackamadoodle3000 · Accepted Answer · 2017-07-03 03:34:45Z

Try this code:

import csv
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait as wait

csvrows = []
def get_css_sel(selector):
    posts = browser.find_elements_by_css_selector(selector)
    for post in posts:
        print(post.text)
        csvrows.append(post.text)

browser = webdriver.Chrome(executable_path=r'//Users/Pranavtadepalli/Downloads/chromedriver')
browser.get("https://icostats.com")
wait(browser, 20).until(EC.presence_of_element_located((By.CSS_SELECTOR, "#app > div > div.container-0-16 > div.table-0-20 > div.tbody-0-21 > div:nth-child(2) > div:nth-child(8)")))

get_css_sel("#app > div > div.container-0-16 > div.table-0-20 > div.tableheader-0-50")              #fetch header of table
get_css_sel("#app > div > div.container-0-16 > div.table-0-20 > div.tbody-0-21 > div")              #fetch rows of table
new=[",".join(elem.split("\n")) for elem in csvrows]
newfile=open("csvfile.csv",'r')
newfile1=open("csvfile.csv",'w')
newstuff=newfile.read()
for elem in new:
    newfile1.write(elem+'\n')
newfile1.close()
newfile.close()

Collectives™ on Stack Overflow

Writing python list values to a csv file

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related