Selenium Python can't properly write csv in for loop

Question

I'm trying to write data in a for loop in a excel spreadsheet, i ultimately, want to have the code like the following:

f.write(ln.text + "," + av.text + "\n")

however, it will not work out the way i want it, so i have to do it this way, here the code below

Main.py

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import csv
from selenium.webdriver.support.ui import WebDriverWait
import unittest





class Test():
    # filename = "list.csv"
    # f = open(filename,"w")
    # headers = "listing name, aval \n"

    # f.write(headers)

    def setUp(self):
        self.driver = webdriver.Chrome()
        self.driver.get("https://orlando.craigslist.org/search/apa?s=100")
        self.filename = "list.csv"
        self.f = open(self.filename,"w")
        self.headers = "listing name, aval \n"
        self.f.write(self.headers)

    def task(self):
        driver = self.driver
        filename = self.filename
        f = self.f
        headers = self.headers
        ln = driver.find_elements_by_xpath('//*[@id="sortable-results"]/ul/li/p/a')
        size = len(driver.find_elements_by_xpath('//*[@id="sortable-results"]/ul/li/p/a'))
        for i in range(0, size):
            ln = driver.find_elements_by_xpath('//*[@id="sortable-results"]/ul/li/p/a')
            ln = ln[i]
            f.write(ln.text + ",")
            self.getLn(ln)
            ln = ln.click()
            av = driver.find_element_by_xpath('/html/body/section/section/section/div[1]/p[1]/span[2]')
            f.write(av.txt + "\n")
            self.getAv(av)

            back = driver.find_element_by_xpath('/html/body/section/header/nav/ul/li[3]/p/a').click()
            if i == 5:
                f.close()
                break





    def getLn(self,ln):
        driver = self.driver
        ln = ln
        if ln:
            print (ln.text)
        else:
            print ("No listing name")



    def getAv(self,av):
        driver = self.driver
        if av:
            print (av.text)
        else:
            print ("No listing name")


    def initialize():
        return Test

    def teardown(self):
        self.driver.quit()



run = Test()
run.setUp()
tas = run.task()
run.teardown()






if __name__ == '__main__':
    unittest.main()

It works, but it does't export on the csv like i want it too.

I ultimately want the output together in the same line like this:

f.write(ln.text + "," + av.text + "\n")

How would i do this ?

You never actually use the csv module... What is your question here? What do you want, and what are you getting? — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Mar 20, 2017 at 4:24
how would i write f.write(ln.text + "," + av.text + "\n") in the for loop i get AttributeError: 'NoneType' object has no attribute 'text' — BARNOWL
– BARNOWL, Commented Mar 20, 2017 at 4:44
If you are getting an error, you should post it in the question. — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Mar 20, 2017 at 7:04
@OwlMan That means that either ln or av contains None, i.e. the element you're searching for with driver.find_elements_by_xpath isn't finding what you're looking for. — Barmar
– Barmar, Commented Mar 20, 2017 at 20:41
Maybe the problem when you try to combine them is because of ln = ln.click(). After you do that, ln doesn't contain the element you found. Don't reuse variables like that, give it a different name. Or maybe you don't need to assign ln.click() to anything at all, since it doesn't return anything useful. — Barmar
– Barmar, Commented Mar 20, 2017 at 20:43

TemporalWolf · Accepted Answer · 2017-03-20 21:18:04Z

1

I would highly recommend taking your code to Code Review because it has serious stylistic issues, but this should solve your immediate problem:

for i in range(0, size):
    ln = driver.find_elements_by_xpath('//*[@id="sortable-results"]/ul/li/p/a')
    ln = ln[i]
    ln_text = ln.text  # copy the text for usage later
    self.getLn(ln)
    ln.click()
    av = driver.find_element_by_xpath('/html/body/section/section/section/div[1]/p[1]/span[2]')
    f.write(ln_text + ',' + av.text + "\n")  # write both parts
    self.getAv(av)

    back = driver.find_element_by_xpath('/html/body/section/header/nav/ul/li[3]/p/a').click()
    if i == 5:
        f.close()
        break

answered Mar 20, 2017 at 21:18

TemporalWolf

8,0121 gold badge33 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Selenium Python can't properly write csv in for loop

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related