Multiple insert statements failing postgresql / psycopg2

Question

I'm using scrapy to scrape a webpage which I then add to a postgres database. The first INSERT statements works fine, and I can select the items from the database. The second one seems to insert data but all the fields are blank

           date            | count 
---------------------------+-------
 04/2013                   | 
 03/2013                   | 
 02/2013                   |

Here is my code:

#Database init
    self.conn = psycopg2.connect("dbname='dataproject' user='xxxx' host='localhost' password='xxxxxx'")
    self.cursor = self.conn.cursor()    

    #CSV files
    self.DatavisItemCsv = csv.writer(open('DatavisTable.csv', 'wb'))
    self.DatavisItemCsv.writerow(['dates', 'counts'])

def process_item(self, item, spider):
    self.DatavisItemCsv.writerow([item['dates'], item['counts']])

    date_list = item['dates']
    count_list = item['counts']

    for s in date_list:
        self.cursor.execute('INSERT INTO ufo_info(date) VALUES (%s);', [s])

    for c in count_list:
        self.cursor.execute('INSERT INTO ufo_info(count) VALUES (%s);', [c])

    self.conn.commit()

Does this have anything to do with my for loops? data race?

Clodoaldo Neto · Accepted Answer · 2013-05-08 15:21:50Z

1

for s, c in zip(date_list, count_list):
    self.cursor.execute(
        'INSERT INTO ufo_info(date, count) VALUES (%s, %s);'
        , (s, c)
    )

answered May 8, 2013 at 15:21

Clodoaldo Neto

127k30 gold badges251 silver badges274 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Donna Over a year ago

very smooth, than you sir!

Collectives™ on Stack Overflow

Multiple insert statements failing postgresql / psycopg2

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related