multidimensional array python

Question

I am creating this program. But I can't get it to insert data into the sqlite3 database.

class Spider(HTMLParser):

  def __init__(self, url):

        HTMLParser.__init__(self)
        req = urlopen(url)
        self.feed(req.read())


  def handle_starttag(self, tag, attrs):
        if tag == 'a' and attrs:
            print "Found link => %s" % attrs[0][1]
            cursor.execute("INSERT INTO queue VALUE((?), (?), (?))",(None,  attrs[0][0], attrs[0][1]))
            connection.commit()

if __name__ == '__main__':
    Spider(starturl)

I think the problem is in the following line.

cursor.execute("INSERT INTO queue VALUE((?), (?), (?))",(None,  attrs[0][0], attrs[0][1]))

Thanks in advance!

What exceptions are being thrown and how are you initializing your connection to the database? Do you realize that the attrs list might have attributes other than the "href" you seem to be looking for if the tag is "a"? — BenTrofatter
– BenTrofatter, Commented Jun 1, 2011 at 0:23

BenTrofatter · Accepted Answer · 2011-06-01 01:03:18Z

2

The VALUES keyword is plural, not singular. You don't have to get rid of the extra parentheses around the question marks, but they're not necessary. Also, you can explicitly tell the SQL server that you want to insert directly into specific columns rather than passing None for an auto-incrementing primary key.

cursor.execute("INSERT INTO queue (url, desc) VALUES (?, ?)", (attrs[0][0], attrs[0][1]))

That being said, you probably want to throw in an additional test to make sure that attrs[0][0] is actually href, because otherwise you'll be persisting any other classes, ids, or DOM events that are attached to the tag as attributes. Also, since the format of the attrs collection is something like [('href', 'http://example.com')], a description isn't really coming out of this, so you might consider renaming your database columns.

edited Jun 1, 2011 at 1:03

answered Jun 1, 2011 at 0:57

BenTrofatter

2,1581 gold badge13 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

multidimensional array python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related