4

When running the following code python code I get the following code:

from urllib.request import urlopen
from bs4 import BeautifulSoup
import datetime
import random
import pymysql
import re

conn = pymysql.connect(host='127.0.0.1', user='root', passwd ='mypass', 
db='mysql',charset='utf8')

cur = conn.cursor()
cur.execute('USE scraping')

random.seed(datetime.datetime.now())

def store(title, content):

    cur.execute("DROP TABLE IF EXISTS pages")

     sql = """CREATE TABLE pages (id BIGINT(7) NOT NULL AUTO_INCREMENT, title VARCHAR(200)
        , content VARCHAR(10000), created TIMESTAMP DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY(id))"""
cur.execute(sql)

cur.execute("""INSERT INTO pages (title, content) VALUES ("%s", "%s")""", (title, content))
cur.connection.commit()

def getLinks(articleUrl):
     html = urlopen('http://en.wikipedia.org'+articleUrl)
     bs = BeautifulSoup(html, 'html.parser')
     title = bs.find('h1').get_text()
     content = bs.find('div', {'id':'mw-content-text'}).find('p').get_text
     store(title, content)
     #return bs.find('div',{'id':'bodyContent'}).findAll('a',href=re.compile('^(/wiki/)((?!:).)*$'))

links = getLinks('/wiki/Kevin_Bacon')
#print(links)

I get the following error message:

  AttributeError: 'function' object has no attribute 'translate'

From what I can tell the failure point seems to be at this point in the code:

cur.execute("""INSERT INTO pages (title, content) VALUES ("%s", "%s")""", (title, content))

I've tried trouble shooting the issue by looking at the following:

- File "C:\Users\mypath\PycharmProjects\Scraper\venv\lib\site-packages\pymysql\converters.py", line 118, in escape_unicode
return u"'%s'" % _escape_unicode(value)

- File "C:\Users\mypath\PycharmProjects\Scraper\venv\lib\site-packages\pymysql\converters.py", line 73, in _escape_unicode
return value.translate(_escape_table)

Any thoughts on what might be causing the issue?

1 Answer 1

1

You forgot to add parentheses in get_text function call, should be:

content = bs.find('div', {'id':'mw-content-text'}).find('p').get_text()
Sign up to request clarification or add additional context in comments.

1 Comment

Worked! Thanks for the help!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.