from selenium import webdriver
driver = webdriver.Chrome()
driver.get("http://www.rezultati.com/utakmica/Q7ckEKB0/#detalji")
print (driver.current_url)
I have list of links:
http://www.rezultati.com/utakmica/ll33wwis/#detalji http://www.rezultati.com/utakmica/zLgwjGzm/#detalji
...
What's the best way of scrape data from all this links(using a loop?), in this program? Please suggest me some example or documentation.
Please check below code, Here I am unable to test(or fetch data)
from bs4 import BeautifulSoup
from selenium import webdriver
class ReadBooksInfo(object):
def __init__(self):
self.driver = webdriver.Chrome()
# self.driver = webdriver.Firefox()
def read_HTML(self, url):
self.driver.get(url)
html = self.driver.page_source
soup = BeautifulSoup(html.text, 'html.parser')
print(soup)
# check for your expected tag here,
rows = soup.find_all('tr')
print(rows)
test = ReadBooksInfo()
urls = ["http://www.rezultati.com/utakmica/Q7ckEKB0/#detalji",
"http://www.rezultati.com/utakmica/ll33wwis/#detalji",
"http://www.rezultati.com/utakmica/zLgwjGzm/#detalji"]
for i, url in enumerate(urls):
print(str(i) + "::" + url)
test.read_HTML(url)
I used urllib2 and BeautifulSoup. But without Selenium. Look at my code on Github. Code is not ideal and perfect, but I scrapped needed data. Github repo
