I've been using phantomjs along with selenium's webdriver in order to parse a page off it's information. The problem is in order to keep parsing I need to click the next button since the information is javascript generated, I've used the following lines
next_page = browser.find_element_by_xpath("/descendant::a[@title='Siguiente'][1]")
webdriver.ActionChains(browser).move_to_element(next_page).click()
in the following context (url is actual parsed page)
from selenium import webdriver
PHANTOMJS_PATH = './phantomjs/phantomjs'
FARMACIA_URL = 'http://www.chedraui.com.mx/index.php/ajusco/endeca/category/view/id/457/'
browser = webdriver.PhantomJS(PHANTOMJS_PATH)
browser.get(FARMACIA_URL)
I have not gotten an error from the xpath not existing or any other error, the page simply does not refresh inside the program when clicked.
I have tried using browser.implicitly_wait(4) before and after the action and substituting the click() method with send_keys("\n"). There seem to be some more elaborated solutions involving javascript events but I have yet to try them and I'm a bit tired.
Any clues as to why this behavior is happening and how to fix it are much appreciated.
Thank you.
