Using Selenium in Python to click a specific button and start for loop

Question

I am using Selenium in Python 3.6 to simulate the search function in this page. (http://www.bobaedream.co.kr/cyber/CyberCar.php?gubun=I) What I trying to do is as below:

click "상세검색열기" button click
click "판매중인차량" checkbox check
start the for loop that simulates "search function" including the manipulation of drop-down menu

When I tried 1~2 (the code between "soup" and "makers") and 3(for loop) on separate code, it worked well. But the code combining 1~3 doesn't work.

Please help me to work this out.

My code is as below:

#!/usr/bin/env python
#-*- coding: utf-8 -*-

import re

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import StaleElementReferenceException
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select

from bs4 import BeautifulSoup
from time import sleep

link = 'http://www.bobaedream.co.kr/cyber/CyberCar.php?gubun=I'
driver = webdriver.PhantomJS()
driver.set_window_size(1920, 1080)
driver.get(link)
sleep(.75)

soup = BeautifulSoup(driver.page_source, "html.parser", from_encoding='utf-8')

driver.find_element_by_xpath('//img[@title="상세검색열기"]').click()
print("상세검색열기 버튼 클릭")
driver.find_element_by_xpath('//input[@title="판매중인 차량"]').click()
print ("판매중인 차량 클릭")

makers = ['아우디', 'BMW', '벤츠'] #아우디 = audi, 벤츠 = benz

for maker in makers:
    # open manufacturer layer
    next_elem = driver.find_element_by_xpath('//a[@title="제조사 선택"]')
    next_elem.click()

    # select manufacturer
    next_elem = driver.find_element_by_link_text(maker)
    next_elem.click()
    print(maker)
    print("====clicked maker")
    sleep(.75)

The Error message is like below:

/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 /Users/chongwonshin/PycharmProjects/Crawler_test/temp_temp2.py
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/bs4/__init__.py:146: UserWarning: You provided Unicode markup but also provided a value for from_encoding. Your from_encoding will be ignored.
  warnings.warn("You provided Unicode markup but also provided a value for from_encoding. Your from_encoding will be ignored.")
상세검색열기 버튼 클릭
판매중인 차량 클릭
Traceback (most recent call last):
  File "/Users/chongwonshin/PycharmProjects/Crawler_test/temp_temp2.py", line 33, in <module>
    next_elem.click()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/selenium/webdriver/remote/webelement.py", line 77, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/selenium/webdriver/remote/webelement.py", line 494, in _execute
    return self._parent.execute(command, params)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
    self.error_handler.check_response(response)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: {"errorMessage":"Element is not currently visible and may not be manipulated","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:50709","User-Agent":"Python-urllib/3.6"},"httpVersion":"1.1","method":"POST","post":"{\"id\": \":wdc:1484839459174\", \"sessionId\": \"55be8bc0-de5b-11e6-980a-4135a6020523\"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/55be8bc0-de5b-11e6-980a-4135a6020523/element/:wdc:1484839459174/click"}}
Screenshot: available via screen


Process finished with exit code 1

Piotr Dawidiuk · Accepted Answer · 2017-01-19 18:58:33Z

1

Focus on your error - ElementNotVisibleException. That means you can't click on invisible element found with driver.find_element_by_xpath('//a[@title="제조사 선택"]').

After inspection

please notice that you can open manufacturer layer simply copying and executing javascript responsible for this - layerShow('layer_maker');$('#layer_maker .order a:first-child').focus().

Replace your loop's code with this

for maker in makers:
    # open manufacturer layer
    driver.execute_script("layerShow('layer_maker');$('#layer_maker .order a:first-child').focus()")

    # select manufacturer
    next_elem = driver.find_element_by_link_text(maker)
    next_elem.click()
    print(maker)
    print("====clicked maker")
    sleep(.75)

and everything works.

answered Jan 19, 2017 at 18:58

Piotr Dawidiuk

3,1021 gold badge27 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

신종원 Over a year ago

I appreciate your help. But I still have one more question. When I put the code from (driver.find_element ... ) to (print ("판매중인 차량 클릭")) into the for loop, the same error message is popped up again. Would you mind to helping me to work this out?

신종원 Over a year ago

Oh..... I just figured out. Never mind... Thanks anyways. I appreciate your help!!

Collectives™ on Stack Overflow

Using Selenium in Python to click a specific button and start for loop

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related