Python Selenium encoding issues on get_attribute function

Question

I am using Selenium in Python to crawl the drop-down menu of this page. I used find_elements_by_css_selector function and get all the data from the second drop-down menu.

But when I tried to print out both the group number and its matched model name through model.get.attribute('href'), the model name written in Korean was broken like below: 5%EC%8B%9C%EB%A6%AC%EC%A6%88 (which is 5-series in BMW).

I know using 'model.text' returns the only model name, but I also need the group name of each model at the same time.

This is why I used model.get.attribute('href') instead of model.text.

Please help me to work this out.

Below is my code.

enter code here#!/usr/bin/env python
#-*- coding: utf-8 -*-

import re

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import StaleElementReferenceException
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select

from bs4 import BeautifulSoup
from time import sleep

link = 'http://www.bobaedream.co.kr/cyber/CyberCar.php?gubun=I'
driver = webdriver.PhantomJS()
driver.set_window_size(1920, 1080)
driver.get(link)
sleep(.75)

s = BeautifulSoup(driver.page_source, "html.parser", from_encoding='utf-8')

makers = ['아우디', 'BMW', '벤츠']

for maker in makers:
    # open manufacturer layer
    next_elem = driver.find_element_by_xpath('//a[@title="제조사 선택"]')
    next_elem.click()

    next_elem = driver.find_element_by_link_text(maker)
    print(maker)
    next_elem.click()
    print("clicked maker")
    sleep(.75)

    # open model layer
    next_elem = driver.find_element_by_xpath('//a[@title="모델 선택"]')
    next_elem.click()

    # select model

    models = driver.find_elements_by_css_selector("#layer_group ul.list li a")
    for model in models:
        print (model.get_attribute('href'))

NarendraR · Accepted Answer · 2017-01-17 08:32:32Z

1

There is no href attribute for the said title. Try this for listing option value-( if any)

 elem = driver.find_element_by_xpath("//select[@title='모델 선택']").
 for option in elem.find_elements_by_tag_name('option'):
     print option.text

edited Jan 17, 2017 at 8:32

NarendraR

7,71811 gold badges49 silver badges89 bronze badges

answered Jan 17, 2017 at 7:49

user1069596

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python Selenium encoding issues on get_attribute function

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related