1

I am trying to scrap a web site News list after searching a certain word. But I can't figure out how to access News list.

First of all, I entered a key work and click search button.

elem = driver.find_element_by_id("query") 
elem.send_keys("kpop")    
elem = driver.find_element_by_id("search_btn")
elem.click()

The result page is this: https://search.naver.com/search.naver?sm=tab_hty.top&where=nexearch&query=kpop&oquery=kpop&tqi=To2nllpySEKsssnbxrsssssssFd-407861

And I want to move on News tab placed on third tab News tab

Please help!

2 Answers 2

1

As per the url you have shared to click on the link to move on to the News tab placed on third tab you can use the following line of code :

driver.find_element_by_xpath("//ul[@class='base']/li[@class='lnb4']/a[@class='tab']/span[@class='spnew']").click()

You may require to induce WebDriverWait for the webelement to be clickable as follows :

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//ul[@class='base']/li[@class='lnb4']/a[@class='tab']/span[@class='spnew']"))).click()
Sign up to request clarification or add additional context in comments.

Comments

1

Right-click the desired element. Copy the xpath. Paste it in .find_element_by_xpath().

button = driver.find_element_by_xpath('//*[@id="nx_search_form"]/fieldset/button')

2 Comments

I'm new to Python..so can you explain how can i find 'nx_search_form'? I did copy the xpath, but I don't know where to find 'nx_search_form'
Not sure I follow you. You don't know where to find the element you want? If you are looking to scrape just the news info you could add your keyword to the href and skip the form all together. url = "https://search.naver.com/search.naver?where=news&sm=tab_jum&query="+your_search_term

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.