Web Scraping error in while using Search.submit()

Question

I am trying to do web scraping to weather data from: https://www.wunderground.com/history/daily/us/dc/washington/KDCA

Here is HTML:

<div _ngcontent-app-root-c136="" id="wuForm" class="ui-front wu-form">
   <div _ngcontent-app-root-c136="" id="wuSearch-contain" class="wu-search-contain ng-star-inserted"><label _ngcontent-app-root-c136="" for="wuSearch" class="visuallyHidden">Search</label><input _ngcontent-app-root-c136="" type="search" name="query" value="" id="wuSearch" placeholder="Search Locations" aria-label="Search" autocomplete="off" class="wu-search ng-untouched ng-pristine ng-valid"><span _ngcontent-app-root-c136="" class="close-search"><i _ngcontent-app-root-c136="" class="material-icons">close</i></span><span _ngcontent-app-root-c136="" class="geolocate-wrap"><i _ngcontent-app-root-c136="" class="material-icons">gps_fixed</i></span></div>
   <!----><!---->
   <search-autocomplete _ngcontent-app-root-c136="" _nghost-app-root-c135="">
      <ul _ngcontent-app-root-c135="" tabindex="0" class="ui-autocomplete ui-front ui-menu ui-widget ui-widget-content ui-corner-all hide">
         <li _ngcontent-app-root-c135="" class="ui-autocomplete-geolocate ng-star-inserted">
            <div _ngcontent-app-root-c135="" class="mimic-a menu-geolocate"><i _ngcontent-app-root-c135="" class="material-icons">gps_fixed</i>Find Nearest Station </div>
         </li>
         <!----><!----><!----><!----><!----><!---->
         <li _ngcontent-app-root-c135="" class="ui-autocomplete-last ui-menu-item manage-favorites"><a _ngcontent-app-root-c135="" tabindex="-1" href="/member/favorites" class="ui-corner-all">Manage Favorite Cities</a></li>
      </ul>
   </search-autocomplete>
</div>

I started with

driver = webdriver.Chrome(r'C:\Users\bno00\Downloads\chromedriver_win32\chromedriver.exe')
driver.get('https://www.wunderground.com/history/daily/us/dc/washington/KDCA')

and it's working fine then when I trying to send zipcode to search field by using .submit():

zipcode='22202'
xpath='//*[@id="wuSearch"]'
search=driver.find_element_by_xpath(xpath)
search.clear() 
search.send_keys(zipcode)
search.submit() #error occurs here

The error is:

NoSuchElementException: Message: no such element: Unable to locate element:{"method":"xpath","selector":"./ancestor-or-self::form"} (Session info: chrome=89.0.4389.114)

Tim Roberts · Accepted Answer · 2021-04-08 21:23:27Z

1

That <input> tag is not part of a form. You can't submit it. You either need to send a newline, or click the gps_fixed icon that triggers the search.

answered Apr 8, 2021 at 21:23

Tim Roberts

55.3k4 gold badges28 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

vitaliis · Accepted Answer · 2021-04-08 21:26:37Z

0

Try using the following:

input_field = driver.find_element_by_css_selector("input[id=wuSearch]")
input_field.click()
input_field.send_keys("some zip code")

Or just:

input_field = driver.find_element_by_id("wuSearch")
input_field.send_keys("some zip code")

Try to avoid click(). I am not sure it is really required. After you input a zip code, you should wait for your search result to appear.

answered Apr 8, 2021 at 21:26

vitaliis

4,1975 gold badges26 silver badges48 bronze badges

Comments

Arundeep Chohan · Accepted Answer · 2021-04-08 21:35:34Z

0

wait = WebDriverWait(driver, 10)
driver.get('https://www.wunderground.com/history/daily/us/dc/washington/KDCA')
zipcode='22202'
xpath='//*[@id="wuSearch"]'    
search=wait.until(EC.element_to_be_clickable((By.XPATH,xpath)))
search.clear() 
search.send_keys(zipcode)
time.sleep(1)
search.send_keys(Keys.ENTER)

Just delay it a bit. Use a webdriver wait to make sure you send the zipcode to the input and then send Keys.Enter.

Import

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from time import sleep

answered Apr 8, 2021 at 21:35

Arundeep Chohan

9,9895 gold badges17 silver badges36 bronze badges

Collectives™ on Stack Overflow

Web Scraping error in while using Search.submit()

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related