Multiple clicking loop on URL issue with Python Selenium

Question

how do I fix this code? As I'm trying to create a multiple clicking on a multiple URL loop but it just kept at the same link over and over.

if the url contains dr.macio and contains this div class ('_3ao649')

from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
import time
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
import csv

import time
url = 'https://shopee.com.my/search?keyword=mattress'

driver = webdriver.Chrome(executable_path=r'E:/users/Francabicon/Desktop/Bots/others/chromedriver.exe')
driver.get(url)
time.sleep(0.8)

# select language
driver.find_element_by_xpath('//div[@class="language-selection__list"]/button').click()
time.sleep(3)

# scroll few times to load all items 
for x in range(10):
    driver.execute_script("window.scrollBy(0,300)")
    time.sleep(0.1)

# get all links (without clicking)

all_items = driver.find_elements_by_xpath('//a[@data-sqe="link"]')
print('len:', len(all_items))

all_urls = []

for item in all_items:
    url = item.get_attribute('href')
    all_urls.append(url)
    print(url)

# now use links

for item in all_urls:
    a = item.splitlines("\n")
    if url.contains("dr.macio"):
       continue
    else:
       driver.get(chr(a))
       driver.back()

if url.contains("dr.macio"): you are never reassigning url in the loop it will always be the last value you assigned in the for item in all_items: loop. You probably want to inspect item or a — Iain Shelvington
– Iain Shelvington, Commented Dec 23, 2019 at 10:58

KunduK · Accepted Answer · 2019-12-23 11:38:58Z

0

If I understood your use case that you would like to visit each product url except which contains dr.macio.

Induce WebdriverWait and visibility_of_all_elements_located() and get all the links href value and then during iteration varify the links contains.

Try below code.

from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
import time
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
import csv

import time
url = 'https://shopee.com.my/search?keyword=mattress'

driver = webdriver.Chrome(executable_path=r'E:/users/Francabicon/Desktop/Bots/others/chromedriver.exe')
driver.get(url)

# select language
WebDriverWait(driver,5).until(EC.element_to_be_clickable((By.XPATH,'//div[@class="language-selection__list"]/button'))).click()

# scroll few times to load all items
for x in range(10):
    driver.execute_script("window.scrollBy(0,300)")
    time.sleep(0.1)

# get all links (without clicking)
all_items=[item.get_attribute('href') for item in WebDriverWait(driver,15).until(EC.visibility_of_all_elements_located((By.XPATH,'//a[@data-sqe="link"]')))]
print(all_items)

for item in all_items:
    #Checking here link contains `dr.macio`
    if "dr.macio" in item:
       continue
    else:
       driver.get(item)
       driver.back()

answered Dec 23, 2019 at 11:38

KunduK

33.4k5 gold badges19 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Francabicon franc Over a year ago

then how do i do this if the url contains dr.macio and contains this div class ('_3ao649') which is called ads

KunduK Over a year ago

@Francabiconfranc ` dr.macio` already checked with if "dr.macio" in item: this

Francabicon franc Over a year ago

cuz there is a thing called ads under DIV class '_3ao649'. so i wou,d like to check with that condition too.

Collectives™ on Stack Overflow

Multiple clicking loop on URL issue with Python Selenium

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related