Selenium / Python: Why isn't my Selenium find_element_by finding elements anymore after finding the first one in my for loop iterations?

Success! I found my own workaround!

FYI: I still could not see anything wrong with my existing code, as it correctly found all the a0 cells when I commented out the function part of that text (#getinfo(current_cell)... thank you @goalie1998 for the suggestion). And, I didn't change anything in that function for this new workaround, which works correctly. So, it must have something to do with Selenium getting messed up when trying to iterate through a loop that (1) tries to find_element_by something on the page (that exists multiple times on the page, and that's why you're creating the loop) and (2) clicks on a link within that loop, goes to a page, goes back to a page, and then is supposed to keep running through the iterations with the find_element_by "function" (probably wrong term usage here) to get the next one that exists on the page. Not sure why Selenium gets messed up from that, but it does. (More experienced coders, feel free to elaborate).

Anyway, my workaround thought process, which may help some of you solve this issue for yourselves by doing something similarly, is:

(1) Find all of the links BEFORE clicking on any of them (and create a list of those links)

Instead of trying to find & click the links one-at-a-time as they show up, I decided to find all of them FIRST (BEFORE clicking on them). I changed the above code to this:

# this is where I'm storing all the links
text_link_list = []

for row in table_rows:

    print("got the row!")
        
    waitfortable = WebDriverWait(driver, 5).until(
        EC.presence_of_element_located((By.XPATH, "/html/body/table[3]/tbody/tr/td[2]/form/table/tbody/tr/td/table/tbody/tr[4]/td/table/tbody/tr[1]/td[1]"))
         ) 
        
    ## Get a0
    try:
        print("we're trying!")
        row.find_element_by_class_name("a0")
        print("we got an a0!")
        
        # this next part is just because there are also blank "a0" cells without 
        # text (aka a link) in them, and I don't care about those ones.

        current_row_has_a0 = row.find_element_by_class_name("a0")
        if str(current_row_has_a0.text) != "":
            text_link_list += [current_row_has_a0.text]
            print("text added to text_link_list!")
        else:
            print("wasn't a text cell!")
     except:
         pass

     continue

(2) Iterate through that list of links, running your Selenium code that includes .click() and .back()

Now that I had my list of links, I could just iterate through that and do my .click() —> perform actions —> .back() function that I created ( getinfo() -- original code in question above).

## brand new for loop, after "for row in table_rows" loop

for text in text_link_list:
    # waiting for page to load upon iterations
    table = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.XPATH, "/html/body/table[3]/tbody/tr/td[2]/form/table/tbody/tr/td/table/tbody/tr[4]/td/table/tbody/tr[1]/td[1]"))
        ) 
    
    # this is my .click() --> perform actions --> .back() function
    getinfo(text)

However, I just needed to make two small changes to my .getinfo() function.

One, I was now clicking on the links via their "link text", not the a0 class I was using before (need to use .find_element_by_link_text).

Two, I could now use my more basic driver.find_element_by instead of my original table.find_element_by ...."table" may have worked as well, but I was worried about the memory of getting to "table" being lost since I was now in my function running the .click() code. I decided to go with "driver" since it was more certain. (I'm still pretty new to coding, this may not have been necessary.)

def getinfo(text):
    link_in_current_cell = driver.find_element_by_link_text(text)
    link_in_current_cell.click()

    waitfortable2 = WebDriverWait(driver, 5).until(
        EC.presence_of_element_located((By.CLASS_NAME, "top-edit-table"))
        ) 
    
    print("Here is the info from this temporary page.")

    driver.back()

    return True

I hope that this can all be helpful to someone. I was stoked when I did it and it worked! Let me know if it helped you! <3

PS. NOTE IF HAVING STALE ERRORS / StaleElementReferenceException: If you are iterating through your loops and clicking a link via something like driver.find_element_by (instead of using .back()), you may run into a Stale error, especially if you're just trying to use .click() on a variable that you assigned earlier or outside of the loop. You can fix this (maybe not in the most beautiful way) by redefining your variable right at that point of the loop when you're wanting to click on the link, with code like this:

my_link = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.LINK_TEXT, "linktext"))
    )
my_link.click()
continue

• "continue" is only necessary if you're wanting to end the current iteration and begin the next one, and is also not likely necessary if this is at the end of your loop
• you can change that red "10" number to whatever amount of time you'd like to give the code to find the element (aka the page to reload, most likely) before the script fails