I have a Python/Selenium script working through a page, and in an effort not to spam the server I have half second sleeps between interactions with page elements. I've noticed an issue where the script "lost track" of items in my list, and just recently discovered that it seems to be due to the page auto-updating/resetting during the script's run. The id's seem to be dynamically generated, so the refresh usually alters some of the id's, and the script crashes.
When I first encountered the issue, I just put a try-catch on it, skipping elements it had lost track of and logging an error, as I didn't have any real information on the issue and it was infrequent.
Giving it some more thought, and watching the script in action for a while, I noticed a slight refresh and the page re-ordering some elements.
The best idea I've come up with is below, where I reload the page entirely and refresh the list MID-LOOP and starting over. This is clearly a wrong way to address this. What is a better option?
try:
browser.execute_script("arguments[0].click();", button)
except link_lost:
print('Lost track of something...')
browser.refresh() # Page glitch fix. Refresh, relocate buttons, start over.
browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")
time.sleep(5) # could be optimized by keeping track of which question we're on and starting from there
target_buttons = browser.find_elements_by_css_selector('[aria-label=target]')