I am trying to scrape a table which is being generated through javascript but I am struggling. My code so far is:
driver = webdriver.Chrome();
driver.get("https://af.ktnlandscapes.com/")
# get table -- first wait for table to fully load
WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, "//*[@id='list-view']/tbody/tr")))
table = driver.find_element_by_xpath("//*[@id='list-view']")
# get rows
rows = table.find_elements_by_xpath("tbody/tr")
# iterate rows and get cells
for row in rows:
# get cells
print (row.get_attribute("listing"))
I want to scrape the "listing=" numbers within the table. I am not sure how to extract the listing numbers and I am struggling to understand how to force the page to open the rest of the rows within the table as they only load when you scroll down the table a bit.