I want to build press clipping script with python selenium webdriver. My idea is to enter a list of keywords to serach field on website and to save results to external HTML file.
With this code i can enter ONE keyword and save serach results to external HTMl file and this is working fine
options = Options()
options.headless = True
driver=webdriver.Firefox(options=options)
print ("Headless Firefox Initialized")
driver.get("https://www.URL.com/")
#check page title
assert "URL TITLE - " in driver.title
time.sleep(2)
driver.find_element_by_class_name("search__open").click()
pretraga_input = driver.find_element_by_id("search-field-head")
pretraga_input.clear()
pretraga_input.send_keys("MY KEYWORD")
time.sleep(1)
pretraga_input.send_keys(Keys.RETURN)
timeout = 60
try:
element_present = EC.presence_of_element_located((By.CLASS_NAME, 'search__results'))
WebDriverWait(driver, timeout).until(element_present)
finally:
print("RESULTS PAGE LOADED")
html_blic=driver.page_source
soup_blic=BeautifulSoup(html_blic,'html.parser')
div_blic=soup_blic.select_one(".search__results")
html_content_blic = str(div_blic)
html_file = open('C://Users//MYUSER//Documents//PY//press.html', 'w', encoding="utf-8")
html_file.write('<head><title> PAGE TITLE </title><link rel="stylesheet" href="style.css"></head>'+'<div id="MYID"><div id="1">MY WEBSITE'+html_content_blic+'</div><div id="2"></div></div>' )
html_file.close()
driver.close()
But how can i enter a list of 10+ keywords and save every keyword search result to HTML file?