Quantcast
Channel: Active questions tagged selenium - Stack Overflow
Viewing all 98926 articles
Browse latest View live

Selenium - Search multiple keywords in Google after each other v2

$
0
0

Selenium - Search multiple keywords in Google after each other

Java - Selenium - Search multiple keywords in Google after each other.

I actually want to perform a search on Google with Selenium (or a search field on a website), whereas the input is a list of keywords say:

I want to search on Google:

  1. Keyword: a
  2. then Keyword: b
  3. then Keyword: n
  4. then Keyword: x

How do I actually automate this? I cannot find anything useful.. I saw something about dictionaries which you can set up, but not in practice.

So Search Google -> for a list of keywords (a, b, n, x)

Any help would be awesome!

what i did sofar:

import org.openqa.selenium.*;import org.openqa.selenium.chrome.ChromeDriver;import org.openqa.selenium.io.FileHandler;import java.io.File;import java.io.IOException;import java.util.List;public class GoogleSearch{    static WebDriver driver;    public static void main(String[] args) throws IOException, InterruptedException {        System.setProperty("webdriver.chrome.driver","C:\\Users\\xx\\Software\\selenium_browser_drivers\\Chrome86\\chromedriver.exe");        driver = new ChromeDriver();        driver.manage().window().maximize();        driver.get("https://google.com/");        Thread.sleep(10000);        System.out.println(driver.getTitle());        driver.findElement(By.xpath("//input[@name='q']")).click();        Thread.sleep(2000);        driver.findElement(By.xpath("//input[@name='q']")).sendKeys("My Search Text");        Thread.sleep(2000);        driver.findElement(By.xpath("//input[@name='q']")).sendKeys(Keys.DOWN);        Thread.sleep(2000);        driver.findElement(By.xpath("//input[@name='q']")).sendKeys(Keys.ENTER);        Thread.sleep(2000);        List<WebElement> links =   driver.findElements(By.xpath("//*[@id='res']//a/h3"));        int count = links.size();        System.out.println(count);        for (int i =0;i<count;i++)        {            String linkname = driver.findElements(By.xpath("//*[@id='res']//a/h3")).get(i).getText();            System.out.println(linkname);        }        getScreenshot();    }    public static void getScreenshot() throws IOException    {        File src= ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);        FileHandler.copy(src,new File("C:\\Users\\xx\\Desktop\\google.png"));    }}

driver.get can't locate the chromdriver

$
0
0

my python code.

DRIVER_PATH = 'D:/chromedriver.exe'driver = webdriver.Chrome(options=options , executable_path=DRIVER_PATH)

please note that my python file and chrome driver are located in the same folder. but still my code generates an error as follows:

Traceback (most recent call last):  File "E:\anaconda\envs\AHG_web\lib\site-packages\selenium\webdriver\common\service.py", line 72, in start    self.process = subprocess.Popen(cmd, env=self.env,  File "E:\anaconda\envs\AHG_web\lib\subprocess.py", line 854, in __init__    self._execute_child(args, executable, preexec_fn, close_fds,  File "E:\anaconda\envs\AHG_web\lib\subprocess.py", line 1307, in _execute_child    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,FileNotFoundError: [WinError 2] The system cannot find the file specifiedDuring handling of the above exception, another exception occurred:Traceback (most recent call last):  File "scrapper_new_EIB.py", line 25, in <module>    driver = webdriver.Chrome(options=options , executable_path=DRIVER_PATH)  File "E:\anaconda\envs\AHG_web\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 73, in __init__    self.service.start()  File "E:\anaconda\envs\AHG_web\lib\site-packages\selenium\webdriver\common\service.py", line 81, in start    raise WebDriverException(selenium.common.exceptions.WebDriverException: Message: 'chromedriver.exe' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

despite the above code, I also added chromedriver.exe to the environment variable as well. and again I get the same error. how can I solve it?

Selenium making a list with the get_attributes method

$
0
0

I'm trying to get random wikipedia articles with the wikipedia API and then get all their href links which I did by:

elems = driver.find_elements_by_xpath("//a[@href]")

Then I made:

for elem in elems:    elements = []    elements.append(elem.get_attribute("href"))

In order to only get the href links from the received list with the previously executed command. I only got a single link which is:

['https://www.mediawiki.org/']

I tried to do:

for elem in elems:    elements = []    elements.extend(elem.get_attribute("href"))

Which resulted in this output:

['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'm', 'e', 'd', 'i', 'a', 'w', 'i', 'k', 'i', '.', 'o', 'r', 'g', '/']

Clearly put: I want to take all href links from a random wikipedia article then put them in a single list called elements which I will later use for a for loop.

Thanks in advance.

Rest of my code if it is needed:

import wikipediaimport seleniumfrom selenium import webdriver#This will get a random wikipedia articlerandom_article = wikipedia.random(1)#Get the articles URLurl = wikipedia.page(random_article).urldriver = webdriver.Firefox()driver.get(url)elems = driver.find_elements_by_xpath("//a[@href]")for elem in elems:    elements = []    elements.extend(elem.get_attribute("href"))print(elements)

How can i get text written inside html/body present inside div in selenium web-driver in java

$
0
0

In this code i want to enter the text for id cke_1_contents and input field is html/body. i tried with xpath and id but it is not working, How can i get written the text inside the text-area

[<div id="cke_1_contents" class="cke_contents cke_reset" role="presentation" style="height: 200px;"><span id="cke_86" class="cke_voice_label">Press ALT 0 for help</span><iframe src="" frameborder="0" class="cke_wysiwyg_frame cke_reset" style="width: 1010px; height: 100%;" title="Rich Text Editor, mailContent" aria-describedby="cke_86" tabindex="0" allowtransparency="true"><html dir="ltr" lang="en"><head><title data-cke-title="Rich Text Editor, mailContent">Rich Text Editor, mailContent</title><style data-cke-temp="1">html{cursor:text;*cursor:auto}img,input,textarea{cursor:default}</style></head><body contenteditable="true" class="cke_editable cke_editable_themed cke_contents_ltr cke_show_borders" spellcheck="false"><p>Hi, this is test mail</p></body></html></iframe></div>

Selenium Webdriver finding an element in a sub-element

$
0
0

I am trying to search for an element in a sub-element with Selenium (Version 2.28.0), but selenium des not seem to limit its search to the sub-element. Am I doing this wrong or is there a way to use element.find to search a sub-element?

For an example I created a simple test webpage with this code:

<!DOCTYPE html><html><body><div class=div title=div1><h1>My First Heading</h1><p class='test'>My first paragraph.</p></div><div class=div title=div2><h1>My Second Heading</h1><p class='test'>My second paragraph.</p></div><div class=div title=div3><h1>My Third Heading</h1><p class='test'>My third paragraph.</p></div></body></html>

My python (Version 2.6) code looks like this:

from selenium import webdriverdriver = webdriver.Firefox()# Open the test page with this instance of Firefox# element2 gets the second division as a web elementelement2 = driver.find_element_by_xpath("//div[@title='div2']")# Search second division for a paragraph with a class of 'test' and print the contentprint element2.find_element_by_xpath("//p[@class='test']").text # expected output: "My second paragraph."# actual output: "My first paragraph."

If I run:

print element2.get_attribute('innerHTML')

It returns the html from the second division. So selenium is not limiting its search to element2.

I would like to be able to find a sub-element of element2. This post suggests my code should work Selenium WebDriver access a sub element but his problem was caused by a time-out issue.

Can anyone help me understand what is happening here?

How to resolve 'Process unexpectedly closed with status 11' for Selenium in IPython?

$
0
0

I run the following code in IPython (from official Python extension VSCode):

# %%from selenium import webdriverdriver = webdriver.Firefox(    executable_path='.driver/geckodriver')

It gives me the following errors:

  • For GeckoDriver 0.28.0
WebDriverException: Message: Process unexpectedly closed with status 11
  • For GeckkoDriver 0.27.0:
WebDriverException: Message: invalid argument: can't kill an exited process

My setup:

Firefox: 82.0GeckoDriver: 0.28.0Selenium: 3.141.0IPython: 7.19.0Ubuntu: 20.10Python3 venv

Note: This script works in terminal but not IPython

Thank you.

python selenium - Unable to locate element in justdial site

$
0
0

I already tried

  1. Implicit and explicit wait
  2. wait.until
  3. Time module

Here I am trying to locate "Load more reviews" link on justdial. Here is the link. Kindly check and help.

from selenium import webdriverfrom webdriver_manager.chrome import ChromeDriverManagerbrowser = webdriver.Chrome(ChromeDriverManager().install())browser.get("https://www.justdial.com/Delhi/S-K-Premium-Par-Hari-Nagar/011PXX11-XX11-131128122154- B8G6_BZDET")elem = browser.find_element_by_xpath('//*[@id="lmrehd"]')elem.click()

Kindly give me some solution with explanation.

python add https:// to url = sys.argv[1] [duplicate]

$
0
0

I'm new in Python, how can I add https:// into "url = sys.argv[1]"?

Below is my script, what can I improve from here?

import sysfrom selenium import webdriverfrom selenium.webdriver.chrome.options import Optionsurl = sys.argv[1]username = sys.argv[2]password = sys.argv[3]options = Options()options.add_argument('--allow-running-insecure-content')options.add_argument('--ignore-certificate-errors')options.add_argument('--start-maximized')driver = webdriver.Chrome(options=options)driver.get(url)driver.implicitly_wait(5)driver.find_element_by_id("name").send_keys(username)driver.find_element_by_id("passwd").send_keys(password)driver.find_element_by_id("login").click()

Selenium NameError Python

$
0
0

I was trying to automate google maps. I am getting an error.

vals = [name, category, address, plus_code, phone, website, avg_rating, total_rating, opening_hrs]

NameError: name 'address' is not defined

###############################

The address variable is defined, not sure what's causing the error

###############################

Please help to identify the error. Your help is highly appreciated.

###############################

from selenium import webdriverimport csvfrom selenium.webdriver.support.ui import Selectfrom selenium import webdriverfrom selenium.webdriver.common.action_chains import ActionChainsfrom selenium.webdriver.common.by import Byfrom selenium.webdriver.support.ui import WebDriverWaitfrom selenium.webdriver.support import expected_conditions as ECfrom selenium.webdriver.common.keys import Keysimport timedriver.get('https://maps.google.com')for each in queries:  input_field = driver.find_element_by_xpath('//input[@aria-label="Search Google Maps"]')  input_field.clear()  input_field.send_keys(each)  input_field.send_keys(Keys.RETURN)  while True:    WebDriverWait(driver, 25).until(EC.presence_of_element_located((By.XPATH, '//div[@class="section-result-content"]')))    for i in range(1, 20):      results = driver.find_elements_by_xpath(f'(//div[@class="section-result-content"])[{i}]')      for result in results:        outFile =  open("data.csv",'a+',newline="")        writer = csv.writer(outFile)        result.click()        time.sleep(3)        try:          name = driver.find_element_by_xpath('//h1[@class="section-hero-header-title-title GLOBAL__gm2-headline-5"]/span').text        except:          pass        try:          category = driver.find_element_by_xpath('//button[@jsaction="pane.rating.category"]').text        except:          pass        try:          address = driver.find_element_by_xpath('//button[@data-item-id="address"]').get_attribute('aria-label')        except:          pass        try:          plus_code = driver.find_element_by_xpath('//button[@data-tooltip="Copy plus code"]').get_attribute('aria-label')        except:          pass        try:          phone = driver.find_element_by_xpath('//button[@data-tooltip="Copy phone number"]').get_attribute('aria-label')        except:          pass        try:          website = driver.find_element_by_xpath('//button[@data-tooltip="Open website"]').get_attribute('aria-label')        except:          pass        try:          avg_rating = driver.find_element_by_xpath('//span[@class="section-star-display"]').text        except:          pass        try:          total_rating = driver.find_element_by_xpath('//span[@class="section-rating-term-list"]//button[@jsaction="pane.rating.moreReviews"]').text        except:          pass        try:          opening_hrs = driver.find_element_by_xpath('(//span[contains(@class, "section-info-hour-text")]/span)[2]').text        except:          pass        vals = [name, category, address, plus_code, phone, website, avg_rating, total_rating, opening_hrs]        writer.writerow(vals)        outFile.close()        WebDriverWait(driver, 25).until(EC.presence_of_element_located((By.XPATH, '//span[contains(text(), "Back to results")]')))        back = driver.find_element_by_xpath('//span[contains(text(), "Back to results")]')        back.click()        time.sleep(3)

Chromedriver on a headless GCP Ubuntu crashes

$
0
0
$ cat /etc/os-release | grep VERSION=VERSION="20.04.1 LTS (Focal Fossa)"$ chromedriver --versionChromeDriver 86.0.4240.22 (398b0743353ff36fb1b82468f63a3a93b4e2e89e-refs/branch-heads/4240@{#378})$ chromium --versionChromium 86.0.4240.183 snap

This is the code I use to run

from selenium import webdriverchrome_options = webdriver.ChromeOptions()chrome_options.headless = Truechrome_options.add_argument("--remote-debugging-port=9222")driver = webdriver.Chrome(options=chrome_options)driver.get("http://www.google.com")print(driver.title)print("works")

This is the error message I always get

  File "minimal_scraper_test.py", line 17, in <module>    driver = webdriver.Chrome(options=chrome_options)  File "/home/ubuntu/venv/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in __init__    RemoteWebDriver.__init__(  File "/home/ubuntu/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__    self.start_session(capabilities, browser_profile)  File "/home/ubuntu/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session    response = self.execute(Command.NEW_SESSION, parameters)  File "/home/ubuntu/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute    self.error_handler.check_response(response)  File "/home/ubuntu/venv/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response    raise exception_class(message, screen, stacktrace)selenium.common.exceptions.SessionNotCreatedException: Message: session not createdfrom disconnected: unable to connect to renderer  (Session info: headless chrome=86.0.4240.183)

I have looked at questions with very similar error message. Most of them are due to version mismatch between chromedriver and chromium. However, both of my versions match. Thank you very much for the help.

Chromedriver on a headless GCP Ubuntu crashes with "unable to connect to renderer"

$
0
0
$ cat /etc/os-release | grep VERSION=VERSION="20.04.1 LTS (Focal Fossa)"$ chromedriver --versionChromeDriver 86.0.4240.22 (398b0743353ff36fb1b82468f63a3a93b4e2e89e-refs/branch-heads/4240@{#378})$ chromium --versionChromium 86.0.4240.183 snap

This is the code I use to run

from selenium import webdriverchrome_options = webdriver.ChromeOptions()chrome_options.headless = Truechrome_options.add_argument("--remote-debugging-port=9222")driver = webdriver.Chrome(options=chrome_options)driver.get("http://www.google.com")print(driver.title)print("works")

This is the error message I always get

  File "minimal_scraper_test.py", line 17, in <module>    driver = webdriver.Chrome(options=chrome_options)  File "/home/ubuntu/venv/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in __init__    RemoteWebDriver.__init__(  File "/home/ubuntu/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__    self.start_session(capabilities, browser_profile)  File "/home/ubuntu/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session    response = self.execute(Command.NEW_SESSION, parameters)  File "/home/ubuntu/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute    self.error_handler.check_response(response)  File "/home/ubuntu/venv/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response    raise exception_class(message, screen, stacktrace)selenium.common.exceptions.SessionNotCreatedException: Message: session not createdfrom disconnected: unable to connect to renderer  (Session info: headless chrome=86.0.4240.183)

I have looked at questions with very similar error message. Most of them are due to version mismatch between chromedriver and chromium. However, both of my versions match. Thank you very much for the help.

How to get rid of cookies notification in chrome browser using Selenium Java

$
0
0

I would like to ask you to help me solve this problem. I want to open chrome window with Selenium, using Java language. Next I would like to find element of input and put some text there. Problem is, that I cannot get rid of cookies notification, because window always opens with me unlogged from google account. How can I open chrome window already logged into my account? Or how can I get rid of google cookies notification?

Thank you very much.

Mock API response on a Flask route

$
0
0

Background

I would like to write a unittest for my Flask application which utilize an external API on some routes.

When running my tests I don't want to make any calls to the external API, rather, I want to pass some static dummy data to the route.

This is how I currently have everything set up.I can't figure out how to refactor my code to achieve what I want.

get_name.py:

from flask import render_template, Blueprint, request, redirect, url_for, sessionname_blueprint = Blueprint('name_blueprint', __name__)@name_blueprint.route("/name", methods=['POST', 'GET'])def name():    name = requests.get('https://name.com/getname') <<< Let's say this api call returns 'Bob'    return render_template('name.html', name=name)

init.py

from flask import Flaskfrom app.name.get_name import name_blueprintdef create_app():    app = Flask(__name__)    app.config['SECRET_KEY'] = 'some-secret-key'    app.register_blueprint(name_blueprint)    return app

name.html

<!DOCTYPE html><html><body><h1>{{ name }}</h1></body></html>

I want to be able to mock the API call and instead of receiving name from the server I want to pass some dummy data to it.

import unittestfrom unittest.mock import patchfrom flask_testing import LiveServerTestCasefrom selenium import webdriverclass TestName(LiveServerTestCase):    def create_app(self):        app = create_app()        app.config['TESTING'] = True        app.config.update(LIVESERVER_PORT=9898)        return app    def setUp(self):        self.driver = webdriver.Chrome()    def test_name_route(self):        self.driver.get(f'{self.driver.get(self.get_server_url())}/name')        with patch('get_name.requests.get') as mocked_get:            mocked_get.return_value.text = 'Michael'        res_name = self.driver.find_element_by_tag_name('h1')        self.assert res_name == 'Michael'if __name__ == '__main__':    unittest.main()

How to use same instance of selenium web driver across different python files?

$
0
0

I needed to use same instance of selenium web driver across different python files. This is my base class.

class Automate(object):    def __init__(self):        self.options = webdriver.ChromeOptions()        self.options.add_argument('--user-data-dir=\\profile path')        self.driver = webdriver.Chrome(executable_path="\\driver path", options=self.options)    def get(self, url):        self.driver.get(url)

Now, when I try to instantiate an object for this class in two different files like temp.py and test.py,

In temp.py

import automate    driver1 = Automate()driver1.get("google.com")

In test.py

import automatedriver2 = Automate()driver2.get("google.com")

The result is opening of two seperate chrome windows.I want these two files to use only one instance of the web driver. I tried searching for answers, someone said to use singleton classes (I don't know much about singleton class).

Can someone please give me an example on how to use the single driver instance across multiple python files.

What I want:I need 1st file to open the browser and 2nd file to send commands like get, find elements by xpath. Also, I want the 2nd file to be re runnable without opening a new browser window.

python selenium code to save text in a variable from clipboard which is copied to the clipboard by click of an element

$
0
0

On my webpage I have a window as shown in the image below. It has a division which shows the password. When I click on the "copy" element, it copies the password onto the clipboard. However this window closes within 30 seconds and then the copied password on the clipboard will be lost.

image

My intention is to save this password, copied on clipboard, into a variable to be used in later part of my python selenium code.

I'm able to successfully click this "copy" element via the automated selenium script which then copies the password to clipboard. This is my code:

''' I will change time.sleep() later to appropriate WebdriverWait. This is just for test. '''''' This also means that I will have to save the copied password from clipboard within 10seconds, '''''' before the window closes '''    time.sleep(20)    copy_button = self.chrome_driver.find_element_by_xpath('//div[@class="button__container__not-show-spinner"]/div[@class="button__text"]')    copy_button.click()

I read an article closest to my query here, which talks about importing tkinter module: Accessing text copied to clipboard by python

Is there a way I can do it using Selenium Webdriver methods in Python?

Please help.


How to keep logged in asp.net while parsing other links in the homepage

$
0
0

I want to scrape vulms website and download announcement of all courses My login attempt is successful but when i go to another url of the same site using requests then my login session is timed out

soup = BeautifulSoup(driver.page_source, 'html.parser')datalist = []for link in soup.find_all('a'):    a = link.get('href')    datalist.append(a)    if a == 'Announcements/AnnouncementsList.aspx':        req = requests.get('https://vulms.vu.edu.pk/'+a)        print(driver.current_url)        soup2 = BeautifulSoup(req.content, 'html.parser')        print(soup2.text)

Here is the output after going to announcement.aspxSession Expired
Either you issued a sign out command or your session has timed-out.
Any data that you had already submitted will be preserved.Click Here to Sign In again.November 07, 2020 05:55 PM

I need to automate scenario with specific dates in web app [closed]

$
0
0

This web app is connected to dynamics and every time i submit an application, the dates are changing to 3 years later.

IS there a way to clean the state of the user after the application is submitted to reuse this account again for my test?

I would imagine this would be done via an API call

Selenium accepting alert gdpr

$
0
0

I've looked everywhere tried everything, I just can't figure it out, can somebody help please?

driver.execute_script('''window.open("https://www.abv.bg/","_blank");''')driver.implicitly_wait(30)driver.switch_to.frame("abv-GDPR-frame")#driver.switch_to.frame("gdpr-consent-notice")otkazTS = driver.find_element_by_xpath('''.//a[@class="accept-all mat-button mat-button-base mat-raised-button"][@style="color: white    background-color: rgb(72, 72, 74)"]''').click()

This is the part that is giving me this error:selenium.common.exceptions.NoSuchFrameException: Message: abv-GDPR-frame

I've looked everywhere, at first I didn't even try to locate the frame, but I saw that I need to first locate the correct frame and than the button, to accept it. If someone can tell me what am I missing, it would be great.Thanks in advance!

Каквручнуюпройтикапчунастраницеоткрытойв selenium

$
0
0

Читалчтоестьспособоткрытьокночерез selenium ипройтикапчувручную. Открываютак:

options = Options()options.add_argument('--headless')options.add_argument('--no-sandbox')driver = webdriver.Chrome(executable_path = "/usr/bin/chromedriver", options = options)driver.maximize_window()driver.get(url_rec)driver.close()driver.quit()

Ссылкаоткрываетсятамвнутри. Акакувидетьреальноеокно, чтобыпройтикапчу.

How do I get the word?

$
0
0

Get word from (in this thing the "test" changes after answering something):

< div class="au-target" au-target-id="137">Test</div>

Code:

word = driver.find_element_by_class_name('au-target')

Error:

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".au-target"}  (Session info: chrome=86.0.4240.183)
Viewing all 98926 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>