I'm trying to automate my spendings&budget, but the website of the bank reveals/hides the element I am looking for, depending on the scroll height.

Example

This is when I am at the top of the page:

And this is when I scrolled a little:

Problem

I am trying to get all elements in this role='transactions-group' but, when I added an option for scrolling:

driver.execute_script("window.scrollTo(0,document.body.scrollHeight)")transactions = driver.find_elements_by_xpath("//div[@role='transactions-group']")print(str(transactions[0].text))

it just loaded everything, but it continued to find the items that in this case were on the bottom of the page.

So my question is: Can I do some kind of loop or something to fix this problem?

We tried for a few days to setup Proxy Authentication with selenium chromedriver in Python.We couldn't set ip up because Chrome throws a pop-up for authentication. Problem is that selenium can't switch to that window and so, can't type. The only solution that worked for us was using pyautogui which is a bad solution for us because we want to use the headless function.

Here are all the methods we tried:

driver.switch_to_window()

driver.switch_to_active_element()

driver.switch_to_alert()

ActionChains(driver).send_keys("test").perform()

driver.switch_to_alert()

options = webdriver.ChromeOptions()options.add_argument('--proxy-server=http://user:pass@3.223.68.195:31112')

driver.get("https://user:pass@google.com")

proxy = {'address': '3.209.253.119:31112','username': 'user','password': 'pass'}capabilities = dict(DesiredCapabilities.CHROME)capabilities['proxy'] = {'proxyType': 'MANUAL','httpProxy': proxy['address'],'ftpProxy': proxy['address'],'sslProxy': proxy['address'],'noProxy': '','class': "org.openqa.selenium.Proxy",'autodetect': False}capabilities['proxy']['socksUsername'] = proxy['username']capabilities['proxy']['socksPassword'] = proxy['password']driver = webdriver.Chrome(executable_path="chromedriver.exe", desired_capabilities=capabilities)driver.get("http://google.com")

Any help would be really appreciated :)

Using clj-webdriver with seleniumhq and chromedriver(86) and when test starts I still get this issue.

Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.collect.ImmutableList.toImmutableList()Ljava/util/stream/Collector;at org.openqa.selenium.chrome.ChromeOptions.asMap(ChromeOptions.java:292)at org.openqa.selenium.MutableCapabilities.merge(MutableCapabilities.java:73)at org.openqa.selenium.remote.DesiredCapabilities.merge(DesiredCapabilities.java:107)at org.openqa.selenium.remote.DesiredCapabilities.merge(DesiredCapabilities.java:33)at org.openqa.selenium.MutableCapabilities.setCapability(MutableCapabilities.java:96)...

It says it cannot find method from com.google.guava but I can see that guava-25.0-jre.jar is fetched in project dependencies but it is somehow ignored.

Also weird thing is that from stacktrace ChromeOptions.asMap() is refering to line 292 in ChromeOptions.java but when I check that class there's no such line so it looks like it is refering to different version of selenium-chrome-driver but then again in deps there's 3.141.59 version.

Checked all the dependencies and cannot seem to find place where old guava or old selenium-chrome-driver should live.

Dependencies looks like:

:dependencies [[org.seleniumhq.selenium/selenium-java "3.141.59"][org.clojure/clojure "1.10.1"][clj-webdriver "0.7.2" :exclusions [     [org.seleniumhq.selenium/selenium-server]     [org.seleniumhq.selenium/selenium-java]     [org.seleniumhq.selenium/selenium-remote-driver]     [com.codeborne/phantomjsdriver]]][org.seleniumhq.selenium/htmlunit-driver "2.44.0"][com.google.guava/guava "25.0-jre"][clj-http "3.10.3"][cheshire "5.10.0"][clj-time "0.15.2"]]

I spent on this like whole day and running out of ideas.Any ideas why this error appears?

I am writing a code that asks the user for a type of music and redirects them to a playlist in YouTube which is related to what the user wrote. While trying to click on the "Filter" button on YouTube to access the diferent playlist I encountered the following error:selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable.

Is there any way of making that button interactable?

While searchinig for a way to solve it, some of the solutions recommended using an explicit wait, but the timer reaches it's limits:

raise TimeoutException(message, screen, stacktrace)selenium.common.exceptions.TimeoutException: Message: 

Here is my code:

from selenium import webdriverfrom selenium.webdriver.common.keys import Keysfrom selenium.webdriver.common.by import Byfrom selenium.webdriver.support.ui import WebDriverWaitfrom selenium.webdriver.support import expected_conditions as EClink= "https://www.youtube.com/"PATH= "C:\Program Files (x86)\chromedriver.exe"driver= webdriver.Chrome(PATH)driver.get(link)print("\n")print(driver.title)genre= input("What kind of music do you want?: ")print("Looking in YouTube's playlists ("+ genre+")...")search_bar= driver.find_element_by_name("search_query")search_bar.send_keys(genre+" playlist")print(driver.current_url)search_button= driver.find_element_by_id("search-icon-legacy")search_button.click()filter_pl = WebDriverWait(driver, 120).until( #Without the explicit wait this line would be: filter_pl= driver.find_element_by_id("button")EC.element_to_be_clickable((By.ID, "button")) #Without the explicit wait this line wouldn't existfilter_pl.click()

Chrome browser is not launching when I click the build now link in jenkins, but showing build is success.

Solution provided in Jenkins : Selenium GUI tests are not visible on Windows are not working for me. Someone please help me in this.

Started by user Karthick RaguRunning as SYSTEMBuilding in workspace C:\Program Files\Jenkins\workspace\killerFrameworkParsing POMsEstablished TCP socket on 51445[karthick.framework] $ "C:\Program Files\Java\jdk1.8.0_181/bin/java" -cp "C:\Program Files\Jenkins\plugins\maven-plugin\WEB-INF\lib\maven35-agent-1.13.jar;C:\Program Files\Jenkins\tools\hudson.tasks.Maven_MavenInstallation\TestMaven\boot\plexus-classworlds-2.6.0.jar;C:\Program Files\Jenkins\tools\hudson.tasks.Maven_MavenInstallation\TestMaven/conf/logging" jenkins.maven3.agent.Maven35Main "C:\Program Files\Jenkins\tools\hudson.tasks.Maven_MavenInstallation\TestMaven" "C:\Program Files\Jenkins\war\WEB-INF\lib\remoting-3.35.jar" "C:\Program Files\Jenkins\plugins\maven-plugin\WEB-INF\lib\maven35-interceptor-1.13.jar" "C:\Program Files\Jenkins\plugins\maven-plugin\WEB-INF\lib\maven3-interceptor-commons-1.13.jar" 51445<===[JENKINS REMOTING CAPACITY]===>channel startedExecuting Maven:  -B -f D:\LEARNING\sel\karthick.framework\pom.xml clean test[INFO] Scanning for projects...[INFO] [INFO] --------------< web.automation.killer:karthick.framework >--------------[INFO] Building karthick.framework 0.0.1-SNAPSHOT[INFO] --------------------------------[ jar ]---------------------------------[INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ karthick.framework ---[INFO] Deleting D:\LEARNING\sel\karthick.framework\target[INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ karthick.framework ---[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent![INFO] Copying 0 resource[INFO] [INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ karthick.framework ---[INFO] Nothing to compile - all classes are up to date[INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ karthick.framework ---[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent![INFO] Copying 0 resource[INFO] [INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ karthick.framework ---[INFO] Changes detected - recompiling the module![WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is platform dependent![INFO] Compiling 3 source files to D:\LEARNING\sel\karthick.framework\target\test-classes[INFO] [INFO] --- maven-surefire-plugin:3.0.0-M3:test (default-test) @ karthick.framework ---[JENKINS] Recording test results[INFO] ------------------------------------------------------------------------[INFO] BUILD SUCCESS[INFO] ------------------------------------------------------------------------[INFO] Total time:  15.339 s[INFO] Finished at: 2019-10-06T23:35:41+05:30[INFO] ------------------------------------------------------------------------Waiting for Jenkins to finish collecting data[JENKINS] Archiving D:\LEARNING\sel\karthick.framework\pom.xml to web.automation.killer/karthick.framework/0.0.1-SNAPSHOT/karthick.framework-0.0.1-SNAPSHOT.pomchannel stoppedTestNG Reports Processing: STARTLooking for TestNG results report in workspace using pattern: **/testng-results.xmlDid not find any matching files.Finished: SUCCESS

When I use "Input Text" to input text in the textbox, sometimes it will occur some error?

But, I just change the "Input Text" to the "Press keys" it will work normally.

Note: XPATH is correct.

Selenium doc:

Input Text-> Types the given text into the text field identified by locator.

Press keys-> Simulates the user pressing key(s) to an element or on the active browser.

complete noob over here but I hope you guys can give me some help with the problem I'm having.

Basically as a first project I'm trying to build a script that checks the weekday and if its mon-fri it should open up my browser with tabs like my college schedule and stuff like that and if it's weekend it should open different tabs.

I'm rather new to classes and I'm 99% sure I'm using them in the wrong way as both classes actually get called when I run this script. Can anyone tell me what I'm doing wrong and why both the weekday and weekend class get called?

Thanks in advance!

from selenium import webdriverfrom selenium.webdriver.chrome.options import Optionsfrom selenium.webdriver.common.by import Byfrom selenium.webdriver.common.keys import Keysimport timeimport datetimeday = datetime.datetime.now()dag = day.weekday()def Testing():    if dag >= 5:        return Weekend()    else:        return Weekday()class Weekend:    options = Options()    options.add_argument("--window-size=1920,1080")    ## options.add_argument("/Users/vadim/Library/Application Support/BraveSoftware/Brave-Browser")    options.binary_location = '/Applications/Brave Browser.app/Contents/MacOS/Brave Browser'    driver_path = '/usr/local/bin/chromedriver'    driver = webdriver.Chrome(options=options, executable_path=driver_path)    driver.get('https://outlook.live.com/mail/0/inbox')    Outlook_Aanmelden = driver.find_element_by_xpath('/html/body/header/div/aside/div/nav/ul/li[2]/a')    Outlook_Aanmelden.click()    Email_Field = driver.find_element_by_xpath('//*[@id="i0116"]')    Email_Field.send_keys('@live.com')    Outlook_Volgende = driver.find_element_by_xpath('//*[@id="idSIButton9"]')    Outlook_Volgende.click()    time.sleep(0.5)    Password_Field = driver.find_element_by_xpath('//*[@id="i0118"]')    Password_Field.send_keys('Password!')    Password_Field.send_keys(Keys.ENTER)    Inlog_Outlook = driver.find_element_by_xpath('//*[@id="idSIButton9"]')    Inlog_Outlook.click()    driver.execute_script("window.open('https://youtube.com');")class Weekday:    options = Options()    options.add_argument("--window-size=1920,1080")    ## options.add_argument("/Users/vadim/Library/Application Support/BraveSoftware/Brave-Browser")    options.binary_location = '/Applications/Brave Browser.app/Contents/MacOS/Brave Browser'    driver_path = '/usr/local/bin/chromedriver'    driver = webdriver.Chrome(options=options, executable_path=driver_path)    driver.get('https://outlook.live.com/mail/0/inbox')    Outlook_Aanmelden = driver.find_element_by_xpath('/html/body/header/div/aside/div/nav/ul/li[2]/a')    Outlook_Aanmelden.click()    Email_Field = driver.find_element_by_xpath('//*[@id="i0116"]')    Email_Field.send_keys('@live.com')    Outlook_Volgende = driver.find_element_by_xpath('//*[@id="idSIButton9"]')    Outlook_Volgende.click()    time.sleep(0.5)    Password_Field = driver.find_element_by_xpath('//*[@id="i0118"]')    Password_Field.send_keys('Password!')    Password_Field.send_keys(Keys.ENTER)    Inlog_Outlook = driver.find_element_by_xpath('//*[@id="idSIButton9"]')    Inlog_Outlook.click()    driver.execute_script("window.open('https://youtube.com');")Testing()

I am building TestNG and Cucumber framework, and want to use TestNG annotations like @BeforeMethod, @AfterMethod which will run setUp and tearDown methods before and after each scenario. It works fine if I keep all my Cucumber scenarios in separate feature files. But if I have multiple scenarios in the same feature file @AfterMethod is not running after each scenario.

Also have issues if using cucumber Before and After, they work fine only if I put them in StepDefinition file below:

public class StepDefinition extends Base {    @Before    public void setUp() throws Exception {        openBrowser();        maximizeWindow();//      implicitWait(30);        deleteAllCookies();//      setEnv();    }    @After    public void quit() throws IOException, InterruptedException {        driver.quit();    }    @Given("^Navigate to ACT landing page$")    public void navigate_to_ACT_landing_page() throws Throwable {        LandingPage landingPage = new LandingPage();        landingPage.navigateToLandingPage();    }    @Then("^Verify landing page is rendered$")    public void verify_landing_page_is_rendered() throws Throwable {        LandingPage landingPage = new LandingPage();        landingPage.landingPageRendered();    }}

But if I put them in separate class (like below) they don't work (in documentation says that they can be put anywhere in the project)

package com.act.hooks;import cucumber.api.java.After;import cucumber.api.java.Before;import java.io.IOException;import com.moodys.act.base.Base;public class Hooks extends Base {    @Before    public void setUp() throws Exception {        openBrowser();        maximizeWindow();//      implicitWait(30);        deleteAllCookies();//      setEnv();    }    @After    public void quit() throws IOException, InterruptedException {        driver.quit();    }}

i know how accept simple alert message on page using selenium. But I'm try accept the box of the pic below on https://messages.google.com/ just to delete a specific SMS, however the Selenium won't recognize that box as a alert

I just need accept, or click on the "Excluir" button, on the red arrow

Already tried:

• driver.switch_to.active_element()
• obj = driver.switch_to.alert
• obj.accept()

html code from that "alert" on page:

<div tabindex="0" class="cdk-visually-hidden cdk-focus-trap-anchor" aria-hidden="true"></div><mat-dialog-container aria-modal="true" class="mat-dialog-container ng-tns-c66-94 ng-trigger ng-trigger-dialogContainer ng-star-inserted" tabindex="-1" id="mat-dialog-13" role="dialog" aria-labelledby="mat-dialog-title-13" style="transform: none;"><!----><mws-dialog _nghost-bdb-c67="" class="ng-star-inserted"><div _ngcontent-bdb-c67=""><h1 _ngcontent-bdb-c67="" class="title mat-dialog-title" mat-dialog-title="" id="mat-dialog-title-13"> Excluir esta conversa? </h1><mat-dialog-content _ngcontent-bdb-c67="" class="content mat-dialog-content"><!----><div _ngcontent-bdb-c67="" class="ng-star-inserted" style="">Todo o histórico de mensagens será removido permanentemente. Não será possível restaurá-lo.</div><!----></mat-dialog-content><mat-dialog-actions _ngcontent-bdb-c67="" class="actions mat-dialog-actions"><!----><button _ngcontent-bdb-c67="" class="action-button cancel mat-focus-indicator mat-button mat-button-base ng-star-inserted" data-e2e-action-button-cancel="" mat-button="" style=""><span class="mat-button-wrapper"> Cancelar </span><span class="mat-button-ripple mat-ripple" matripple=""></span><span class="mat-button-focus-overlay"></span></button><!----><button _ngcontent-bdb-c67="" class="action-button confirm mat-focus-indicator mat-button mat-button-base ng-star-inserted" data-e2e-action-button-confirm="" mat-button="" style=""><span class="mat-button-wrapper"> Excluir </span><span class="mat-button-ripple mat-ripple" matripple=""></span><span class="mat-button-focus-overlay"></span></button><!----></mat-dialog-actions></div></mws-dialog></mat-dialog-container><div tabindex="0" class="cdk-visually-hidden cdk-focus-trap-anchor" aria-hidden="true"></div>

I'm trying to scrape info from Betclic website with python and BeautifulSoup + Selenium.
Given the URL for each game has the structure: "domain"/"sports_url"/"competition_url"/"match_url"
Example: https://www.betclic.pt/futebol-s1/liga-dos-campeoes-c8/rennes-chelsea-m2695669
You can try it in your language, they translate the actual URL string but the structure and ID's are the same.The only thing that's left is grabbing all the different "competition_url"

So my question now is from the "sports_url" (https://www.betclic.pt/futebol-s1) how can I get all sub "competition_url"?
The problem is with the "hidden" URL's under each country's name on the left panel. Those only appear after you click the arrow next to each country's name, like a drop-down list. The click event actually adds one class "is-active" to the <li> for that country and alsoan <ul> at the end of that <li>. It's this added <ul> that has the URL's list I'm trying to get.

Code before click:

<!----><li class="sportList_item has-children ng-star-inserted" routerlinkactive="active-link" id="rziat-DE"><div class="sportList_itemWrapper prebootFreeze"><div class="sportlist_icon flagsIconBg is-DE"></div><div class="sportlist_name">Alemanha</div></div><!----></li>

Code after click (reduced for presentation):

<li class="sportList_item has-children ng-star-inserted is-active" routerlinkactive="active-link" id="rziat-DE"><div class="sportList_itemWrapper prebootFreeze"><div class="sportlist_icon flagsIconBg is-DE"></div><div class="sportlist_name">Alemanha</div></div><!----><ul class="sportList_listLv2 ng-star-inserted"><!----><li class="sportList_item ng-star-inserted" routerlinkactive="active-link"><a class="sportList_itemWrapper prebootFreeze" id="competition-link-5" href="/futebol-s1/alemanha-bundesliga-c5"><div class="sportlist_icon"></div><div class="sportlist_name">Alemanha - Bundesliga</div></a></li>(...)</li>(...)</li>(...)</li></ul></li>

In this example is that "/futebol-s1/alemanha-bundesliga-c5" that I'm looking for.
Is there a way to get all those URL's? Or the "hiden" <ul> for that matter?
Maybe a way to simulate the click and parse the HTML code again?

Thanks in advance!

So pyperclip is a module used to copy data that can later be pasted or used for other purposes. I have been working on creating a program that automates the process of using a plagiarism checking website and I soon plan on making my plagarism checking software as well. I used both selenium and pyperclip to help me accomplish my task. This is my code:

import sysimport timefrom selenium import webdriverfrom selenium.webdriver.common.keys import Keysfrom selenium.webdriver.support.ui import WebDriverWaitfrom selenium.webdriver.support import expected_conditions as Ecimport pyperclipprint("This is a basic plagarism checker that removed of the hassle that comes with doing everything yourself")i = 0while i == 0:    open_file = input("What do you want to check, copy and paste it here:")    if len(open_file) >= 30: #ensures that text is sufficient enough to be checked        i += 1    else:        print("It is hard to check plagarism for a small amount of words, please enter some more!")        i = 0driver = webdriver.Chrome("C:/Users/Shitty Horrible Pc/PycharmProjects/learningpython/pytjom/chromedriver.exe") #access the webdriverpyperclip.copy(open_file) #copys your inputdriver.implicitly_wait(10) #tells program to wait for specific objects on the website to load indriver.get("http://plagiarisma.net/") #goes to the plagarism checker websiteelement = driver.find_element_by_id("query") #find text boxelement.send_keys(open_file) #pastes in text boxelement = driver.find_element_by_name("Submit") #finds checker buttonelement.click() #presses it and gives an authenticity report)driver.implicitly_wait(10)html = driver.find_element_by_tag_name('html')if len(open_file) >= 200:    time.sleep(22)else:    time.sleep(8)for letter in "AAAAAAAAAAAAAAAAAA":    html.send_keys(Keys.ARROW_DOWN)

As you can see, I used pyperclip to copy an input given by the user and used selenium to paste it into a text-book. Everything works fine, but when I try to copy a very lengthy input, only the first line of the input is pasted (the input is broken into segments in my pycharm console). I have manually verified that the website I am using does not share that same restriction. This leads me to assume that pyperclip might have a character limit. Does anyone know if my assumptions are correct? If not, any ideas of what else it could be?

I wonder how I can use selenium to click on web element "automation_quiz" at this javascript:_doPostBack(), The following is a snippet from the html page:

automation_quiz

I could not find the Xpath of "automation_quiz" by using firefox Xpath-finder.

I was wondering if someone could share with me what their Build Dependency looks like for Selenium for Java.I have been trying to create a Firefox driver, but have been encountering this error.

Exception in thread "main" java.util.ServiceConfigurationError: org.openqa.selenium.remote.http.HttpClient$Factory: Provider org.openqa.selenium.remote.http.okhttp.OkHttpClient$Factory could not be instantiated    at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:581)    at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:803)

None of the previous stack forums have helped. I tried downloading a few dependencies (i.e. Guava), but still am not able to resolve this error.Note I am using Java 1.8.0.221, and Eclipse IDE

I'm using api30 and trying to override the permission specific pop up as showed in the image but its not able to override and while starting the driver getting 500 error. I have tried overriding capabilities but those are not working

e.g.

cap.setCapability("autoDismissAlerts", true);cap.setCapability("autoGrantPermissions", true);cap.setCapability("autoAcceptAlerts", true);

I'm using below code while starting the driver

public static void main(String[] args) throws MalformedURLException {        AndroidDriver < AndroidElement > dr = null;        File f = new File("src");        File fs = new File(f, "ApiDemos-debug.apk");        DesiredCapabilities cap = new DesiredCapabilities();        cap.setCapability(MobileCapabilityType.DEVICE_NAME, "Pixel2API30");        cap.setCapability(MobileCapabilityType.APP, fs.getAbsolutePath());        try {            dr = new AndroidDriver < AndroidElement > (new URL("http://127.0.0.1:4723/wd/hub"), cap);        } catch (Exception e) {}        System.out.println(dr.currentActivity());    }

Also below error I'm getting[debug] [BaseDriver] Event 'newSessionStarted' logged at 1605273095642 (18:41:35 GMT+0530 (India Standard Time))[debug] [MJSONWP] Encountered internal error running command: Error: Cannot start the 'io.appium.android.apis' application. Visit https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/android/activity-startup.md for troubleshooting. Original error: Cannot start the 'io.appium.android.apis' application. Visit https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/android/activity-startup.md for troubleshooting. Original error: 'ApiDemos' or 'io.appium.android.apis.ApiDemos' never started. Visit https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/android/activity-startup.md for troubleshooting[debug] [MJSONWP] at ADB.startApp (C:\Users\anirudha.agnihotri\AppData\Roaming\npm\node_modules\appium\node_modules\appium-adb\lib\tools\apk-utils.js:177:11)[debug] [MJSONWP] at AndroidUiautomator2Driver.ensureAppStarts (C:\Users\anirudha.agnihotri\AppData\Roaming\npm\node_modules\appium\node_modules\appium-uiautomator2-driver\lib\driver.js:529:7)[debug] [MJSONWP] at AndroidUiautomator2Driver.startUiAutomator2Session (C:\Users\anirudha.agnihotri\AppData\Roaming\npm\node_modules\appium\node_modules\appium-uiautomator2-driver\lib\driver.js:405:7)[debug] [MJSONWP] at AndroidUiautomator2Driver.createSession (C:\Users\anirudha.agnihotri\AppData\Roaming\npm\node_modules\appium\node_modules\appium-uiautomator2-driver\lib\driver.js:221:7)[debug] [MJSONWP] at AppiumDriver.createSession (C:\Users\anirudha.agnihotri\AppData\Roaming\npm\node_modules\appium\lib\appium.js:358:35)[debug] [MJSONWP] Destroying socket connection[HTTP] <-- POST /wd/hub/session 500 55943 ms - 733

I know many similar questions have been asked but not a single answer helped me fix my issue.

I have a very basic selenium program in python which seems to crash when I try and run it headless.

Here is what I have so far.

from selenium import webdriverimport timeoptions = webdriver.ChromeOptions()options.add_argument('--headless')options.add_argument('--user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36')options.add_argument('--no-sandbox')options.add_argument('--disable-dev-shm-usage')  driver = webdriver.Chrome("chromedriver", options=options)driver.get("https://tinder.com/")driver.set_window_size(1920, 1080)time.sleep(0.5)driver.find_element_by_xpath("//*[@id=\"content\"]/div/div[1]/div/main/div[1]/div/div/header/div[1]/div[2]/div/button").click()currentWindow = driver.window_handles[0]time.sleep(1)driver.find_element_by_xpath('//*[@id="modal-manager"]/div/div/div[1]/div/div[3]/span/div[1]/div/button').click()time.sleep(1)newWindow = driver.window_handles[1]driver.switch_to.window(newWindow)time.sleep(1)driver.find_element_by_css_selector("input[type=email]").send_keys("someemail@mail.com")driver.find_element_by_xpath("//*[@id=\"identifierNext\"]/div/button").click()time.sleep(1)driver.find_element_by_xpath("//*[@id=\"password\"]/div[1]/div/div[1]/input").send_keys("somepasswordthatyouhave")driver.find_element_by_xpath("//*[@id=\"passwordNext\"]/div/button").click()driver.switch_to.window(currentWindow)# driver.save_screenshot("tinderOpen.png")time.sleep(5)driver.save_screenshot("crash.png")driver.find_element_by_xpath("//*[@id=\"modal-manager\"]/div/div/div/div/div[3]/button[1]").click()time.sleep(0.5)driver.find_element_by_xpath("//*[@id=\"modal-manager\"]/div/div/div/div/div[3]/button[1]").click()driver.save_screenshot("tinderOpened.png")

Now when I run this in the terminalI get this beautiful error:

Traceback (most recent call last):  File "tinder.py", line 33, in <module>    driver.save_screenshot("crash.png")  File "/home/rodude123/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 1055, in save_screenshot    return self.get_screenshot_as_file(filename)  File "/home/rodude123/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 1032, in get_screenshot_as_file    png = self.get_screenshot_as_png()  File "/home/rodude123/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 1064, in get_screenshot_as_png    return base64.b64decode(self.get_screenshot_as_base64().encode('ascii'))  File "/home/rodude123/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 1074, in get_screenshot_as_base64    return self.execute(Command.SCREENSHOT)['value']  File "/home/rodude123/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute    self.error_handler.check_response(response)  File "/home/rodude123/.local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response    raise exception_class(message, screen, stacktrace)selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crashfrom tab crashed  (Session info: headless chrome=85.0.4183.102)

See, I've tried to use no sandbox and I've even tried disabling dev shm but nothing works. I don't know what to do, please help.

Thanks, rodude123

I am writing a script that is constantly looping and will need to stop when it detects that there is text in a certain element on the page. I couldn't find any help in the selenium python docs about doing this but there must be a way. In case you still don't understand what I need to do, here is some code:

While #element is not visible:   #do stuff#when element is visible break out of statement and do something else

EDIT: I know the exact text and the exact xpath of the element i am looking for. However, I am just looking for a general response so I don't mind an answer lacking detail.

import time from bs4 import BeautifulSoup from bs4.element import Tagimport pip._internal.distributions from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.support.ui import WebDriverWait n = ['2020','2019','2018'] base = 'https://www.bseindia.com' browser =webdriver.Chrome('/home/anuj/PycharmProjects/firstfrog/chromedriver')wait = WebDriverWait(browser, 10)browser.get('https://www.bseindia.com/stock-share-price/financials/annualreports/500104/')alert_name = browser.find_elements_by_xpath('//*[@class="ng-scope"]/td')print(alert_name) for value in alert_name:              if value.text in n:                        url_d = browser.find_elements_by_xpath('//*[@class="ng-scope"]/td/td/a')           print(url_d)           print(value.text)## Heading ##

visibilityOfElementLocated() method in selenium is not working well, sometimes this method passes true, and after that when i trying to fetch element by findElement() is gives error is element is not visible. Following is the code.

waitPath.until(ExpectedConditions.visibilityOfElementLocated(By.id("Table1")));WebElement we1 = pathfinderdriver.findElement(By.id("Table1"));waitPath.until(ExpectedConditions.visibilityOfElementLocated(By.id("Table3")));

Even i have checked with following code but sometimes it also fails.

while(!(we1.findElement(By.id("Table3")).isDisplayed() ))            {           Thread.sleep(3000);}Thread.sleep(5000);WebElement we2 = we1.findElement(By.id("Table3"));

I'm building a web crawler using Python 3.8. What I want to do is transform the table below in a pandas dataframe using Selenium, pandas, bs4.

<table class="no_border_top" width="95%" align="center"><tbody><tr><th align="left" valign="top" rowspan="100" width="8%">1.2.12</th><th align="left" colspan="4">Outras cotas de Fundos de Investimento</th>          </tr><tr><td width="40%"><b>Fundo</b></td><td width="22%"><b>CNPJ</b></td><td width="15%"><b>Quantidade</b></td>           </tr><tr><td>Itaú Soberano RF Simples LP FICFI</td><td>06.175.696/0001-73</td><td>247.719,87</td><td>11.996.245,91</td>            </tr><tr><td>Itaú TOP RF Referenciado DI FICFI</td><td>05.902.521/0001-58</td><td>77.085,90</td><td>372.686,27</td></tr></tbody></table>

Btw, this is the link im trying to scrap data:

• https://fnet.bmfbovespa.com.br/fnet/publico/visualizarDocumento?id=111845&cvm=true

The Problem

If you were able to open the link (and if that doesn't work out i'll edit this question posting some prints from the website html ), you'll see that the table im interested is one of many tables inside a html document inside the webpage html im scrapping info. All the tables inside this nested html have the same class, they are all <table class="no_border_top" width="95%" align="center">...</table>.selenium provides some tools to cath elements given the id, class, xpath of the element and etc. I've tried to grab this table using find_element_by_xpath()( passing the xpath collected by copying the full XPath given by Chrome Dev Tools) and find_element_by_link_text(). but none of those options worked out for me, this is the code im using:

def make_selenium_browser():    options = Options()    options.headless = True    options.add_argument('--no-sandbox')    options.add_argument('--disable-dev-shm-usage')            browser = Firefox(options=options)    return browserurl = 'https://fnet.bmfbovespa.com.br/fnet/publico/visualizarDocumento?id=111845&cvm=true'browser = make_selenium_browser()browser.get(url)#not workingdata = browser.find_element_by_xpath('/html/body/table[28]')#this is also not workingbrowser.find_element_by_partial_link_text('Outras cotas de Fundos de Investimento')

The error

 File "/home/occhima/miniconda3/envs/wdev/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 462, in find_element_by_partial_link_text    return self.find_element(by=By.PARTIAL_LINK_TEXT, value=link_text)  File "/home/occhima/miniconda3/envs/wdev/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element    return self.execute(Command.FIND_ELEMENT, {  File "/home/occhima/miniconda3/envs/wdev/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute    self.error_handler.check_response(response)  File "/home/occhima/miniconda3/envs/wdev/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response    raise exception_class(message, screen, stacktrace)selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: Outras cotas de Fundos de Investimento

How can I make Selenium find the table I'm interested in?

`PATH = "C:\Program Files (x86)\chromedriver.exe" driver= webdriver.Chrome(PATH) for x in range (1,25):    url='http://catalog.aucegypt.edu/content.php?catoid=36&catoid=36&navoid=1738&filter%5Bitem_type%5D=3&filter%5Bonly_active%5D=1&filter%5B3%5D=1&filter%5Bcpage%5D='+str(x)+'#acalog_template_course_filter'    driver.get(url)     Courses_names = driver.find_elements_by_xpath('//td[@class="width"]/a')      #print(course_Prerequisites)#num_courses=len(Courses_names)#for i in range(num_courses):    #print(Courses_names[i].text)   for i in Courses_names:    print(i.text)           hrefs=i.get_attribute('href')    print(hrefs)    course_content=hrefs.find_elements_by_xpath('//td[@class="block_content"]')    print(course_content.text)    driver.close()`

I need to get the content inside of Herf, but this error comes up course_content=hrefs.find_elements_by_xpath('//td[@class="block_content"]/a')AttributeError: 'str' object has no attribute 'find_elements_by_xpath'