Quantcast
Channel: Active questions tagged selenium - Stack Overflow
Viewing all articles
Browse latest Browse all 98814

Can't select drop-down menu in Python app

$
0
0

enter image description here

I have a script to scrape values from a website, with a drop-down hover menu in which the parent menu needs to be hovered over and a drop-down option selected. This was working perfectly up until a few days ago. But now is failing randomly.

This is my current code:

def select_dropdown():

    WebDriverWait(driver, 10).until(ec.visibility_of_element_located(
        (By.CSS_SELECTOR, "div.menu.menuTopCenter > ul > li:nth-child(3) > a")))
    investment = WebDriverWait(driver, 10).until(ec.element_to_be_clickable(
        (By.CSS_SELECTOR, "div.menu.menuTopCenter > ul > li:nth-child(3) > a")))
    actions.move_to_element(investment).perform()
    WebDriverWait(driver, 10).until(
         ec.element_to_be_clickable((By.CSS_SELECTOR, "li:nth-child(3) > div > div:nth-child(1) > a"))).click()

See the HTML below for the parent menu:

<a class="menuLink mainMenuItem" href="#" renderstyle="NONE" action="" 
 controller="" clientmodulesubid="0" renderlocation="" 
 clearcontentheader="False" 
 lobsystemuserids="Ls6EVzOdmMPVcdHshYcUbg==">Investment</a>

See the HTML below for the option to be selected (sub_menu):

<a class="menuLink mainMenuItem" href="#" renderstyle="REPLACE" action="Index" 
 controller="Portfolio" clientmodulesubid="6109" 
 renderlocation="contentHeaderContainer" clearcontentheader="True" 
 lobsystemuserids="Ls6EVzOdmMPVcdHshYcUbg==">Investment summary</a>

This is the error I get:

Traceback (most recent call last):
  File "C:/Users/SChogle/PycharmProjects/Web Scraping All Sites (With BDay).py", line 55, in <module>
    select_dropdown()
  File "C:/Users/SChogle/PycharmProjects/Web Scraping All Sites (With BDay).py", line 29, in select_dropdown
    ec.element_to_be_clickable((By.CSS_SELECTOR, "li:nth-child(3) > div > div:nth-child(1) > a"))).click()
  File "C:\Users\SChogle\AppData\Roaming\Python\Python37\site-packages\selenium\webdriver\support\wait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: 

This is the full HTML for the parent menu and all sub-menus:

<li class="has-children">
<a class="menuLink mainMenuItem" href="#" 
 renderstyle="NONE" action="" controller="" clientmodulesubid="0" 
 renderlocation="" clearcontentheader="False" 
 lobsystemuserids="8qWAwlpk6Jdje8MVVaH1Tw==">Investment</a>

<div class="subMenu"> 
 <div>

<a class="menuLink mainMenuItem" href="#" renderstyle="REPLACE" 
 action="Index" controller="Portfolio" clientmodulesubid="6109" 
 renderlocation="contentHeaderContainer" clearcontentheader="True" 
 lobsystemuserids="8qWAwlpk6Jdje8MVVaH1Tw==">Investment summary</a>
</div>

<div>
<a class="menuLink mainMenuItem" href="#" renderstyle="REPLACE" 
 action="GetLazyTransactionHistoryByAccountGroup" controller="Transactions" 
 clientmodulesubid="6108" renderlocation="contentHeaderContainer" 
 clearcontentheader="True" 
 lobsystemuserids="8qWAwlpk6Jdje8MVVaH1Tw==">Transaction history</a>
</div>

<div> 
 <a class="menuLink mainMenuItem" href="#" renderstyle="REPLACE" 
 action="LoadIncomeDistribution" controller="IncomeDistribution" 
 clientmodulesubid="7005" renderlocation="contentHeaderContainer" 
 clearcontentheader="True" lobsystemuserids="8qWAwlpk6Jdje8MVVaH1Tw==">Income 
 distribution</a>
</div>

<div>
<a class="menuLink mainMenuItem" href="#" 
 renderstyle="REPLACE" action="PortfolioFees" controller="Fees" 
 clientmodulesubid="6159" renderlocation="contentHeaderContainer" 
 clearcontentheader="True" lobsystemuserids="8qWAwlpk6Jdje8MVVaH1Tw==">Advisor 
 charges</a>
</div>

<div><a class="menuLink mainMenuItem" href="#" 
 renderstyle="REPLACE" action="GetRecurringInstructionsDetail" 
 controller="RecurringInstructions" clientmodulesubid="6158" 
 renderlocation="contentHeaderContainer" clearcontentheader="True" 
 lobsystemuserids="8qWAwlpk6Jdje8MVVaH1Tw==">Recurring instructions</a>
</div> 

<div>
<a class="menuLink mainMenuItem" href="#" renderstyle="REPLACE" 
 action="GenerateAdHocWebStatement_SAIP" controller="AdHocWebStatement" 
 clientmodulesubid="6121" renderlocation="contentHeaderContainer" 
 clearcontentheader="True" lobsystemuserids="8qWAwlpk6Jdje8MVVaH1Tw==">Investor 
 statement</a>
</div>

<div>
<a class="menuLink mainMenuItem" href="#" 
 renderstyle="REPLACE" action="GenerateAdHocWebStatement_SAIP" 
 controller="AdHocWebStatement" clientmodulesubid="8939" 
 renderlocation="contentHeaderContainer" clearcontentheader="True" 
 lobsystemuserids="8qWAwlpk6Jdje8MVVaH1Tw==">Tax certificate</a>
</div>
</div> 
 </li>

I think MAYBE what is happening is that the script is trying to find the element before the page has loaded, but I am unsure. As I said it works sometimes and fails other times.


Viewing all articles
Browse latest Browse all 98814

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>