Quantcast
Channel: Active questions tagged selenium - Stack Overflow
Viewing all articles
Browse latest Browse all 97774

Scraping an Angular website with selenium and C# returns the angular script, not the rendered web page

$
0
0

This seems to be by design as far as I can tell. Selenium can see the initially loaded HTML, but not the HTML after it's been massaged. I've tried IE, Chrome and PhantomJS and they all show the same behavior. So does the built-in Chrome debugger, until you inspect an element on the page, you can't query any of the rendered HTML.

I'm looking for any suggestions about how to scrape the web page. The only option I see right now is finding the chrome process, triggering the inspector, clicking inside, then running the Javascript. Needless to say, this sounds fragile.

I also haven't been able to find anything on capturing the Ajax calls from selenium so I can make them and capture the JSON. When tried copy / paste from the chrome network tab into selenium I got a missing application block message.

Does anyone have any other advice?

Since I can replicate the issue in the chrome debugger, I don't see posting code as useful. It looks like a design decision.

Ralph


Viewing all articles
Browse latest Browse all 97774

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>