I'm trying to scrape many parts of many websites, with for loop.
I tried BeautifulSoup and Selenium, but it didn't return all the html part that I want. Actually, the result that I got from
soup = BeautifulSoup(src, 'lxml')
,or other parsers, is not the same as the full html code I saw in "Developer tools" in Chrome
So I'm trying to scrape the code from "Developer tools" instead, likes copying the full html code in "Developer tools" after entering the website.
How can I do that? It would be appreciated if you could give some code for example.
Thank you in advance.