Quantcast
Channel: Active questions tagged selenium - Stack Overflow
Viewing all articles
Browse latest Browse all 98817

Scrapy Is not Crawling Anything

$
0
0

When I am running it in terminal the output isenter image description hereScrapy Crawler is Crawling when i run the code in terminal but it is not working when i am running the same in editor. My Code is:

from scrapy import Spider
from selenium import webdriver
from scrapy.selector import Selector
from scrapy.http import Request
class BooksSpider(Spider):
    name = 'books'
    allowed_domains = ['books.toscrape.com']


def start_request(self, response):
    self.driver=webdriver.Chrome(r'C:\Users\Night-Watch\Desktop\chromedriver.exe')
    self.driver.get('http://books.toscrape.com')
    sel=Selector(text=self.driver.page_source)


    books=sel.xpath('//h3/a/@href').extract()
    print(books)

Viewing all articles
Browse latest Browse all 98817

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>