Answer the question
In order to leave comments, you need to log in
Why is the request not returning a response in Scrapy?
Hello
CODE:
import scrapy
from scrapy.spiders import CrawlSpider
from scrapy import Request
class TestSpider(CrawlSpider):
name = "test_spyder"
allowed_domains = ["toster.ru"]
start_urls = ['https://toster.ru/tag/scrapy/questions']
def parse(self, response):
title_1 = response.xpath('//h1/text()').extract_first()
next_url = 'https://toster.ru/tag/crawling/questions'
title_2 = Request(url=next_url, callback=self.parse_some)
yield {'title_1': title_1, 'title_2': title_2}
def parse_some(self, response):
return response.xpath('//h1/text()').extract_first()
{"title_1": "Scrapy", "title_2": "<Request GET https://toster.ru/tag/crawling/questions>"}
Answer the question
In order to leave comments, you need to log in
Sorry, but this code is wrong. I wanted to start writing in detail, but this will be a whole article and not an answer.
Try redoing the scrapy tutorial step by step: https://doc.scrapy.org/en/latest/intro/tutorial.html
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question