Answer the question
In order to leave comments, you need to log in
Python(scrapy) am I doing everything right?
Good evening,
I'm trying to parse this page vulners.com as a parameter I pass CVE (for example: vulners.com/cve/CVE-2017-0147), then on this page I try to get the entire contents of the DIV container "vulners-card-text", with python and scrapy I encountered for the first time, I compiled this script using examples on the network:
#!/usr/bin/python
import scrapy
from scrapy.crawler import CrawlerProcess
"""EX:CVE-2017-0147"""
class PythonEventsSpider(scrapy.Spider):
name = 'pythoneventsspider'
start_urls = ['https://vulners.com/cve/'+sys.argv[1], ]
events_dict = {}
def parse(self, response):
i = 0
for events in response.xpath('//div[contains(@class, "vulners-item-full")]'):
event_type = events.xpath('.//div[contains(@class, "vulners-item-description")]').extract_first()
event_title = events.xpath('.//div[contains(@class, "vulners-item-cpe")]').extract_first()
event_desc = events.xpath('.//div[contains(@class, "vulners-item-references")]').extract_first()
events_dict[i] = [event_type, event_title, event_desc]
i += 1
if __name__ == "__main__":
process = CrawlerProcess({'LOG_LEVEL': 'ERROR'})
process.crawl(PythonEventsSpider)
spider = next(iter(process.crawlers)).spider
process.start()
Traceback (most recent call last):
File "./pars.py", line 6, in
class PythonEventsSpider(scrapy.Spider):
File "./pars.py", line 8, in PythonEventsSpider
start_urls = [' https:/ /vulners.com/cve/ '+sys.argv[1], ]
NameError: name 'sys' is not defined
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question