Answer the question
In order to leave comments, you need to log in
Scrapy & PyQt4: how to send start to Scrapy from GUI and receive responses in GUI?
Hello fellow toasters.
I was developing a GUI for Scrapy and got confused: in the GUI, you need to load the module with Scrapy in order to run it with parameters, and in Scrapy, import the GUI in order to send information to QLabel and the second window. But when importing
To do this, I import a function from Scrapy and a class from the GUI, but I get:
File "C:\Users\Eugene\Scrapy\Majestic\gui\gui_for_scrapy.py", line 5, in <module>
from spider_find_domains import run
File "C:\Users\Eugene\Scrapy\Majestic\gui\spider_find_domains.py", line 16, in <module>
from gui_for_scrapy import MyWindow
File "C:\Users\Eugene\Scrapy\Majestic\gui\gui_for_scrapy.py", line 5, in <module>
from spider_find_domains import run
ImportError: cannot import name run
def run():
options = {
'CONCURRENT_ITEMS': concurrent_items,
'USER_AGENT': 'Googlebot/2.1 (+http://www.google.com/bot.html)',
'DOWNLOAD_DELAY': download_delay,
'CONCURRENT_REQUESTS': concurrent_requests,
'HTTPERROR_ALLOW_ALL': True,
'SPIDER_MIDDLEWARES': {
'scrapy.contrib.spidermiddleware.httperror.HttpErrorMiddleware': 300,
'scrapy.contrib.spidermiddleware.offsite.OffsiteMiddleware': 400,
}
}
spider = SpiderParse()
settings = get_project_settings()
settings.overrides.update(options)
crawler = Crawler(settings)
crawler.signals.connect(reactor.stop, signal=signals.spider_closed)
crawler.install()
crawler.configure()
crawler.crawl(spider)
crawler.start()
log.start(logfile="results.log", loglevel=log.DEBUG, crawler=crawler, logstdout=False)
reactor.run()
Answer the question
In order to leave comments, you need to log in
> Scrapy - import GUI in order to send information to QLabel and second window.
and for this you import your own GUI module, which in turn imports your own Scrapy module, and so on? Why not just connect the necessary part of Qt, create a signal and connect the signal to the GUI already in the GUI module itself?
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question