Answer the question
In order to leave comments, you need to log in
Why does docker-scrapyd use Python 2.7 instead of Python 3.7?
Using what seems to be a nice updatable vimagick/scrapyd image along with thecjw/scrapydweb , the .egg file is built for Python 3.7 and uses type annotations. Error loading .egg in scrapyd
{
"auth": null,
"message": "See details below",
"node_name": "2d354ebaf1d7",
"status": "error",
"status_code": 200,
"url": "http://scrapyd:6800/addversion.json",
"when": "2019-06-04 13:28:25"
}
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/local/lib/python2.7/dist-packages/scrapyd/runner.py", line 40, in <module>
main()
File "/usr/local/lib/python2.7/dist-packages/scrapyd/runner.py", line 37, in main
execute()
File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 149, in execute
cmd.crawler_process = CrawlerProcess(settings)
File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 264, in __init__
super(CrawlerProcess, self).__init__(settings)
File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 142, in __init__
self.spider_loader = _get_spider_loader(settings)
File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 351, in _get_spider_loader
return loader_cls.from_settings(settings.frozencopy())
File "/usr/local/lib/python2.7/dist-packages/scrapy/spiderloader.py", line 61, in from_settings
return cls(settings)
File "/usr/local/lib/python2.7/dist-packages/scrapy/spiderloader.py", line 25, in __init__
self._load_all_spiders()
File "/usr/local/lib/python2.7/dist-packages/scrapy/spiderloader.py", line 47, in _load_all_spiders
for module in walk_modules(name):
File "/usr/local/lib/python2.7/dist-packages/scrapy/utils/misc.py", line 63, in walk_modules
mod = import_module(path)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/tmp/JaAuto-2019-06-04T10_26_36-oRHPI3.egg/JaAuto/spiders/__init__.py", line 17
def parse(self, response: scrapy.http.HtmlResponse):
^
SyntaxError: invalid syntax
Answer the question
In order to leave comments, you need to log in
Hypothesis: because the image is based on Debian and everything is installed via pip instead of pip3.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question