M
M
maximkalga2015-02-05 17:35:16
Python
maximkalga, 2015-02-05 17:35:16

How to work with Scrapy via command line on Ubuntu?

Can you please tell me how to implement the function of passing start_urls, start_page, end_page through command line arguments in Scrapy?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
S
Stanislav Fateev, 2015-02-05
@svfat

Here is an elementary example of working with arguments, parse the input, shove what you need into variables and give it to scrapy

#!/usr/bin/python
# -*- coding: UTF-8 -*-

import sys

if __name__ == "__main__":
    for param in sys.argv:
        print (param)

S
sakuradaj, 2015-02-05
@sakuradaj

If you understand correctly what you need then:
stackoverflow.com/questions/15611605/how-to-pass-a...
doc.scrapy.org/en/latest/topics/spiders.html#spide...

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question