A
A
ArtemZA2017-08-29 20:38:36
Python
ArtemZA, 2017-08-29 20:38:36

What is the best way to scrape information from a website?

Hello.
There is a site https://strelkacard.ru/ - the site allows you to find out the balance on the card.
In order not to constantly climb into the browser, I decided to make a script in Python.
But I don't really know how best to solve this problem
. As a start, I wrote a simple script using Selenium, I managed to get the balance, but it's still the same browser, albeit automated.
I decided to try Scrapy, but it gives an error when installing via pip

Cleaning up...
  Removing temporary dir C:\Users\Artem\AppData\Local\Temp\pip_build_Artem...
Command python setup.py egg_info failed with error code 1 in C:\Users\Artem\AppData\Local\Temp\pip_build_Artem\cffi
Exception information:
Traceback (most recent call last):
  File "D:\Program Files\Python\lib\site-packages\pip\basecommand.py", line 122, in main
    status = self.run(options, args)
  File "D:\Program Files\Python\lib\site-packages\pip\commands\install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "D:\Program Files\Python\lib\site-packages\pip\req.py", line 1229, in prepare_files
    req_to_install.run_egg_info()
  File "D:\Program Files\Python\lib\site-packages\pip\req.py", line 325, in run_egg_info
    command_desc='python setup.py egg_info')
  File "D:\Program Files\Python\lib\site-packages\pip\util.py", line 697, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command python setup.py egg_info failed with error code 1 in C:\Users\Artem\AppData\Local\Temp\pip_build_Artem\cffi

Answer the question

In order to leave comments, you need to log in

4 answer(s)
S
screen_sailor, 2017-08-30
@screen_sailor

Try installing not via pip, but as the doc says:
Though it's possible to install Scrapy on Windows using pip, we recommend you to install Anaconda or Miniconda and use the package from the conda-forge channel, which will avoid most installation issues.

B
Bjornie, 2017-08-29
@Bjornie

Command python setup.py egg_info failed with error code 1 in C:\Users\Artem\AppData\Local\Temp\pip_build_Artem\cffi

D
Dmitry, 2017-08-29
@kashamalasha

You can try Requests + Beautiful Soup.

X
x67, 2017-08-30
@x67

What for steroid pumping like selenium, when a task of 5 lines is solved by the standard requests library. In the same place, you don’t need anything at all, send a GET request and get a response in JSON format. This is almost an open API, nothing complicated and everything is convenient. Do not forget to copy all the headers from the browser so that the site does not swear (just in case).

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question