Answer the question
In order to leave comments, you need to log in
What is the best way to scrape information from a website?
Hello.
There is a site https://strelkacard.ru/ - the site allows you to find out the balance on the card.
In order not to constantly climb into the browser, I decided to make a script in Python.
But I don't really know how best to solve this problem
. As a start, I wrote a simple script using Selenium, I managed to get the balance, but it's still the same browser, albeit automated.
I decided to try Scrapy, but it gives an error when installing via pip
Cleaning up...
Removing temporary dir C:\Users\Artem\AppData\Local\Temp\pip_build_Artem...
Command python setup.py egg_info failed with error code 1 in C:\Users\Artem\AppData\Local\Temp\pip_build_Artem\cffi
Exception information:
Traceback (most recent call last):
File "D:\Program Files\Python\lib\site-packages\pip\basecommand.py", line 122, in main
status = self.run(options, args)
File "D:\Program Files\Python\lib\site-packages\pip\commands\install.py", line 278, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "D:\Program Files\Python\lib\site-packages\pip\req.py", line 1229, in prepare_files
req_to_install.run_egg_info()
File "D:\Program Files\Python\lib\site-packages\pip\req.py", line 325, in run_egg_info
command_desc='python setup.py egg_info')
File "D:\Program Files\Python\lib\site-packages\pip\util.py", line 697, in call_subprocess
% (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command python setup.py egg_info failed with error code 1 in C:\Users\Artem\AppData\Local\Temp\pip_build_Artem\cffi
Answer the question
In order to leave comments, you need to log in
Try installing not via pip, but as the doc says:
Though it's possible to install Scrapy on Windows using pip, we recommend you to install Anaconda or Miniconda and use the package from the conda-forge channel, which will avoid most installation issues.
Command python setup.py egg_info failed with error code 1 in C:\Users\Artem\AppData\Local\Temp\pip_build_Artem\cffi
What for steroid pumping like selenium, when a task of 5 lines is solved by the standard requests library. In the same place, you don’t need anything at all, send a GET request and get a response in JSON format. This is almost an open API, nothing complicated and everything is convenient. Do not forget to copy all the headers from the browser so that the site does not swear (just in case).
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question