R
R
RAMS_0072018-10-26 21:09:06
Parsing
RAMS_007, 2018-10-26 21:09:06

What are the modern methods of site parsing?

Good afternoon colleagues. tell me what progressive humanity is now using to parse sites?
I'm sure there's something newer, usually CURL.
In particular, I ran into the problem of parsing such a page:
https://www.bloomberg.com/profiles/companies/00035...
tried:
CURL on the command line,
CURL in php,
Guzzle (the same curl but in a beautiful wrapper),
PhantomJS ,
python requests,
python scrapy,
nightmareJs (the only one who showed the page the first time but then did not work again)
none of this helped. somehow the site determines the parsers and does not return the data. Strangely, a normal POSTMAN request works fine.
Share your experience, or in this case only Selenium can handle it?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
D
Dimonchik, 2018-10-26
@dimonchik2013

Scrapy + everything you need from them, incl. for JS
Bloomberg, if cho, little boys don’t allow themselves to be robbed just like that, but with proxy vpns within reasonable limits easily

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question