D
D
Denis Goncharenko2018-05-15 10:08:15
API
Denis Goncharenko, 2018-05-15 10:08:15

Through what to parse?

I made a parser, it was necessary to get a Google page with search results. I did it with curl. Everything works well on the localhost, on my server too, but now it doesn’t work from the customer’s cloud hosting. And it’s not about the headers or the user agent, how much information you don’t send to Google with or without cookies, it doesn’t matter to him, he stupidly doesn’t like the ip on which the site hangs. Due to the fact that many sites in the cloud hang on it and apparently everyone sends requests to Google, it considers this to be suspicious activity and an IP block, redirecting it to a captcha page (this can usually happen when using vpn in a browser). I see 2 solutions to this problem: you can use a proxy, but curl requests are not sent through a proxy, even on my server, it only works on local, apparently hosters block outgoing requests on non-standard ports. The second option is to buy a dedicated ip for a site that won't be considered suspicious by google, but zach says he can't get a private ip on cloudflare. There is only one way out, to send requests through some extraneous "valid" server. Send him a request so that he parses the Google page and returns the data. Accordingly, this server must be available all the time, if there are any services with similar services? Or do you recommend something else? if there are any services with similar services? Or do you recommend something else? if there are any services with similar services? Or do you recommend something else?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
P
Puma Thailand, 2018-05-15
@opium

The obvious solution is to move to the VDS and use a proxy from there

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question