M
M
MefistofelKr2016-09-22 17:41:29
Parsing
MefistofelKr, 2016-09-22 17:41:29

What Language and what library should be used for the Football Betting Site Parser?

I want to write a service for myself to parse certain events and find forks. It is necessary to select a language for parsing HTML with the ability to hide and replace the server ip. Ideally, a server platform, on which I am making a project on laravel that will output forks. The parser must hide its ip and enter data into mysql
. There is already a php code with the DiDOM library and everything worked successfully, everything worked on the vps server and found plugs, but recently my vps grid was blocked and now the sites are not available.
Question. What parser can work through proxy, vpn, Mysql output and mimic browsers?

Answer the question

In order to leave comments, you need to log in

4 answer(s)
W
WebDev, 2016-09-22
@kirill-93

And what about the programming language? You make requests through a proxy, the language is not important.
CURL knows how to use proxy.

R
Rou1997, 2016-09-22
@Rou1997

What parser can work through a proxy, vpn,

Any.
It's no, do it yourself.
Anyone, but it will take a lot of effort to support all headers, cookies, etc. to implement it yourself on HTTP, it may be worth using an embedded browser ("engine") to simulate a browser, for example, specially designed Selenium and Phantom.js, and also look here .

W
wscms, 2016-09-22
@wscms

Pull pages via CURL using list of proxies
Save individual cookies for each proxy
Pulled pages parse using the same Simple HTML Dom

V
Vyacheslav Plisko, 2016-09-23
@AmdY

Out of the box, guzzle is already available to you, it supports proxies docs.guzzlephp.org/en/latest/request-options.html
Well, most likely the site has an api where there will be no problems with bans and the format will not be html, contact its support.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question