What is the problem when downloading a web page and how to solve it?

K

kos_dev2017-02-17 17:52:49

HTML

kos_dev, 2017-02-17 17:52:49

Hello, I'm trying to parse information from an HTML document, here's the address: https://youhack.ru/forums/51/ using the WebClient.DownloadString() method;
Here is my code:

WebClient webClient = new WebClient();
String response;

response = webClient.DownloadString("https://youhack.ru/forums/51/");
document.Text = response;
string pattern = @"data.previewUrl..threads.[0-9]+.preview..\w{0,70}\W{0,70}" + searchTextBox.Text + "\\w{0,70}\\W{0,70}";
Regex rgx = new Regex(pattern);
int count = 0; 
foreach (Match match in rgx.Matches(response))
{
     count++;
     resultTextBox.Text += "youhack.ru/" + match.Value.ToString().Substring(17, 14) + Environment.NewLine;
     resultsMatched.Text = count.ToString();
}

Instead of a web page from this address, I get this page:

<html><head></head><body><script>var xmlhttp = new XMLHttpRequest();function 
eraseCookieFromAllPaths(name) { var pathBits = location.pathname.split("/");var pathCurrent = " path=";
document.cookie = name + "=; 
expires=Thu, 01-Jan-1970 00:00:01 GMT;";
for (var i = 0; i < pathBits.length; i++) {pathCurrent += ((pathCurrent.substr(-1) != "/") ? "/" : "") + pathBits[i];document.cookie = name + "=;
 expires=Thu, 01-Jan-1970 00:00:01 GMT;"
 + pathCurrent + ";";}}
eraseCookieFromAllPaths("BHC");
xmlhttp.onreadystatechange=function() { 
if (xmlhttp.readyState==4 && xmlhttp.status==200) { var a=xmlhttp.responseText;document.cookie="BHC="+a+";
 path=/";
document.location.href="/forums/51/"; } };
xmlhttp.open("GET", "/banhammer/pid", true);xmlhttp.send();</script></body></html>

I didn’t quite understand, but it seems to me that the matter is in cookies, tell me how to be, how to get around this problem?

Reply

Answer the question

In order to leave comments, you need to log in

1 answer(s)

A

alex maslakoff, 2017-02-17
@teke_teke

in theory, it will crash sometimes, even if you log in from the browser, because this is some kind of mechanism for this site. so no way. or check that it is she and make a new request.