HTTP request C# .NET?

Question

Microsoft

I use HTTP request to get site pages.
Something like this:
...
HttpWebRequest request = WebRequest.Create(Url) as HttpWebRequest;
…
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
...
Sometimes everything works like clockwork, but sometimes it happens that a page with a size of several KB is not loaded ... it just hangs, and in the end everything falls down on a timeout.
Attention! Question:
Why do all the lags disappear when I turn on the transparent proxy on the localhost? A simple example of such a proxy is fiddler (aka HTTP sniffer).
If I turn off the proxy, these lags immediately return.

Answer 1

See what the difference is in the headings. Nothing more can be said here.

Answer 2

Then only the last option is that the proxy caches and therefore there are no lags. If your proxy is transparent, then in fact there is no difference in the direct connection or through it, only the cache remains.

Answer 3

If you compile (in the studio or without), then the lags may be due to the firewall, because. it determines that a new program is breaking into the Internet.
Through a proxy - everything is OK, because the proxy program is still the same, and the firewall rules are the same as before.

Answer 4

Don't put too much hope in fiddler, it works really weird sometimes. Once, after the normal completion of fiddler, my program stopped working for me, although I compiled it again and ran it under the studio and without it. Everything was resolved only at the subsequent launch of the fidler.
And falling on a timeout is normal. All the same, these exceptions need to be caught, and the request should be sent again. Just do not make an infinite loop, you still need to limit the number of connection attempts.

Answer 5

I had a similar problem, after 2 days of parsing it turned out that a certain sequence of requests was violated, i.e. Before fulfilling the required Request, it was also necessary to perform an empty POST to a certain page.

Answer 6

Compiled for release.
Created a firewall rule.
Nothing helped.
Everything also works fine with the included fiddler, and still timeouts without it.
The proxy does not cache, it is essentially a sniffer, it does not need to cache.
ZY
Checked on 3 computers (working, home, remote server behind the cordon)
Maybe the problem is in the HttpWebRequest itself?

Answer 7

Absolutely not.
This is excluded by the facts of checks in 3 different places.
And in general, I never noticed jambs.

Answer 8

If there are a large number of requests, then there may be a problem in the transport layer. By default, 2 physical connections are available for the application (if Windows 7 or 2008 R2, you can see them in the resource monitor). This is solved by setting in the config or in the ServicePoint.ConnectionLimit code