S
S
Satangelus2019-01-01 20:31:43
C++ / C#
Satangelus, 2019-01-01 20:31:43

How to limit the number of running tasks at the same time?

There is a wonderful site whose search script only looks for one day's worth of data.
I wrote a program that automates the process, but it turned out there is a very big problem. If the simultaneous number of sessions from one ip exceeds a certain number, the server starts returning 503.
1. How to limit the number of simultaneous tasks performed?
2. When executing Task.WhenAll(tasks.ToArray()); the ui-stream is blocked, and the informer stub form that I show during the search is not normally drawn.
How to deal with blocking, I want to show a normal informant for the duration of the search.

public static async Task ProcessUrlLawAsync(String Урл)
        {
            try
            {
                using (var webClient = new WebClient())
                {
                    string data = await webClient.DownloadStringTaskAsync(new Uri(Урл));
                    // run checks here.. 
                    Парсинг(data);
                }
            }
            catch (Exception ex)
            {
                //Catch my error here and handle it (display message box)
                log.Error("Не удалось скачать  ссылку="+Урл, ex);
            }
        }

        private async void button1_Click(object sender, EventArgs e)
        {
            var tasks = new List<Task>();
            for (DateTime date = полеДатаС.Value.Date; date.Date <= полеДатаПо.Value.Date; date = date.AddDays(1))
            {
                foreach(var текСтрока in СписокУрл)
                {
                    Task fooWrappedInTask = Task.Run(() => ProcessUrlLawAsync(текСтрока) );
                    tasks.Add(fooWrappedInTask);
                }
            }
            await Task.WhenAll(tasks.ToArray());
        }

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Alexander, 2019-01-02
@alexr64

1. How to limit the number of tasks performed at the same time?

Count the number of already running ones, add the excess to the queue, as soon as the execution of a task is over, take the next one from the queue and execute it.
Do Task.WhenAll(tasks.ToArray()); in another thread.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question