Is there an example of a multi-threaded HTTP client on OpenSSL?

Question

OpenSSL

Guys, please share an example of a multi-threaded HTTP client on OpenSSL.
Interested in:
1. OpenSSL 1.1+ (the API for multithreading has changed there)
2. HTTP 1.1 (in particular Keep-alive)
3. Which handles (ctx, ssl, bio) can be stored globally, and which ones are needed for each thread?
4. What and when should I lock (CRYPTO_THREAD_write_lock) in a thread? Now I'm locking everything, maybe it's possible to lock only the connect moment?
5. How to determine that the client has closed the connection and reconnect? Preferably before writing to it.
We will omit the verification of certificates and other security for simplicity. In the example, each thread connects to the same host, in fact, to different hosts. Within one thread, of course, the connection and the host are permanent.
Now something like this (for Windows, but only in terms of creating a thread):

CRYPTO_RWLOCK *lock;
SSL_CTX* ctx;
SSL* ssl; // может быть static?

static void init() {
SSL_library_init();
SSL_load_error_strings();
ERR_load_BIO_strings();
OpenSSL_add_all_algorithms();
lock = CRYPTO_THREAD_lock_new();
ctx = SSL_CTX_new(TLS_method());
SSL_CTX_set_options(ctx, SSL_OP_ALL | SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3);
}

static void finish() {
CRYPTO_THREAD_lock_free(lock);
SSL_free(ssl);
SSL_CTX_free(ctx);
}


#define host "example.com"

DWORD WINAPI ThreadProc( LPVOID lpParameter) {
if(CRYPTO_THREAD_write_lock(lock)) { // лочим ctx
BIO* bio = BIO_new_ssl_connect(ctx);
BIO_get_ssl(bio, &ssl); // сдается мне, что ssl должен быть свой для каждого connect, нет?
if (ssl) {
   SSL_set_mode(ssl, SSL_MODE_AUTO_RETRY);
   BIO_set_conn_hostname(bio, host);
   BIO_set_conn_port(bio, "443");
   SSL_set_tlsext_host_name(ssl, host);
   if (BIO_do_connect(bio) == ) { // коннектимся к хосту
      for(int i=0; i<5; ++i) { // для примера отправим последовательно 5 одинаковых запросов в одно соединение
         const char* buffer = "GET / HTTP/1.1\r\nHost: example.com\r\nConnection: keep-alive\r\nAccept: */*\r\n\r\n";
         int r = 0;
         do { // отправляем запрос
            r = BIO_write(bio, buffer, strlen(buffer));
            if (r < 0) {
               if (!BIO_should_retry(bio)) {
                  CRYPTO_THREAD_unlock(lock);
                  return EXIT_FAILURE;
                 }
              }
          } while (r > 0 || BIO_should_retry(bio));

         http_parser parser; // парсер chunked ответа

         int r = 0, rall = 0;
         char buffer[4096];
         do { // читаем ответ
            r = BIO_read(bio, buffer, sizeof(buffer));
            if (r > 0) {
               rall += parser.Parse(buffer, r);
               }
            else if (r < 0) {
               if (!BIO_should_retry(bio))
                  CRYPTO_THREAD_unlock(lock);
                  return EXIT_FAILURE;
                 }
               }
            } while ((r > 0 || BIO_should_retry(bio)) && !parser.IsError() && !parser.IsFinished());
         }
      }
   CRYPTO_THREAD_unlock(lock); // может можно разлочить сразу после connect?
   }
   BIO_ssl_shutdown(bio); // закрываем соединение
   SSL_free(bio);
   }
return EXIT_SUCCESS;
}

#define THREADS 10


int main() {
init(); // иннициализация библиотеки
for( int i=0; i<THREADS; i++ ) { // создаем 10 потоков
     hThreadArray[i] = CreateThread(NULL, 0, ThreadProc, NULL, 0, NULL);
     }
WaitForMultipleObjects(THREADS, hThreadArray, TRUE, INFINITE); // ждем-с
for(int i=0; i<THREADS; i++) { // закрываем
        CloseHandle(hThreadArray[i]);
        }
finish(); // освобождаем ресурсы библиотеки
return EXIT_SUCCESS;
}

Maybe there are some inaccuracies in the code - this is pseudocode.
PS I know that it is possible and better through non-blocking I / O, but it is necessary to do it in such an architecture.

Answer 1

https://github.com/facebook/proxygen
https://github.com/facebook/proxygen/tree/master/p...