How to implement multi-threaded launch of programs in linux in C?

Question

linux

I am developing test software for testing hardware. And I have a task to run testing programs in parallel to each other. And not just run, but also process their output and test results.
I do it in the following way: I launch a separate testing thread, and already in it, using popen, I launch the tested thread. The problem is that the processes are executed sequentially, despite the multithreading. Let me explain with a code example. Error checking omitted.

static id=0; //счётчик запущенных процессов
int runner(void * ini_file_data_st_ptr, LPVOID sys_hwd_ptr)
{
....
  int result;
  pthread_t th_usbtest, th_ledtest, th_comtest, th_framtest, th_hddtest;

// test USB
  if (ifd ->usbCount>0) { //если у нас есть тестирование USB
    id++; //увеличиваем счётчик
    result = pthread_create(&th_usbtest, NULL, usbtest, &id); //запускаем поток "usbtest"
    printf("Creating the usbtest thread\n");
  } 
//test LED
  if (ifd ->ledsFlag) {
    id++;
    result = pthread_create(&th_ledtest, NULL, ledtest, &id); //запускаем поток "ledtest"
  }
//test COM
  if (ifd ->comCount>0) {
    id++;
    result = pthread_create(&th_comtest, NULL, comtest, &id); //запускаем поток "comtest"
  }
//... и так далее остальные процессы

  while (id!=0) { //костыли для ожидания окончания работы процессов, чтобы не возиться с мьютексами и семафорами
    sleep(1);
  }
}

All process functions (usbtest, ledtest, comtest, etc.) are about the same. Therefore, I will give an example of one of them.

#define USB_TEST_PATH "./usbtest"
void *usbtest (void *arg){
  FILE* f;
  ptrSysHWD->usbCount=0; //технические данные
  gl_tst_result.usb=true;
  char buf[128];
  char path[128];
  sprintf(path,"%s %d", USB_TEST_PATH, ifd ->usbCount); //формируем команду запуска программы с параметрами (передаём количество юсб)
  f = popen(path,"r"); //запускаем программу
  while (fgets(buf,127,f)) //читаем всё то что программа выводит
  {
// ********************* !!!
    printf("%s\n",buf); //выводим на экран
    fflush(stdout); // ФЛУШИМ!!!
// ********************* !!!
    if ((strstr(buf, "error") != NULL) || (strstr(buf, "not found") != NULL ) || (strstr(buf, "Error") != NULL) || (strstr(buf, "failed") != NULL)) {
        printf ("USB HARDWARE ERROR\n");
        gl_tst_result.usb=false;
    } //проверяем на ошибки
    if (strstr(buf, " OK!") != NULL) { //считаем количество устройств
      ptrSysHWD->usbCount++;
    }
  }
  fclose(f); //после того, как прочитали закрываем программу
  id--; //минусуем счётчик потока
}

The remaining threads are similar, they run other hardware tests. The trick is that some tests can run for several tens of seconds. And here is the paradox that despite fflush(stdout); I have a consistent output of all programs in the logs. Those. My understanding is that if the streams are running in parallel and the open file descriptors of standard output are the same, then they should have crossed output. And this is not.
From which we can conclude: threads and programs are executed SEQUENTIALLY ! Although the launch of each thread goes in parallel. What am I doing wrong? And how to fix it?

Answer 1

// crutches for waiting for the end of the processes, so as not to mess with mutexes and semaphores

It is enough to call pthread_join for each of the running threads.
Should be pclose(f).
In vain you so, with dirty hands in the general variable.
Non sequitur. But for the sake of completeness, I would like to see the Makefile.