How in Go to asynchronously execute incoming tasks and display the final result?

Question

go

There are N similar tasks. You need to execute them asynchronously and write the results.
To complete a task, I used to run N goroutines with different parameters for each task, and add the results to MAP
. I waited for all functions to complete using sync.WaitGroup and then generated a response.
Now there is a need to supplement the list of N tasks directly from an already running function.
For example, 10 goroutines were launched, and inside one goroutine there was a need to perform 2 more tasks. (In which goroutine, how many new tasks will need to be added and whether it will be necessary to add anything at all - we cannot know in advance) That is, the total number becomes 12, and these two additional tasks should somehow immediately start execution without waiting for the end initially running 10 tasks.
And it is already necessary to wait for the completion of all 12 tasks.
Also, each of the two new tasks in progress may also want to start M tasks.
And in the end, if each task has completed and has not launched any new tasks, stop waiting for the tasks to complete and respond to the user.
Is it possible?
Is it possible to implement this using the same sync.WaitGroup, or is it necessary to already connect channels?
What channels to use - with or without a buffer? If with a buffer, then with what, if the number of tasks is unknown.
I tried to make a channel with a large buffer. Sent the required information to it.
In another subroutine, I do a range over the channel - I launch tasks asynchronously, and inside some tasks I send a message about a new task to the channel, but how can I track in this case that they have already been completed and no new tasks need to be performed?
And maybe there is a ready-made solution on how to implement this.
I'm waiting for tips.

Answer 1

I would do so.
1. There is a channel where all tasks go.
2. There is a goroutine that cleans up the channel and for each message from the channel launches a goroutine to process the message.
3. If necessary, this goroutine puts messages in the same channel.
4. When starting the handler goroutine, add to the wait group and remember to 'defer wg.Done()'
Something like this: https://play.golang.org/p/z0fS5O8ynSI
Not sure if this is good code, but should show the general idea.
Buffer size - the maximum possible number of tasks that can come as subtasks (taking into account nesting, if a subtask can have its own subtasks + 1 (for the current task)

Answer 2

It looks like you here: https://gobyexample.com/worker-pools
Well, that is, there is a classic example for a known number of tasks, but you can have your own task counter (you just need to use one of the asynchronous data types).
It is better to add new jobs in the body of the jobs themselves, which generate new ones.