Why does filepath.Walk hang the whole script tightly?

Question

go

Hello, there is a problematic script
Link to the playground
in brief what it should do
1. Main is launched
2. A separate convert goroutine is created that sits and listens to the channel to take objects from there and pass it to the runconvert function if there is an object, then increase counter by 1, if the counter is less 5 if we don't do anything else
3. run in an endless loop bypassing the directory and all subdirectories that were specified at startup as the first argument.
4. The filepath.Walk function calls walkfunc on each found file
5. Walkfunc, in turn, creates an object and pushes it into the channel where it is already waiting.
6. the runconvert function "works out" and reduces counter by 1 and atk in a circle
everything goes according to plan for 5-6 iterations and then hangs tightly, while if we replace filepath.Walk with a simple counter, let’s say from 1 to 1000, we create an object and shove it into the channel, then everything works out with a bang. How can I fix all this?
UPD: The counter is needed in order for the runConvert s to be NO MORE THAN 5 at one time, because if the gorutinf bcdt is enough to get everything that gets into the channel, they will soon spawn their runconverts by the number of files, and as it is written in the source code "imitation of violent activity" activity Naturally stormy and if you cry uncontrollably, then the server hangs itself until it rakes everything, and if it's 10,000,000 files, I'll lose it for more than a week nafig)

Answer 1

You are changing the global variable counter from multiple goroutines at the same time, this is not thread safe.
Changing the counter to +1 or -1 is actually quite a few steps.
1. Read the value.
2. Perform an increment or decrement operation with it
. 3. Write the value.
Now imagine that two goroutines are simultaneously trying to change the value of the counter. The counter is now 5 and the goroutines are trying to decrease it by one. They both read 5 to themselves, decremented it by 1, got 4, and both wrote down 4. If working correctly, we should get 3, since we completed two runConverts, and received 4.
Plus, you will have a loop on for {} with counter>=5, which will push the other goroutines out of the scheduler.
Why do you need this counter at all, if there is a wonderful thing - channels!
pastebin.com/714YTT8U
You just need to run 5 goroutines through the go converter () and with the help of channels, tasks will be perfectly distributed between them.

Answer 2

Most likely not Walk is hung up, and there is a deadlock of the channel.
For example, your counter changes in different goroutines. In these cases, locks must be used before modification with mutex.Lock and Unlock. And so at some point in time you have accumulated counter> 10,
runConvert does not clear the channel anymore, and listFunc cannot write anything there.
And in general, why enter counter?
Just
for {
go runConvert(<-sourceFileChan)
}
should work.