Accessing a global variable in multiprocessing?

Question

Python

Hello!
I wrote the code, initially sequential, without any parallelism.

def getInfo(url):
    response = requests.get(url)
    needToSend = true # В зависимости от логики описанной где-то выше ставим флаг
    if needToSend and url not in sentLinks: # Если надо отправить и еще не отправляли
        msg = ... # Формируем сообщение
        bot.send_message(12345678, msg) # Шлём в телеграм
        sentLinks.append(url) # И сохраняем урл чтобы на следующем круге не писать его в телеграм снова

Let's say there are 300 links. In a loop, we call our getInfo and pass one link.
We reach the end of the list and run a new script ( while true: getInfo )
It so happened that the number of links increased, and the script began to work very slowly.
I had to use multiprocessing Pool. I use 20 threads.
Faced the problem that each process has its own sentLinks . As a result, 20 identical messages from each thread arrive in telegrams.
Tell me, please, how to solve the problem?
Is it possible to use a global list variable in which we will write the url already sent to telegram?

Answer 1

Global variables are evil, especially in concurrent code. Just don't use them anywhere and ever. In your case, it's easy to replace the global variable with a queue .