M
M
Maxim2017-05-08 10:51:45
Django
Maxim, 2017-05-08 10:51:45

How to properly work with threads in python?

Greetings, I used streams a little, but I don’t have a final understanding of how to cope with them.
python3 + django 1.9
When the server starts, 1 main thread starts and 2 child threads from it, what it looks like

def scheduled():
    while True:
        # выполняется некоторая фильтрация по бд в итоге получаем список send, который передаем дочернему потоку
        t2 = threading.Thread(target=sending, args=[send])
        t2.start()
        t3 = threading.Thread(target=statistics)
        t3.start()

t = threading.Thread(target=scheduled)
t.start()

All this works, maybe the truth is written holivarno.
This whole thing is cooked, but when a new code pool happens or I manually restart the server, the calculations that are performed in the threads go astray and give out horrendous indicators, mainly this is the collection of statistics.
  1. Is it safe to stop threads from executing? So that the data that is being processed is correctly saved.
  2. I also considered other implementation options, put them into a separate process, and possibly into a separate mini-project that will collect statistics.

Purpose: To avoid incorrect calculations.
In general, what would you advise dear gurus?

Answer the question

In order to leave comments, you need to log in

3 answer(s)
M
Maxim, 2017-05-09
@maximkv25

Having studied in more detail the materials on flows, having listened to the advice, the problem was solved quite simply.
The script will be launched via manage.py, so you don’t have to rewrite the current functionality and with this launch a separate process will be created in the system, let’s run it in the background using surepvisord/systemd.

import threading
import time


def example(name ,event):
  i = 0
  while event.is_set():
    print('Thread: %s, %d', % (t1.name, i))
    i += 1
    time.sleep(10)


try:
  event = threading.Event()
  event.set()
  t1 = threading.Thread(target=example, args=('first', event))
  t1.start()
except KeyboardInterrupt:
  event.clear()
  t1.join()

To correctly complete this whole thing, let's write a script to throw an exception, after which the event will change the flag and the threads will be completed.

M
maxfox, 2017-05-08
@maxfox

I don't know what your task is, but it's better to use Celery or other similar tools.

E
Eugene, 2017-05-08
@Eugen_p

Here is a cool video about process threads from D. Beasley. Somewhere there was even a translation into Russian.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question