Multiprocessing django?

Question

Django

I have a loop like this:

name = 'something'
for hash, color in hashes:
        ColorHashes.objects.create(name = name , hash = hash, color = color )

The trouble is that I have about 120 thousand. objects need to be created and the process is still very slow, I want to try to make multithreading, how can this be implemented

Answer 1

Try bulk_create

CHUNK_SIZE = 10000  # Подберите наиболее подходящий размер экспериментальным образом

def chunkify(items, chunk_size):
    for i in range(0, len(items), chunk_size):
        yield items[i:i+chunk_size]


name = 'something'
for chunk in chunkify(hashes, CHUNK_SIZE):
    color_hashes = [ColorHashes(name=name , hash=hash, color=color) for hash, color in chunk]
    ColorHashes.objects.bulk_create(color_hashes)

Answer 2

No
You need to disable autocommit in this block of code and let's make a commit in the code after 10k positions
https://docs.djangoproject.com/en/2.2/topics/db/tr...
Or even insert directly through the DBMS console