How to correctly and quickly transfer data from one database to another?

Question

Django

I'm doing the data transfer from the byza to the database, but I ran into a problem with the speed of the transfer. The fact is that the ForignKey fields are looked up each time for too long and with each new addition, because the object is not overwritten, but recreated.
Here is an example of the overflow function:

def load_d_merop(cursor):
    cursor.execute('SELECT * FROM d_merop')
    for row in cursor:
        obj = Merop()
        obj.id = row[0]
        obj.name = row[1]
        obj.idProj = Project.objects.get(id=row[2])
        if row[4]:
            obj.description = row[4]
        obj.save()

def LoadTables():
    conn = psycopg2.connect(dbname='vkmt', user='...',
                            password='...', host='localhost')
    cursor = conn.cursor()

    try:
        load_d_merop(cursor)
        print('merop: loaded')
    except:
        cursor.close()
        conn.close()

This is a small function for pouring the fields of a small object, but I have large objects with a lot of connections and they take about forty minutes to load. Maybe there is a way to transfer the base better than mine, if so, then I will be glad at least for an example, at least for documentation.

Answer 1

The main thing is to do it inside one transaction. Then the little things, for example, if Project and the like occur a bunch of times, then cache them in a dictionary so that each time it is not read from the database.
ZY if bases of one type, then through their dump.