A
A
Andrey Grinevich2021-12-11 14:51:35
Python
Andrey Grinevich, 2021-12-11 14:51:35

How to have a common base class for dynamic instances?

Good afternoon!
Having fun with dynamic class creation, discovering the power of `types.new_class` and having a factory function like

def create_instance(cls_name, module,  args=(), bases=()):
        def clsexec(ns):
        ns["__module__"] = module
        if args:
            ns["__reduce__"] = lambda self: args

    cls = types.new_class(cls_name, bases, {}, clsexec)
    return cls()


The real example is a little richer, but an important nuance is that reduse is thrown here depending on the input parameters. Everything works until I do `pickle.dumps` of the classes created in this way, for example
class Sample:
     pass

main = [Sample(), Sample()]
pickle.dumps(main, protocol=2)

returns an adequate set of opcodes
0: \x80 PROTO      2
    2: ]    EMPTY_LIST
    3: q    BINPUT     0
    5: (    MARK
    6: c        GLOBAL     'source_module Sample'
   43: q        BINPUT     1
   45: )        EMPTY_TUPLE
   46: \x81     NEWOBJ
   47: q        BINPUT     2
   49: h        BINGET     1
   51: )        EMPTY_TUPLE
   52: \x81     NEWOBJ
   53: q        BINPUT     3
   55: e        APPENDS    (MARK at 5)
   56: .    STOP

and if you feed the pickle the same, but dynamically created classes
main = [create_instance("Sample", "source_code"), create_instance("Sample", "source_code")]
pickle.dumps(main, protocol=2

it will be like
0: \x80 PROTO      2
    2: ]    EMPTY_LIST
    3: q    BINPUT     0
    5: (    MARK
    6: c        GLOBAL     'source_module Sample'
   43: q        BINPUT     1
   45: )        EMPTY_TUPLE
   46: \x81     NEWOBJ
   47: q        BINPUT     2
   49: c        GLOBAL     'source_module Sample'
   86: q        BINPUT     3
   88: )        EMPTY_TUPLE
   89: \x81     NEWOBJ
   90: q        BINPUT     4
   92: e        APPENDS    (MARK at 5)
   93: .    STOP


A little debugging showed the way why this happens, namely:
- ` save ` is called on the first instance, then we fall into ` save_reduce `
- for example, for the second version of proto, we start saving the contents of the object and information about its classes / types
- for the first time we get under the memoization check , while it is empty there (since the first meeting with the class), then we analyze the class through dispatching and honestly remember.

- when the second instance arrives - we again reach the memoization of the class , but we fail because the dynamically created instance also dynamically created another class and I fail the ID check, the code is executed again=> once again `GLOBAL` is written in the pickle instead of ``BINGET`

The essence of the question is how to have a common class for identical entities from the same module / name so that memoization for the pickle works, but does not break the dynamic definition of attributes for instances?

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question