A
A
Andrey Dugin2015-03-14 15:07:32
Python
Andrey Dugin, 2015-03-14 15:07:32

How to invert a dictionary without using additional memory?

I am writing a memory and performance critical application. For some repetitive calculations, I decorated the functions using the memoization technique. I store large repeatedly repeated objects in the following cache:

from collections import defaultdict
from itertools import count
cache = defaultdict(count().next)
Thus, each object added to the cache receives a unique sequential index, i.e. In a dictionary, not only the keys are unique, but also the values. After the calculations are completed, it is required to restore objects from the cache by index, for which it is necessary to invert the dictionary. It takes hundreds of megabytes, so I'm looking for a way to invert cache -> xcache in such a way that would not double the amount of data in memory. I want to remove an element from one dictionary and immediately add it to another, so that the data flows smoothly. In other words, two dictionaries with a full set of keys and values ​​in each should not be in memory at the same time. The main problem that comes up is iterating over a changing dictionary. How to implement it correctly?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Artem Klimenko, 2015-03-14
@adugin

> It takes hundreds of megabytes
, do you mean that the data stored in the dictionary takes hundreds of megabytes, or the dictionary object itself takes hundreds of megabytes?
Since there will be no copies of the keys and values ​​in memory from the fact that you invert the dictionary, the memory drag will only be on the very structure of the new dictionary.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question