S
S
so-olitary2016-07-31 01:38:10
Python
so-olitary, 2016-07-31 01:38:10

Python: key value database on-disk to store on same machine?

Hello to all!
It's very simple: I'm looking for a database accessible from python 2.7 on Windows.
To put into it (on disk), as in a bottomless bag of a pair:
a key (string)
and a value (a dictionary of strings and arrays of strings).
You need to place a 16-GB text file on your hard drive in a normal format on your machine,
so that later you can search and retrieve it by line, respectively. Everything should be as simple and reliable as possible.
In general, the story is this, I started with pickleDB (I realized that it was in-memory), I tried the standard shelve (too slow), unqlite db (it’s not clear whether it works or not), leveldb (not installed on windows) and CodernityDB ( it seems to work, fast BUT some kind of counter overflow and index settings are required ...) - it's not that.
I'm definitely doing something wrong... I'm tired.
It is unfortunate that key-value storage is called both server, and distributed, and small home databases.

Answer the question

In order to leave comments, you need to log in

3 answer(s)
S
sim3x, 2016-07-31
@so-olitary

If you couldn't build levedb for yourself, then use sqlite

A
Alexander Pitenin, 2016-07-31
@Sasha_Pitenin

You can use memcacheDB (there are builds for windows). If mamcacheDB doesn't work, there is also a built-in module in python dbm (I'm not responsible for performance :-) ).

A
Alexey Cheremisin, 2016-07-31
@leahch

Install elasticsearch. Load your file into it line by line, one document - one line, key - line number. And look for as much as you like.
And if you just need to run the entire file through a hundred or two algorithms, then load your search queries into the percolator and process the entire file at once!

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question