I
I
Ilya2014-09-29 15:17:45
SQL
Ilya, 2014-09-29 15:17:45

How to organize the storage of a large number of records in the database?

The situation is the following. There is a database (SQLite, Windows) with one table with only three columns:
- Record ID (text, unique)
- Owner's email (normalized)
- Additional text (can be very voluminous)
Operations performed:
- Adding new records
- Checking for availability identifier in the table (to prevent duplicates)
- Selecting records by email
With a small amount of data, there were no problems. But now the database has grown to 15 gigabytes and everything is tossing and turning very slowly. There are no problems with recording, but requests for selection by email can take minutes. And each read request locks the table for writing, which is critical.
Questions:
- Is it possible to implement fast addition and parallel queries to the database while staying on SQLite or moving to another DBMS?
- How to speed up query execution time?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
A
Andrey Ezhgurov, 2014-09-29
@Gorily

It's easier to move to any normal SQL database (starting with MySQL) with indexes for the id and e-mail columns. There such requests will be fulfilled quickly.

S
Sergey, 2014-09-29
Protko @Fesor

www.sqlite.org/lockingv3.html
+ uniq indexes

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question