W
W
whoami?root root_toor2018-11-19 17:39:03
PostgreSQL
whoami?root root_toor, 2018-11-19 17:39:03

Which distributed database to choose for a project on Spring Boot 2?

It is necessary to adapt an existing project on Spring Boot 2 with Spring Data JPA 2 (PostgreSQL) to the new architecture:
Jobs with pre-installed and configured software (one and the same program) are flexibly added to the system.
There are no more than a few dozen jobs. About 1 million records (of various entities in terms of JPA) enter the distributed database from each workplace per day. The database must store the bulk of the data on disk and can be up to several TB. The IMDG (in memory data grid) cache for each workstation must be able to request up-to-date information from each node (cluster) of the system. Perhaps the timeout causes the data to be flushed to disk. The database itself must be backed up on disk so that it can be deployed in the event of a system failure.
Should be compatible with Spring Data 2/JPA/Spring boot 2. It is supposed to work on Windows/Linux.
From the general mass of distributed databases, I chose Redis, Apache Ignite and Hazelcast as the most popular and easy to learn.
But, as far as I understand, only Ignite has the ability to save data to disk (Ignite Persistence) and fits the above requirements. I tried to add saving entities to Redis - it works, but only as an in-memory cache. I also need the data to be read from a persistent database (on disk) and, perhaps, some amount was stored in RAM (for frequently used elements, or for recently added ones).
1) Is it possible to use Redis with a persistence provider like PostgreSQL? Or there are some other ways to make the radish work as described (dump the accumulated data from the RAM to the disk).
I also tried using Ignite, but the latest version at the moment is 2.6, which uses Spring Data 1, and my project is running on Spring Data 2, support for which will be added in version 2.7.
I have downloaded nightly build 2.7 but I don't know how to install it in my local maven repository.
2) Does anyone know how to slip the downloaded Ignite into the project?
And finally, is the optimal way to organize the interaction of system nodes chosen? What system should be used in this case? How to set up a distributed database?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
I
Igor Aleksandrovich, 2018-11-29
@1q2w1q2w

Why do you need a distributed database for 10 computers and with such a small load?
Redis can persist on HDD just like Ignite https://redis.io/topics/persistence this is decided by the settings.
But Ignite will warm up the caches when the application starts, and this can take a long time when the application starts. At the same time, Ignite makes backups on neighboring nodes for fault tolerance and all this is stored in RAM.
Imagine what kind of infrastructure you need to maintain and how difficult it is to profile, catch errors, etc. compared to a relational DBMS.
I'm already silent about the consistency of data in caches, see how difficult it is to do simple things:
https://www.youtube.com/watch?v=YQGmJt_UP0E
That being said, why bind to JPA. We need to see how often it is supported for ignite. And it won't die completely. Ignite has a pretty good Java API.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question