What technology stack is used for frequent polling of large databases?

Question

Highload

It became interesting how such tasks are implemented:
There is a relational database with millions of records in it.
You need to provide very fast access to these records, given that the queries will be very frequent and complex (filter on many fields).
Let's say a couple of thousand people on the web interface click on buttons that form a request filter and immediately receive some kind of response.
Do relational databases like Oracle, PostgreSql cope with this task?
Or is my question not correct and is it necessary to talk about the whole structure of the server / server language / something else?

Answer 1

Here you need to talk about the type of information and the type of requests. Well, also about the architecture of the entire system.
In other words, with one organization the bases will not cope, with another they will.
For example, there is a big table with full name and passports and address of residence (city, house, apartment), there are indexes for full name, passport number and address. Such a database will work fine when looking for specific records, but if we want to make a selection by city, we may fail due to the large number of records in the table.
Accordingly, you can try to optimize - break the table into cities. Each city has a separate table - then the selection by full name and passport number will have to be done by a query to many tables at once, but the selection by city will be much faster.
You can still optimize - create a separate table by full name and passport indicating city tables, or you can separately make tables for each letter of the alphabet.
And so on.. This is called the normalization of information. But it all depends on the types of requests.
It is impossible, alas, to make a database for all queries at once ... But there are OLAP cubes for this, but this is a separate song ...
And yes, both Oracle and Cybase and Rostgres can both cope and fail - how the data will be organized , and how queries will be built to them. All these databases normally work with millions of records in tables, the only question is how we will operate with these tables ...
As for the architecture: very often you need to cache some information outside the databases - for example, on intermediate servers, 3tier or multitier application architecture en.wikipedia.org/wiki/Multitier_architecture (and the garbage is written in the Russian version of this article)