Data search based on a large number of criteria

Question

Database

I immediately remembered, approximately similar to my task:
Yandex.Market selection of mobile phones

Take a look at the panel on the right, you can choose a lot of any criteria and it works very quickly. How are things like this made? What approaches are used, the database device? Ideally, we need some kind of universal mechanism that could search by any arbitrary very large set of criteria, by a huge number of objects (> 100,000) with a different set of fields, do this as quickly as possible, avoiding over 9000 servers if possible.

Answer 1

Look towards Entity-attribute-value model .
Also a good answer on StackOverflow on the topic Product table, many kinds of product, each product has many parameters

Answer 2

In Java, it is very convenient to do Spring Roo using finders. Although, it is probably not the ideal tool for this task.
In what environment are you solving this problem?
I put myself RSS on this topic - I'm also very interested.

Answer 3

“There will be a database of products (several hundred thousand and growing), each product can contain an arbitrary set of data, you need to make a selection for queries with an arbitrary set of parameters”
IMHO - rigid typing of parameters into separate directories will enable quick selection and comparison, but you will pay with filling speed directory.
And in terms of architecture - a multi-level cache and good thinking over the structure of the database. If you are not afraid - noSQL databases with map / reduce or columnar databases. There will be a zoo, but (in theory) work faster. Or store properties in a classic relational and cunning materialization on something like Redis - search by parameters - a classic hash, if I understand the task correctly.

Answer 4

Normally the task is solved on RDBMS. You just need to describe not a food storage model, but a food storage model model. Those. a set of metadata and automatic creation of scripts based on them.
All the same, products (more precisely, new types of products) will be added to you less often than searched (as I understand it). So you can adjust the internal model automatically from time to time.
In such systems, by the way, not the search is the most difficult and important thing. There everything is optimized in understandable ways.
It is much more difficult to maintain the database itself in an up-to-date and complete form. How is this issue resolved?