D
D
Dmitry2014-01-23 20:05:29
PostgreSQL
Dmitry, 2014-01-23 20:05:29

How to save data about large objects in the database (somewhere around 120-200 fields)?

Greetings!
On a streaming project, there was a task to save data about large objects in the database (about 120-200 fields). This data will almost never be requested at the same time. Most likely they will be requested in portions of 20-40 fields. Also, it is known that the samples will rarely overlap, most often the samples will go in segments, for example, from the 1st to the 20th field, from the 21st to the 40th, etc., but situations may still arise when the samples will be cross.
Question:
What is the best way to represent such an object in the database schema?
I'm not a db-architect, so I can be wrong, but I see 2 possible options:
1. Create several tables according to the expected segments. Supposed advantages: better cache, the possibility of easy sharding compared to partitioning a monolithic table. Cons: overhead on key fields and joins, overhead on resolving at the application logic level, not very intuitive scheme and not very easy data control.
2. Create one big table. Pros: supposedly better cache, no overhead on joins and key fields. Cons - alleged problems with scaling.
The answer seems to be obvious, but I want to ask for advice specifically in terms of performance and experience of seniors) The system is large and complex, and we can afford to write code a little more difficult, but achieve better performance. Even a 5% increase will be valuable. Haven't done any tests yet. Decided to ask first.
Thank you and wish you all the best!

Answer the question

In order to leave comments, you need to log in

3 answer(s)
K
kompi, 2014-01-23
@hsc

Look towards materialized views (9.3)

D
Dmitry, 2014-01-24
@hsc

I'll supplement the answers with the information I found:
Postgres: many columns or several tables?

Z
zen, 2014-02-10
@zen

The fields on which you need to do something intensively must be stored separately, and the rest - in the hstore. hstore we came up with just for such tasks, when there is a lot of "trash" that is needed only for display. True, later, we added indexes and now you can also search in this trash :)

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question