S
S
SemenDemon2019-01-14 12:57:42
MySQL
SemenDemon, 2019-01-14 12:57:42

What to sacrifice when sharding?

mysql property table.

+------------+------------+-----+-------------+
| real_id    | date       | ... | locality_id |
+------------+------------+-----+-------------+
| 1          | 12.12.2012 | ... | 2           |
| 2          | 12.12.2013 | ... | 2           |
| ...        | ...        | ... | ...         |
| 23.000.000 | 12.12.2018 | ... | 5           |
+------------+------------+-----+-------------+

table with photo
+-------------+-----------+---------+
|   foto_id   |    url    | real_id |
+-------------+-----------+---------+
|           1 | 1.jpg     |       1 |
|           2 | 2.jpg     |       1 |
|         ... | ...       |     ... |
| 200.000.000 | 98989.jpg |   55555 |
+-------------+-----------+---------+

I want to cut the table with the photo into shards of 5 million. but then a simple possibility would be lost. Wouldn't it hurt if you store links to photos via implode in a column?
select * from foto_table where real_id = 2;
+------------+------------+-----+-------------+---------+
| real_id    | date       | ... | locality_id |  foto   |
+------------+------------+-----+-------------+---------+
| 1          | 12.12.2012 | ... | 2           | 1_2     |
| 2          | 12.12.2013 | ... | 2           | 7_4_3   |
| ...        | ...        | ... | ...         | ..._... |
| 23.000.000 | 12.12.2018 | ... | 5           | 5_6_3   |
+------------+------------+-----+-------------+---------+

and bring the table with the photo to mind
+-------------+-----------+
|   foto_id   |    url    |
+-------------+-----------+
|           1 | 1.jpg     |
|           2 | 2.jpg     |
|         ... | ...       |
| 200.000.000 | 98989.jpg |
+-------------+-----------+

search still runs on Elasticsearch. The object table will be cut by locality_id. Those. try to store all data in a record, so as not to search for them by shards.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
A
Andrey, 2019-01-14
@VladimirAndreev

Excuse me, how many entries do you have in the photo table, what do you want to shard them?
and by the way, would it be cheaper to store a bunch of real_id: [{"id": foto_id, "url": url}] in memcached / mongodb?

A
Andrey Shatokhin, 2019-01-14
@Sovigod

Cut objects as you like. And put the pictures in the same shard where the object is.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question