V
V
Vitaly2016-05-26 18:37:24
PostgreSQL
Vitaly, 2016-05-26 18:37:24

Background task, large table, CURSOR and update, is it worth opening 2 sessions?

Hello.
There is a php CLI utility that iterates over the entire table (potentially unlimited in size) and, under certain conditions, updates some rows. The bypass is slow, the speed does not matter, the frequency of the bypass is "when the administrator needs it."
Decided to do such pass through cursors. Since the cursor works within a transaction (WITHOUT HOLD), then all updates until the end of the table traversal are accumulated in this transaction and are applied only after the commit.
Here I see two variants:
1) Let updates'y accumulate in transactions. But are postgres transactions designed for large amounts of data? Presumably, there may be thousands of updates, and later, under certain conditions, tens of thousands.
2) Run 2 sessions, create a cursor through one, and do updates through the second (because there is no need to see update'a within the cursor transaction)
Actually the question is: is it worth worrying about the size of the transaction buffer in this situation and open 2 sessions to base or I in vain take a steam bath from scratch?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Alexey Cheremisin, 2016-05-26
@leahch

Actually, it's a waste of time. But, there are small buts. I don’t remember how in postgres, but by analogy with versioned databases (and postgres is just that, like oracle), the records within the transaction are copied with the new version, and upon completion of the transaction they simply remain in place. Here is a rollback of a transaction, especially a large one, can be very painful.
In general, you can rest against only two points, lack of memory, and lack of disk. When it comes down to it, you can select records in pieces of 1000 or more pieces. Process them and commit, then select the next piece. This will make it easier for the base and for tracking.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question