Answer the question
In order to leave comments, you need to log in
Is there a limit on the length of the SQL query (big IN)?
Greetings!
There is a big table on 30 columns and 5 million lines. It is necessary to go through each line, and perform additional data processing. I decided to select rows in batches of 100 thousand rows at a time.
I do it in two passes:
1. First, I select 100-300 thousand id in a cycle
2. By the second request, I pull out the remaining columns by these id:
Is it correct to list 100 thousand id in the IN block? And 300 thousand?
Is there a request length limit? How to achieve maximum execution speed?
I would be grateful if you share your experience. Thank you. SELECT id FROM tbl LIMIT X, 100000
SELECT * FROM tbl WHERE id IN (...)
Answer the question
In order to leave comments, you need to log in
And what is the sacred meaning of choosing first IDs with LIMIT OFFSET, and then making a second request through IN?
Why is it normal not to select full data through the limit?
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question