N
N
Northxstar2019-12-29 19:43:42
Django
Northxstar, 2019-12-29 19:43:42

How to slice in Django ORM without all()?

The situation is simple to disgrace. The content of the table arrives on the page through pagination, but pagination cuts the queryset after the query to the database is executed, and the query looks like this:

fooset = barmodel.objects.all().order_by('-foobar')

Since there are about 10,000,000 rows in the table, the query takes an uncultured time. How to implement pagination at the time of the request, without using all(), what to write in the filter?
Now pagination is done like this:
current_page = Paginator(fooset,15)
page = request.GET.get('page')
try:
  context['foo'] = current_page.page(page)  
except PageNotAnInteger:
  context['foo'] = current_page.page(1)  
except EmptyPage:
  context['foo'] = current_page.page(current_page.num_pages)

Answer the question

In order to leave comments, you need to log in

1 answer(s)
D
Dmitry Sviridov, 2019-12-29
@Northxstar

If you think that in this case all records are pulled out of the database, and then some of them are cut off at the code level, then this is not so. all() does not query the database. The request together with LIMIT is already executed inside the paginator.
This is me to the fact that, most likely, you saw that the page with the paginator takes an insanely long time to load and assumed that all records were being raked out of the database, but this is not at all the case. It takes a long time to do this because on so many lines, standard pagination solutions will generally work poorly, because on "far" pages the database cursor will iterate over all the records up to this page. In addition, the paginator performs a count query, which takes a long time, because this is a fullscan of the entire table, where there are millions of records.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question