Django & PostgreSQL MemoryError + script not working properly?

Question

PostgreSQL

I'll start by explaining the architecture of the project. There is a bot on aiogram through which orders are created. Orders consist of lines (items), prices and order IDs.

Task:
Process all unpaid orders with the script and if the order has not been paid for more than N minutes, then the product rows are returned to the product table, and the order is deleted.

My implementation:

while True:
            users = MyUser.objects.all().exclude(bot_username='null')
            now = timezone.now()
            for admin_user_obj in users:
                for order in User_purchase.objects.raw('SELECT id, time_created, chat_id, balance_to_return FROM user_purchases WHERE is_paid=FALSE and belongs=%s LIMIT 5', [admin_user_obj.id]):
                    if order.time_created + timezone.timedelta(minutes=admin_user_obj.booking_time) < now:
                        try:
                            bot_user = Bot_user.objects.get(belongs=admin_user_obj.id, chat_id=order.chat_id)
                            if order.balance_to_return:
                                bot_user.balance += order.balance_to_return
                            bot_user.active_orders -= 1
                            bot_user.save()
                            send_order_canceled(order, admin_user_obj)
                            for z in User_purchase.objects.raw('SELECT id, strings, item FROM user_purchases WHERE id=%s', [order.id]):
                                add_strings(z.item, z.strings)
                        except:
                            pass
                        order.delete()

SQL queries because otherwise MemoryErrors occur.

After running and monitoring the logs, the script starts spamming me with a product line:


And the client told me that his 1 product line turned into 23 million, and several lines in the armor turned into 7 million lines. To say that I was surprised is an understatement.

models.py

class User_purchase(models.Model):
    class Meta:
        db_table = 'user_purchases'
        verbose_name = 'Платеж пользователя в боте'
        verbose_name_plural = 'Платежи пользователей в боте'

    chat_id = models.IntegerField()
    belongs = models.IntegerField()
    title = models.TextField()
    strings = models.TextField()
    amount = models.DecimalField(
        decimal_places=2,
        max_digits=10,
        default=0
    )
    balance_to_return = models.DecimalField(
        decimal_places=2,
        max_digits=10,
        default=0)
    note = models.TextField(null=True, blank=True)
    coupon = models.IntegerField(null=True, blank=True)
    is_paid = models.BooleanField(null=True, blank=True, default=True)
    item = models.IntegerField(null=True, blank=True)
    time_created = models.DateTimeField(default=timezone.now)
    time_of_commission = models.DateTimeField(default=timezone.now, null=True, blank=True)

docker-compose

db:
    image: postgres:12.0-alpine
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    env_file:
      - ./.env.prod.db
    command: >
      -c work_mem=512MB
      -c maintenance_work_mem=256MB
      -c max_wal_size=1GB

  premium_daemon:
    build: ./app
    command: python manage.py premium_daemon
    volumes:
      - ./app/:/usr/src/app/
    env_file:
      - ./.env.prod
    depends_on:
      - db

Answer 1

I liked the implementation
try:
All the logic
except:
pass
If this homosexuality is removed, then I could understand where I made a mistake in the logic, and why be surprised?

Answer 2

As it was already correctly noted here, it is not necessary to catch all Exception indiscriminately and not even display any error. Nothing good will come of this anti-pattern.
At the very least, you should start with:

try:
  ...
except Exception as e:
  print (e)

Now, maybe an Exception is happening, but we don't even know it.
Next, don't mix ORM with regular queries. This is also a source of potential problems, because the ORM can cache data, including data that has not yet been saved. It is possible that an Exception occurred before the save call, the data in the cache differs from the data in the database, but with a direct request to the database, we will get something different from what we would get from the cache.
These are the little things that can create problems, which at the same time are not even reflected anywhere. And it's lucky that the effect is so noticeable. And if the effect was manifested in one case out of a thousand? How to catch it later and what to answer to users from whom extra money was debited?

Answer 3

T - testing. D - debugging.