How to interact with custom python code?

Question

Django

Hello. Google didn't give a specific answer to my question.
I have a parse.py that parses a website and sends information to MySql.
1) How can I run this file through a normal html button?
I found a temporary solution, I submit the form (using the POST method), in the view

# view.py
import parse

def parse_site(request):
    if request.user.is_superuser: # Страница доступная только администратору
        if request.method == 'POST':
            parse.main() # Мой парсер
    
        return render(request, 'parse.html')

But immediately questions and problems arise:

• How to display some information on the page (not the result of parsing, but intermediate, for example, the response code from sent requests), which comes from parse.py
  
• How to stop the parsing process
  
• How to pass some arguments to parse.py
  
• After I submit the form, the page hangs (parsing takes place), but what if I still need to click somewhere on this page
  

I'm new and just don't understand how these things interact.

Please kick in the right direction, what to read, watch

Answer 1

1) How can I run this file through a normal html button?

When the button is clicked, the request is processed by the controller. Until the action is completed, the client waits, staring blankly at the screen.
Therefore, it is initially designed in such a way that the controller quickly transfers control to a third-party component with all user parameters and others necessary for processing, and itself returns a 200 or 202 response.
The third-party component provides a task queue manager. All arguments are put into the "packing" task, "packed" and marked in the database that task X was created at time T and it is in the pending status.
There can be several or quite a large number of task queue handlers (a process separate from the web application). Each such handler marks in the same database that task X is being processed and starts parsing with the received arguments. At the end, it marks what has been done, and saves the parsing results in the database.

How to display some information on the page (not the result of parsing, but intermediate, for example, the response code from sent requests), which comes from parse.py

This will be a complication for the system, but it is quite possible. After all, the response statuses will be received very quickly and they will need to be stored in the database, and the client must periodically poll the statuses or receive them via SSE. And sometime you will have to clear the database from old results.

How to stop the parsing process

The handler that started the parsing should determine by periodic access to the database that the task has been cancelled.

Answer 2

Celery