K
K
kawashirov2017-03-01 21:21:11
Python
kawashirov, 2017-03-01 21:21:11

How to make "long" events in Python Tornado?

I'll explain with an example. I am writing a TCP proxy server for one game, it not only processes packets, but also sends various requests on behalf of the game to collect statistics, sometimes it answers itself instead of the game. The bottom line is that it all grew so much that I decided to make a simple plugin system and break all the message processing logic into many plugins.
Then a typical plugin, if simplified, starts to look like this:

class SimplePlugin:
  @coroutine
  def on_request_from_client(self, event):
    чё-то делаем с запросом
  @coroutine
  def on_response_from_server(self, event):
    чё-то делаем с ответом

The problem is that the context is "split" between two methods, and if the plugin needs to process a specific response to a specific request, it has to save everything in self and then consider each response and determine which request it came from, restore the context. It's inconvenient as hell. Not the pythonic way at all. I would like to see something like this:
class SimplePlugin:
  @coroutine
  def on_request_from_client(self, event):
    чё-то делаем с запросом
    yield event.wait_for_response()
    чё-то делаем с ответом

Those. The plugin "engine" fires a request event to the plugins, one at a time. If "I want to wait for an answer" is thrown out, then it goes to the next plugin, accumulating all the "Wishlist". Further, when request processing is completed, it sends a request to the server, catches the response, and then begins to return the response one by one to the coroutines with the Wishlist so that they continue processing the response. In theory, such a loop can be extended indefinitely like event.want_that(), event.want_this()
Knowledgeable people may notice that you can do this:
class SimplePlugin:
  @coroutine
  def on_request_from_client(self, event):
    чё-то делаем с запросом
    def callback():
      чё-то делаем с ответом
    IOLoop.add_future(event.response_future, callback)

But there are a couple of problems here. First, right after the "engine" receives a response and resolves event.response_future, then all callbacks will run "in parallel". Well, i.e. they will be executed, of course, sequentially, but if one callback does something heavy in terms of I / O, it is interrupted and the next one starts to be executed. Those. The first one has not yet finished, and the second has already begun. So you need to cover yourself with Locks, etc. It's not easy anymore. And secondly, again, this is not quite a pythonic way, and certainly not a tornado way.
So far, I'm studying tornado sorts and thinking about how to make my @coroutine implementation for this case.

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question