I
I
Ilya Myasin2017-10-12 08:01:27
CMS
Ilya Myasin, 2017-10-12 08:01:27

A layer of a web application that combines queries to the database - does it happen?

Hey! Background: I'm doing something strange now - I'm sawing a layer on the front that analyzes requests to the back, and if the data is already loaded, it takes local ones, and if it's partially loaded, it changes the request so that only the right one is loaded.
And in the morning I thought: maybe something like this happens for the server? Here we have a CMS or something like that, there are blocks that do not know anything about each other. You can also make requests for data from different blocks (widgets) not executed immediately, but first gathered in a heap, then somehow cunningly grouped, executed, and then the data was sent back to the requesting widget. More or less like this:

// было
class Widget {
  public function actionFoo() {
    $news = News::where('id', 1)->one();
    echo $news ? $news['title'] : "ох =(";
  }
  public function actionBar() {
    $news = News::where('id', 2)->one();
    echo $news ? $news['title'] : "все плохо =(";
  }
}

// стало
class Widget {
  public function dataFoo() {
    return [ News::where('id', 1) ]
  }
  public function actionFoo($news) { 
    echo $news ? $news['title'] : "ох =(";
  }
  
  public function dataBar() {
    return [ News::where('id', 2) ]
  }
  public function actionBar($news) {
    echo $news ? $news['title'] : "все плохо =(";
  }
}

Accordingly, for this example, the magic layer should execute a query like: and return the result, split into two parts. Has anyone thought in this direction? Or will there not be much benefit? Or is it difficult to write logic? Or the client code will turn out to be inconvenient?
select * from news where id in (1, 2)

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Alexey Ukolov, 2017-10-12
@dubr

The biggest problem with this system is dependent data. If in widget 1 query A that works with table X depends on the data from query B to table Y (it takes the entity id from there, for example), then query B must be sent immediately, without aggregating it with query C from widget 2 that works with the same data, but sent later. Most requests depend on each other, so this thing will not bring tangible benefits.
Plus, in most projects, backends are synchronous - the execution of all code stops before the end of the execution of the current function, so it is not so easy to implement some kind of asynchronous aggregator, most likely, you will have to rewrite the entire application altogether.
Plus, how does such a system even know that all requests have arrived, and it's time to aggregate and execute them?
Plus, widgets that work with the same data are not often found - the system will idle.
Plus, automatically collected queries can be very suboptimal, which will only worsen performance.
Plus, if queries for the same data are generated with a different structure, the DBMS will not be able to cache them efficiently.
On the front-end, the paradigm is different and the requests are simpler (they are easy to distinguish by URL), so it’s easier to make such a thing and less likely to shoot yourself in the foot. Well, the delays there are much more serious - going over the network to the backend and going to the database on the backend is like flying to Alpha Centauri and going to the other end of the city, so if you have the opportunity not to fly, you should use it.
On the backend, such a task (optimization) is more efficiently solved with the help of a cache, moving part of the data to lightweight storage and the good old best practices for working with the database: the correct schema, correctly composed queries, indexes on frequently used columns.
But here is a real case where I implemented and use a similar system: there is a C ++ program that constantly makes requests to an external API. This external API sets a limit on the number of requests per second, which the program does not always fit. There are two ways out: set delays before requests or aggregate them into packages (the API supports this). The second solution is obviously better in terms of speed. But I did not implement the analysis and merging of similar queries in it - it's hard to do, easy to mess up, and the profit will be relatively small. This whole thing was done only due to the fact that all operations are asynchronous and requests are executed through the scheduler. On some standard php project, this cannot be achieved.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question