What to learn, due to which the quality of the model increases?

Question

Machine learning

When a feature is added to a model (for example, a non-linear one), its quality increases. How to understand whether the quality has increased due to the added feature or due to an increase in the complexity of the model?
UPD: model complexity does not mean computational complexity. Rather, we are talking about the fact that a sign has been added and the trees (if used) have the opportunity to form more complex branching.

Answer 1

If the model has remained the same (for example, you have not changed the regularization, have not added a couple of new layers to the neural network), the complexity of the model has not changed. Feature importance
is used to evaluate features . Some libraries allow you to do this out of the box.

Answer 2

First you need to decide, but what do you understand by the term "model complexity"?
Because "difficulties", indeed, are different - at least "structural" and "computational". But the campaign, you mixed them up a little.
Let's take your Random Forest example - increasing the number of splits does not mean increasing the structural complexity of the model. In the same way as adding a new variable to a non-linear regression model of a given order does not lead to an increase in the structural complexity of the model. But a change - for example - from a quadratic model to a cubic one - leads to a change in both structural complexity and, as a result, computational complexity.
If you clearly imagine this, then it becomes clear that the introduction of a new feature within the framework of one model is an action, the result of which affects the "quality of the model" (by the way, it also needs to be defined, but let's assume that you mean "accuracy"). And the structure has nothing to do with it. And changing the structure of the model from quadratic to cubic can lead to a similar increase in the accuracy of the model even without adding new variables.