How to build a neural network?

Question

Neural networks

Hello.
I need to understand how neural networks work for my thesis.
After reading several articles and part of the book, I could not understand how they are built.
The general principle of operation is clear: a set of input data that, through weight coefficients, enters all neurons of the neural layer at once. In them, the signals are added, then converted by the activation function and go to the input of the neurons of the next neuronal layer. In the end, all this activates some kind of output neuron (s), which will be the result of the work.
But here's how many neurons should be, how many layers should be, whether each of the neurons carries some kind of semantic load (checks a specific parameter based on the input data) or everything is random. What to rely on when selecting all this is not clear!
Please help me figure this out using a simple problem as an example.
Suppose I want to write a neural network for adding/subtracting numbers from 0 to 9.
In this case (as I understand it) I will have three input neurons (the first number, plus or minus sign and the second number) and 28 output neurons (numbers from -9 to 18).
But what should be in the middle?
How many neurons, how many layers, by what principle should all this be selected?

Answer 1

There is a first approximation, the complexity of the task almost linearly affects the number of neurons, but how exactly to evaluate this complexity numerically depends on the task.
There is an assumption that each subsequent layer (for deep learning) is actually a separate task of converting data into another form, selecting criteria, internal classification, etc., and the number of neurons in a layer is the complexity of this particular transformation.
If the dependence between inputs and outputs has a complex form (not just if-then), then each extra layer defines exactly this dependence, but they all work for all conditions in the dependencies at once...
And in general, neurons in a neural network are like a hologram, they all simultaneously affect the result. I dug superficially in the available information, basically everyone walks around studying a specific task.
Another direction of studying the complexity of a neural network is its modification. First, you make the network complex enough in your opinion that it describes your subject area, and then reduce it by removing individual weights and neurons (re-training is necessary after each modification).
It is quite possible to play with the input training data, they can be grouped, separate neural networks can be allocated for their grouping, and different networks can be trained for different groups.
And another feature of recent years is the use of tensors in order to train not the neural network itself, but its simplified version (tensor, as I understand it, we convert the weights of the neural network into a simplified tree, train based on it and then bring the result back to the network ) - this allows you to reduce the resource requirements for training by an order of magnitude. Unfortunately, there are no beautiful examples in the Russian-speaking segment of the Internet, and even in English ... only a couple of open libraries have a ready-made implementation for working with tensors, tensorflow from Google and theano

Answer 2

How to build a neural network?
After reading several articles and part of the book, I could not understand how they are built.

Read the articles again and finish the book.