What happens when the RAM is full?

Question

linux

Let's imagine a situation: some program started to load more RAM than it is physically available. Any reason - they forgot to free the memory, we load the entire multimedia, and perhaps there is really so much data and this is normal for certain mega-calculations.
Question: what happens in this case?
Primarily interested in Linux, but I would also like to know how things are on other operating systems.

Answer 1

Primarily interested in Linux, but I would also like to know how things are on other operating systems.

Your own (swap) will start to fill up, in other OS (windows) about the same system, about "completely different OS" - I don't know.
And in Linux and in Windows and in other operating systems for which there are C compilers, there is a function, if you look at the description, for example, of such a function as malloc , then we will see from there that:

If the function could not allocate the requested block of memory, a null pointer is returned.

That is, the program itself must control whether memory is allocated to it or not. In high-level languages, this process happens in a similar way, it's just that the memory allocation function is usually "hidden" from the programmer's eyes and this process happens automatically, but the result in theory should be the same.
You can read a little about SWAP here.
Some programs, such as Photoshop for example, knowing in advance that there will not be enough RAM for sure, have their own "swap" mechanisms, i.e. creating some kind of "paging" on the disk. For example, when opening one template (layout), Photoshop can easily "bite off" 100GB from the disk, and if the history of changes is also large, it can be more.
Separately, I want to draw your attention to the fact that in Linux there is a certain mechanism of "self-preservation" in the kernel, which, under certain conditions, begins to "kill" processes that are out of control. At what point in time this happens - unfortunately, I can’t say for sure, because. still in the process of learning such mechanisms, but I suspect that "out of control" is when a process and its descendants (forks) begin to consume an excessive amount of processor time and possibly memory. I observed such a picture during a DDoS attack on the server, at that moment Linux began to "crush" everything in a row, incl. and the web server got under distribution. Among other things, I remember that somewhere in the documentation I read that the process can be set resp. "flag" or "priority" that affects this mechanism.
* Once again, I draw your attention to the paragraph above - I did not study this mechanism in detail and did not find information in the "quick" documentation, therefore, in the last paragraph - the information may not be entirely accurate.

Answer 2

Specifically for Linux:
1) The program issues a request to allocate additional memory
2) The operating system checks if it can allocate this memory.
3) If memory is available, it is allocated and a positive response is returned to the program.
The subtlety at this stage: in fact, there may not be enough memory. Physically, the memory is not engaged yet.
4) The program writes data to the allocated memory. It is at this stage that free memory can run out.
5) The operating system handles the OOM state (out of memory). To begin with, it tries to free memory by such "bloodless" actions as reducing the size of the file cache, I / O buffers, pushing inactive pages to the swap partition / file, etc.
6) If it was not possible to release the memory "bloodlessly", the OOM Killer is called - the "judge", which forcibly terminates one of the running processes (guided by several criteria) until the OOM state is overcome. The program that caused the OMM state has a higher chance of being targeted than others.
Read more:
https://habrahabr.ru/company/yandex/blog/250753/
https://www.insight-it.ru/linux/2015/chto-stoit-zn...

Answer 3

Will load data into swap. If it also ends, the program will issue something like "can not allocate memory"