Why is SSH key authorization considered more secure than a password?

Question

SSH

The key file can be stolen from a computer, phone or flash drive. For example, I store some keys in my phone. Passwords, on the other hand, can only be stored in the head, and therefore it is more difficult to steal them.
Here I have a question - why is SSH authorization by key considered safer than a password?
brute force is very unlikely: ssh on a non-standard port + fail2ban

Answer 1

Because:

1. The probability of a successful key brute force is extremely small, unlike
   
2. Private keys must be stored in encrypted form, and preferably in specially designated hardware keys.
   
3. There are measures against thermorectal cryptanalysis in the form of SSH CA and, accordingly, the transformation of keys into a kind of SSL certificates
   

brute force is very unlikely: ssh on a non-standard port + fail2ban

Any RCE vulnerability in the application on the server will allow poking into the ssh server past fail2ban in the absence of isolation.