In this connection, flash drives behave very strangely?

Question

linux

Hello!

Kernel: 4.10.0-32-generic x86_64 (64 bit)
Desktop: MATE 1.18.0  Distro: Linux Mint 18.2 Sonya

After installing the new kernel 4.10.0-32, some oddities appeared during operations with flash drives (two flash drives of average wear); although when switching to the previous kernel, the situation does not change. Flash drives are formatted in FAT32 in Windows.
Firstly, when deleting files from a flash drive (in 99% of cases - video files), the files are deleted very quickly, and, after removing / inserting the flash drive, they are returned back to the flash drive. There was also a problem similar to the infamous bug #12309 - when transferring files to a USB flash drive, the computer starts to slow down wildly, the cursor moves abruptly, etc.
For example, today, having made 3-4 attempts, the files were still deleted. I started uploading a 1.5 giga file to a USB flash drive, the computer started to slow down, in the end everything was thrown off. Then he threw off several files of 500 mb without any problems. Then, I removed and inserted the USB flash drive back, and it stopped being read, it refers to the Input / Output error.
I remember that a month ago I dumped about 6 GB of data of various types on a USB flash drive (another one); no brakes or other problems were found.
Please tell me what it could be?
PS There are a few more oddities in data transfer. Dropping, for example, 5 files in 575 MB, the progress in the status bar quickly reaches 570 MB, hangs for a while, lags, then again quickly reaches 1140 MB (as if having already downloaded two files), freezes, etc. .d. Also, file transfer operations are canceled for a long time - it may take a minute or two after pressing the corresponding button.

Answer 1

There was also a problem similar to the infamous bug #12309 - when transferring files to a USB flash drive, the computer starts to slow down wildly, the cursor moves abruptly, etc.
For example, today, having made 3-4 attempts, the files were still deleted. I started uploading a 1.5 giga file to a USB flash drive, the computer started to slow down, in the end everything was thrown off.

It's the freakiness of the Linux I/O system. When a program writes any data to disk, in fact they are written to memory, and direct writing occurs when this buffer reaches the size specified by the sysctl vm.dirty_ratio parameter, which is set as a percentage of the amount of available (MemFree + Cached) RAM( may actually be earlier, but more on that below). On modern systems, it sometimes turns out to be a hell of a lot (in your case, ≈ 570 MB) and at the time of recording, the device starts to go nuts from a suddenly fallen data stream for recording (how much is the write speed of your flash drive? 15-30 Mb / s at best?), and in general, all processes using any disk are tightly blocked until the buffer is freed. If there are many such processes, then all processor cores hang in iowait and the system starts to lag noticeably.
To somehow live with this garbage, you need to adjust the size of the buffers in such a way that the flush process does not cause blocking for a long time. For example:
sysctl vm.dirty_bytes=10485760
sysctl vm.dirty_background_bytes=2621440
vm.dirty_bytes - similar to vm.dirty_ratio, but in bytes, not percentage of a spherical horse in a vacuum. If vm.dirty_bytes is set, then vm.dirty_ratio is set to zero and ignored.
vm.dirty_background_bytes - the same, but for the background process of flushing data (pdflush), which periodically wakes up and sees if something needs to be dumped to disk (it may start flushing before the buffer is full, or may not start if will not have time to wake up (=).
These are too low values, but your problem should be completely solved. Play around with the numbers to find values ​​at which data will not be written too often and the system will not freeze when copying to slow devices. You can save them between reboots via /etc/sysctl.conf.
If the flash drive has a recording indicator, then do not pull it out while it is blinking, even if your file manager says that it is already possible (he, the brute, is lying!). If there is no indicator, then just go make yourself some tea after pressing the unmount / eject button or whatever it is.
PS The specified parameters are described in sufficient detail in the documentation
. PPS could have confused something in the details, since the time is already late, but the parameters are exactly the same.

Answer 2

On arch, with kernel 4.10 or 4.11, there were also problems with flash drives, but different. When you turn on the flash drive with 3.0, the files were dumped very slowly, and moreover, there was no guarantee that they would still be dumped.

Answer 3

From personal experience, I can say that usually problems with flash drives are caused by the OS.
I remember one flash drive for 1 g on XP often lost data, but on 8 it worked well.
When switching to ku 7 and above, everything works perfectly.

Answer 4

Could this be the result of a simple "incorrect removal" of the USB flash drive from the computer (or its freezing, or the contact was bad in the connector)?
I have a whole mountain of battered client flash drives that behave in a detailed manner on Windows, and of course, many owners of these flash drives thought that "it's nonsense to remove the flash drive correctly, and everything works fine, but computer scientists are all fools ..."
And in general, now everything is prematurely outdated ( learn to make everything more unreliable) - take at least those contact tracks in the flash drive connector - due to frequent re-plugging, they simply wear out (like sandpaper) and the contact is lost, and the flash drive just kind of "falls off" right during recording ...
I myself have personally, several flash drives of 64GB are unused for this reason)
But taking this opportunity, I’ll immediately ask experts here: how to check the file system for fat ... / ntfs errors in Linux? Ato, after all, Linux more adequately connects broken partitions (after freezing Windows / computer)