V
V
Vyacheslav Abramov2020-09-08 09:20:55
linux
Vyacheslav Abramov, 2020-09-08 09:20:55

Linux how to implement searching and moving files to automatic directory structure?

The bottom line is, there are more than 8 million files and folders.... but the problem is that they are collected in one folder and the structure is simple:

../img/
../img/1/file1.jpg
../img/2/file1.jpg
...
../img/12345/file1.doc
../img/12345/file1.jpg
../img/1234567/filerandomname1.jpg
../img/1234567/newfile18.xls


and as a result, after porting to a new server, we encountered a bunch of problems, starting with the inability to work normally with the directory, ending with freezing, and so on.

the idea arose - to break the structure into a format:

../img/1/2/3/4/5/6/7/filerandomname1.jpg

they tried to do it through php, but the script just hangs and processes 10k at the most per day ....... the idea arose to use find and mv to speed up

Are there options based on the command line, bash to remake img/folder/file.doc to img/f/o/l/d/e/r/file.doc

Answer the question

In order to leave comments, you need to log in

2 answer(s)
C
ComodoHacker, 2020-09-08
@ComodoHacker

And what will the new catalogs be called, is it not important at all? Then why bind to old names, it's easier to generate new ones like 1,2,3, etc.
PS And the new structure will not lead to "impossibility to work normally", due to the fact that the files cannot be found?

R
Roman Mirilaczvili, 2020-09-08
@2ord

Try migrating to S3 compatible services. If you want to manage your hosting resources, then take MinIO.
Using the mc (minio-client) utility, you can copy the entire directory structure as is to your cloud service or to any S3-compatible services.
Connect a caching server and then the load on the storage will subside.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question