A
A
alpolle2013-01-18 16:03:13
Data synchronization
alpolle, 2013-01-18 16:03:13

Synchronization tool needed?

SW. habrausers!
Prompt software for synchronization of two directories, but with certain requirements.
It appears so, there is a server on Win2008R2, on which there is a directory with about 63000 subdirectories, in which there are about 220000 small files (from 10KB to 3MB in size). On average, about 70-80 new files are added per day.
You need to sync this directory to your Synology Rackstation.
The current utilities that are used for synchronization (nnbackup, robocopy) are quite slow because of the need for a full directory scan. According to the latest data, it took robocopy 1 hour and 44 minutes to copy 37 files.
Ideally, I see some service that monitors this directory (perhaps some kind of index), and when a file is added, it only synchronizes it.
Thank you!

Answer the question

In order to leave comments, you need to log in

5 answer(s)
L
la0, 2013-01-18
@la0

It is logical that any event-oriented tool.
Eg. dklab realsync (from the simplest thing on my mind)

A
adminimus, 2013-01-18
@adminimus

if the total size of the files is not too large, then you can try to transfer them to a ramdisk. A few years ago I solved the problem of synchronizing a bunch of small files, but there were slightly different conditions: more files were created per day, but almost all of them were constantly added during the day. The ramdisk and rsync solution still works.
If your files don't change after they've been created, it's actually easier to track events than it is to constantly scan directories.

A
Anastasia_K, 2013-01-18
@Anastasia_K

Haven't seen synology cloud station?

V
Valentine, 2013-01-18
@vvpoloskin

cygwin + rsync :)
cygwin is just a useful thing with a bunch of built-in programs that do not need to be installed later, and rsync is included

S
sl_bug, 2013-01-19
@sl_bug

git + rsync you can try, not the fact that it will master, but you can try

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question