You need a private cloud on 500Tb. From which components to assemble in proportion high quality and reasonable price?

Question

Cloud computing

You need to organize a cloud for personal use at home or in the office. I travel a lot and shoot on a RED camera, in which one minute of video weighs several gigabytes, I don’t see the point in carrying a bunch of disks with me, I think it’s easier to put a cloud in Moscow at home or at work and throw everything into it via the Internet. All problems with the Internet while traveling are quite easily solved. But the video is confiscated in some countries, and if it was transferred to the cloud, then the footage would not be lost.
Get to the point. Tell me from what and how to build a cloud for personal use? What components do you recommend? Where and how is it cheaper to buy them? Is it necessary for someone in Moscow to monitor the state of the cloud, or can it work autonomously without third-party participation? I will be the only one using it.

Answer 1

Ugh, I started counting prices here for a 500PB hundredfold ...
We buy semi-unit servers (on an atom / junior pentiums) + JBODs for them (I just found this offhand , but there are also cheaper ones with a denser placement of disks, only in Russia not sold).
If you don't want any prettiness, then we collect it on each raid6, set up disk/raid monitoring and upload files via sftp.
If you want prettiness - we collect gluster from all nodes, put on a pair of owncloud nodes, mount gluster there and rejoice.
If you take 6 TB disks, then you, taking into account raids, will need 9 shelves of 12 disks. Half node with an atom costs about a thousand dollars, JBOD - the same amount. Accordingly, pieces of iron without disks will cost about $18k (you can shrink to 12k if you connect 2 JBODs to one half-node, but here you need to consult with the hardware guys). The drives are suitable and desktop in fact, but not cheap lines like WD Green. Yes, and server SATA drives are not much more expensive than ordinary ones (desktop Western Digital WD60EZRX 108 pieces - 1620k rubles, server Western Digital WD60PURX - 1728k rubles).
Next, you need to find a DC where all this will live (half a rack) - 20-30 thousand per month in Moscow. Plus, keep a supply of disks for replacement - count yourself. Well, it’s not bad to actually take an admin to monitor the raids (otherwise it will fall apart and you won’t notice) - 5 thousand per month, maybe less (in fact, any support of the hoster can handle it, but still).
There is another option with 60-70 disk shelves, but this option is less reliable (in such shelves, disks heat up more and fly out faster), and one such shelf costs from one and a half million (and in fact - 3-4 million rubles), and you You need at least 2 of them. But it will be more compact placed in the rack.
In general, offhand - 2.5-3 million rubles for hardware (this is already with the first purchase of 108 disks) + 25-35 thousand per month for maintenance + disks for reserve months).
You can go crazy and look for tighter JBODs (I know models of 15 disks in 1U) and win on the content, but I'm not sure that then the price of hardware will be lower (due to delivery-customs).
With software, there are generally a billion options, but I would advise not to come up with anything fun and progressive, but simply use each half-node as a separate storage server (and if you combine them, then just collect NFS in one place, and not do one network storage ).
(oh yes, if the office has a rack with free 25U, then you can plug it in there. Electricity, by the way, does not eat all this very much, it seems 2-3 kilowatts).

Answer 2

All problems with the Internet while traveling are quite easily solved.

That is, you have no problems uploading a terabyte of data to a personal server in Moscow in the evening? Even if you suggest that you have one gigabit optical fiber at home and in the field too ... and no packet loss, etc. it will take more than two hours to download a terabyte of data.
You can take some Amazon s3, but half a petabyte storage will cost 30K per month, excluding traffic.

Answer 3

Now I'll try to surprise)
www.vdslr.ru/post/628/amazon-anonsirovala-oblako-d...
Amazon offered an unlimited cloud for 60 bucks a year)
And most importantly!
Pleased with one of the comments:
UPD. Do not forget that files on the cloud can disappear due to a variety of reasons.

Answer 4

Budget?
Option A. Buy a server (and a disk shelf) with the required characteristics and place it in a data center (at home or at work).
Option B. Rent a server in a data center with the required characteristics.
Option C. Rent the necessary capacities in one existing cloud.
PS The Internet is not fast everywhere, gigabytes while traveling are not as easy to send as it seems at first glance.

Answer 5

Before tricks with the course, a 900 gigabyte HDD in an HP shelf cost 150,000 rubles. Do you have about a million dollars for such a toy?
It makes no sense to consider office disks in such a role - most of the time you will be replacing failed disks.
And how much electricity will be spent on food and cooling ...