Hi HN! A couple of us from the Magic Pocket software team are around to answer questions if anyone has some.
Dropbox’s Exodus from the Amazon Cloud
191–200 of 252 posts
Re: Dropbox’s Exodus from the Amazon Cloud
#192Hi HN! A couple of us from the Magic Pocket software team are around to answer questions if anyone has some.
Which OS and FS the rust code is running on ?
Our previous version of the storage layer ran on top of XFS which was probably a bad choice since we ran into a few XFS bugs along the way, but nothing serious.
Re: Dropbox’s Exodus from the Amazon Cloud
#193Earlier quoted context omitted.
Yes, balancing links and distributing work w/out congestion collapse became a really interesting and nontrivial problem.
I can't even imagine. Did you guys get into custom network hardware or still a Juniper/Cisco shop?
Re: Dropbox’s Exodus from the Amazon Cloud
#194I am curious to know, with so many disks densely packed in a 4U configuration, I am guessing there is definitely increased heat generation and not to mention vibration. How do you handle these? Also, does it have any effect on MTBF?
Yeah, our hardware teams put a lot of qualification time into hardware profiles before we buy in bulk. And part of what we test is power utilization, heat, etc, under normal software loads vs extreme loads, to make sure we don't exceed power budget, don't spin everything up at the same time, etc. This process can take months. A lot of the system design is about talking closely to vendors about MTBF for specific loads…
Re: Dropbox’s Exodus from the Amazon Cloud
#195Earlier quoted context omitted.
So NVMe is "ready for production"? Do you think you'd use it at home, or only in the enterprise?
I mean, if at home you need 400k IOPS, 1GB/s of writes, and 2.2GB/s of reads... go for it! I sure don't, but more power to you. :-)
Re: Dropbox’s Exodus from the Amazon Cloud
#196Earlier quoted context omitted.
> Are you using spindle or ssd or flash? (admittedly I dont know if you consider flash and ssd to be the same) We have flash caches (NVMe) in the machines, but the long-term storage devices are spindle--PMR and SMR. > What is your price per GB raw? I cannot disclose, exactly, but it is well below any other public price I've seen.
So NVMe is "ready for production"? Do you think you'd use it at home, or only in the enterprise?
Re: Dropbox’s Exodus from the Amazon Cloud
#197Is there a link without the anti-adblock crap?
Re: Dropbox’s Exodus from the Amazon Cloud
#198Earlier quoted context omitted.
So NVMe is "ready for production"? Do you think you'd use it at home, or only in the enterprise?
We have been primarily using Intel NVMe storage for our database servers since fall of 2014 with no major complaints. Our high end desktop/laptop systems are also using the latest Samsung NVMe M.2 drives which are screaming fast.
Re: Dropbox’s Exodus from the Amazon Cloud
#199Hi HN! A couple of us from the Magic Pocket software team are around to answer questions if anyone has some.
Do you know what Amazon is going to do with their excess 500PB of capacity? Are they scaling up fast enough that it isn't a big deal? Is Glacier selling your vacated storage? You must have left a large hole in AWS' revenue stream. I assume their fees for transferring your data out of their system helped cover that short-term, but 500PB of data is a lot of storage capacity to sell. Your other answers indicate your sep…
Re: Dropbox’s Exodus from the Amazon Cloud
#200Earlier quoted context omitted.
We have several dozen very, very good engineers working on a few discrete teams--infrastructure security, product security, etc. I don't feel comfortable getting any more specific than that; we keep security information pretty close to the vest.
To be honest, that response is less than comforting. But I understand how that'd be your answer given your position in the organization. Hopefully someone else at Dropbox feels like sharing more at some point and being more forthright about your company's practices in protecting data others entrust to them. Feel free to refer others to this thread if you think it'd be helpful.