41–50 of 374 posts
Here's an unpopular answer - don't store 10PB of data. Find a way for your startup to work without needlessly having to store insane amounts of data that will likely never be needed.
We are a photo/video storage service.
Here's an unpopular answer - don't store 10PB of data. Find a way for your startup to work without needlessly having to store insane amounts of data that will likely never be needed.
"We use ML to ensure we only store the highest quality data, freeing you from the chains of having too much worthless data and nothing to do with it."
It's going to depend entirely on a number of factors. How are you storing this data? Is it tons of small objects, or a smaller number of massive objects? If you can aggregate the small objects into larger ones, can you compress them? Is this 10PB compressed or not? If this is video or photo data, compression won't buy you nearly as much. If you have to access small bits of data, and this data isn't something like Par…
We don’t touch the data at all.
If data storage isn't your startup's job then I would negotiate heavily on the AWS contract.
Are you fundamentally a data storage business or are you another business that happens to store a tremendous amount of data? If it's the former, then investing in-house might make sense (a la Dropbox's reverse course).
He's the CTO of KeepSafe.
Here's an unpopular answer - don't store 10PB of data. Find a way for your startup to work without needlessly having to store insane amounts of data that will likely never be needed.
Excellent advice for a data backup startup.
This is like an ISP asking how they can get hooked up to the internet.
Earlier quoted context omitted.
To be fair, the data scientists rarely have the foresight to know what the data scientists need the information for. The only time I've seen a data scientist correctly include all the data they needed (but still be wrong) was when they answered "All of it. We need all of the data".
So true. Tough to know in advance which data will hold the secrets.
Here's an unpopular answer - don't store 10PB of data. Find a way for your startup to work without needlessly having to store insane amounts of data that will likely never be needed.
You could probably put a fun marketing spin on that. "We use ML to ensure we only store the highest quality data, freeing you from the chains of having too much worthless data and nothing to do with it."
Backblaze B2, ingress and egress are free through Cloudflare, and it's S3 compatible. It's peanuts by comparison but I've been storing ~22TB on there for years and love it. Wasabi and Glacier would be my 2nd choices.