Like with any other new Google product, better wait a few years to see if it sticks before investing in its usage. In most cases, you'd be better off searching for an alternative from the start.
Colossus for Rapid Storage
51–60 of 129 posts
Re: Colossus for Rapid Storage
#52Terrifyingly complicated and buzzword packed. I really don't know what to make of any of this or what it does, and I work with AI applications in my day job.
I'm guessing the $300 of Google Cloud credit offered in this webpage wouldn't go very far using any of this stuff?
Re: Colossus for Rapid Storage
#53Earlier quoted context omitted.
Delivering "HDD prices" is a bold claim there. They charge $20/TB/month for basic cloud storage. You can build storage servers for $20/TB flat. If you add 10% for local parity, 15% free space, 5% in spare drives, and $2000/rack/month overhead, then triple everything for redundancy purposes, then over a 3 year period the price of using your own hard drives is $115/TB and google's price is $720. Over 5 years it's $145…
You forgot paying yourself to set that up.
You're paying the same for "cloud engineers".
Also, don't forget the hidden cost/risk of giving a third party full access to your data.
Re: Colossus for Rapid Storage
#54Earlier quoted context omitted.
You forgot paying yourself to set that up.
You're paying the same for "cloud engineers". Also, don't forget the hidden cost/risk of giving a third party full access to your data.
Clicking yourself a Bucket takes 5 Minutes.
Building a Server and keeping it secure and up-to-date and fixing hardware issues, takes relevant time
Re: Colossus for Rapid Storage
#55Like with any other new Google product, better wait a few years to see if it sticks before investing in its usage. In most cases, you'd be better off searching for an alternative from the start.
[flagged]
Re: Colossus for Rapid Storage
#56Is it like PureStorage?
Re: Colossus for Rapid Storage
#57Earlier quoted context omitted.
Struggling to find a definition, but seemingly zonal just means there's a massive instance per cluster. Did find some interesting recent (March 28th, 2025) reads though! Colossus under the hood: How we deliver SSD performance at HDD prices https://cloud.google.com/blog/products/storage-data-transfer... I kind of thought you meant ZNS / https://zonedstorage.io/ at first, or it's more recent better awesomer counterpart…
Delivering "HDD prices" is a bold claim there. They charge $20/TB/month for basic cloud storage. You can build storage servers for $20/TB flat. If you add 10% for local parity, 15% free space, 5% in spare drives, and $2000/rack/month overhead, then triple everything for redundancy purposes, then over a 3 year period the price of using your own hard drives is $115/TB and google's price is $720. Over 5 years it's $145…
I like your comparison with self-built storage, but comparing $20/TB/month with other CLOUD offerings, we see:
* hetzner storage box starts from $4/month for 1TB, and then goes down to $2.4/TB/month if you rent a 10TB box.
* mega starts from €10/month for 2TB, and goes down to €2/TB/month if you get a 16TB plan
* backblaze costs (starts from?) $6/TB/month
I was looking for a cheap cloud storage recently, so have a a list of these numbers :)
Moreover, these are not even the cheapest one. The cheapest one I found had prices starting from $6.5 for 5TB, going down to $0.64/TB/month for plans starting with 25TB (called uloz, but I haven't tested them yet).
Also, looking at lowendbox you can find a VPS in Canada with 2TB storage for $5/month and run whatever you want there.
How all that compares to $20/TB/month?!
Please feel free to correct me if i'm comparing apples to oranges, though. But I can't believe all of these offers are scam or so-called "promotional" offers which cost companies more than you pay for it.
Re: Colossus for Rapid Storage
#58Hats off to whoever convinced management that selling Colossus via cloud was Artificial Intelligence. Bravo.
They're not saying that it's AI. They're saying it's for customers who do AI. Training means lots and lots of reads from a big data store, and if you're reading from, like, big Parquet files, that probably means lots of random reads. This is for that. Speedier data access, presumably at the cost of durability and availability, which is probably a great trade-off for people doing ML training jobs.
calling everything 'for AI' is the new standard
>if you're reading from, like, big Parquet files, that probably means lots of random reads
and it also usually means that you shouldn't use s3 in the first place for workloads like this. Because they are usually very inefficient comparing to distributed fs. Unless you have some prefetch/cache layer, you will get both bad timings and higher costs
Re: Colossus for Rapid Storage
#59Re: Colossus for Rapid Storage
#60Everyone needs to learn to use a single, unique, unambiguous URL for new product announcements like this. Google aren't the only company that consistently mess this up, but given how they built a 1.95 trillion company on top of crawling URLs on the web they really should have an internal culture that values giving things unique URLs! [I had to learn this lesson myself: I used to blog "weeknotes" every week or two whe…
Google's not really at fault here: the OP submitted a link to an article called "Introducing Ironwood TPUs and new innovations in AI Hypercomputer" that happens to mention Rapid Storage way down the page. In case marketing seems to move faster than documentation though, since I can't find any mention of this in the main GCS docs. https://cloud.google.com/search?hl=en&q=rapid%20storage
It's down here too: https://cloud.google.com/products/storage
No link and no details though.