Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Colossus for Rapid Storage

cloud.google.com

Post reply on HN
Print view View original

61–70 of 129 posts

Re: Colossus for Rapid Storage

#61

Post by ddorian43 »

Lex-2008 wrote: post #57
Dylan16807 wrote: post #47

Earlier quoted context omitted.

Delivering "HDD prices" is a bold claim there. They charge $20/TB/month for basic cloud storage. You can build storage servers for $20/TB flat. If you add 10% for local parity, 15% free space, 5% in spare drives, and $2000/rack/month overhead, then triple everything for redundancy purposes, then over a 3 year period the price of using your own hard drives is $115/TB and google's price is $720. Over 5 years it's $145…

I like your comparison with self-built storage, but comparing $20/TB/month with other CLOUD offerings, we see: * hetzner storage box starts from $4/month for 1TB, and then goes down to $2.4/TB/month if you rent a 10TB box. * mega starts from €10/month for 2TB, and goes down to €2/TB/month if you get a 16TB plan * backblaze costs (starts from?) $6/TB/month I was looking for a cheap cloud storage recently, so have a a…

Of what you mentioned, only backblaze is similar (object storage with S3-like API), all others are apples to oranges.

Re: Colossus for Rapid Storage

#62

Post by ksec »

Lex-2008 wrote: post #57
Dylan16807 wrote: post #47

Earlier quoted context omitted.

Delivering "HDD prices" is a bold claim there. They charge $20/TB/month for basic cloud storage. You can build storage servers for $20/TB flat. If you add 10% for local parity, 15% free space, 5% in spare drives, and $2000/rack/month overhead, then triple everything for redundancy purposes, then over a 3 year period the price of using your own hard drives is $115/TB and google's price is $720. Over 5 years it's $145…

I like your comparison with self-built storage, but comparing $20/TB/month with other CLOUD offerings, we see: * hetzner storage box starts from $4/month for 1TB, and then goes down to $2.4/TB/month if you rent a 10TB box. * mega starts from €10/month for 2TB, and goes down to €2/TB/month if you get a 16TB plan * backblaze costs (starts from?) $6/TB/month I was looking for a cheap cloud storage recently, so have a a…

Thank You. So backblaze for $6/TB a month. I could have a TB of Data backed up safely against file corruption? I wonder how have I missed that.

Now you could use it with Synology NAS and it is a lot cheaper than doing RAID 5 for ZFS / BTRFS with Muti redundancy.

I wonder if there are any NAS that does that automatically? Any drawbacks? Also wonder if the price could go down to $5 / TB in a few years time.

Re: Colossus for Rapid Storage

#63

Post by p_l »

zifpanachr23 wrote: post #24

Reading the press release about the "Hypercomputer" and I can't tell what part of this is real and what part is marketing. They say it comes in two configuration, 256 chips or 9,216 chips. They also say that the maximal configuration of 9,216 chips delivers 24x the compute power of the world's largest supercomputer (which they say is called El Capitan). They say that this comes to 42.6 exaFLOPs. This implies that the…

Also the set of supported/accelerated operations in the fastest path is different no matter whether you use 8, 16, or 32bit floats, thus the common use of "TOPS" as benchmark number recently.

Re: Colossus for Rapid Storage

#64

Post by Twirrim »

Dylan16807 wrote: post #47
jauntywundrkind wrote: post #8

Earlier quoted context omitted.

Struggling to find a definition, but seemingly zonal just means there's a massive instance per cluster. Did find some interesting recent (March 28th, 2025) reads though! Colossus under the hood: How we deliver SSD performance at HDD prices https://cloud.google.com/blog/products/storage-data-transfer... I kind of thought you meant ZNS / https://zonedstorage.io/ at first, or it's more recent better awesomer counterpart…

Delivering "HDD prices" is a bold claim there. They charge $20/TB/month for basic cloud storage. You can build storage servers for $20/TB flat. If you add 10% for local parity, 15% free space, 5% in spare drives, and $2000/rack/month overhead, then triple everything for redundancy purposes, then over a 3 year period the price of using your own hard drives is $115/TB and google's price is $720. Over 5 years it's $145…

Leverage erasure encoding for durability and avoid both the tripling and local parity. You'll get better durability than 3x while only taking up significantly less than 2x the space Backblaze open sourced their library and talk about it here, https://www.backblaze.com/blog/Reed-Solomon. They use a 17:20 ratio that'll get them 3 drive failure resistance for just 1.17x stretch (ie a 100mb file gets that resilient while taking up 117mb of space)

Re: Colossus for Rapid Storage

#65

Post by onlyrealcuzzo »

phonon wrote: post #25
zifpanachr23 wrote: post #24

Reading the press release about the "Hypercomputer" and I can't tell what part of this is real and what part is marketing. They say it comes in two configuration, 256 chips or 9,216 chips. They also say that the maximal configuration of 9,216 chips delivers 24x the compute power of the world's largest supercomputer (which they say is called El Capitan). They say that this comes to 42.6 exaFLOPs. This implies that the…

Supercomputers are measured based on 64 bit floating point operations. Here they (inaptly) compared it to their 8 bit floating point operations (which are only useful for AI workloads).

Why is it inapt?

If all you care about is an 8-bit AI workload (there's definitely a market for that), it's nice to have 24x the speed.

Re: Colossus for Rapid Storage

#66

Post by ddorian43 »

ksec wrote: post #62
Lex-2008 wrote: post #57

Earlier quoted context omitted.

I like your comparison with self-built storage, but comparing $20/TB/month with other CLOUD offerings, we see: * hetzner storage box starts from $4/month for 1TB, and then goes down to $2.4/TB/month if you rent a 10TB box. * mega starts from €10/month for 2TB, and goes down to €2/TB/month if you get a 16TB plan * backblaze costs (starts from?) $6/TB/month I was looking for a cheap cloud storage recently, so have a a…

Thank You. So backblaze for $6/TB a month. I could have a TB of Data backed up safely against file corruption? I wonder how have I missed that. Now you could use it with Synology NAS and it is a lot cheaper than doing RAID 5 for ZFS / BTRFS with Muti redundancy. I wonder if there are any NAS that does that automatically? Any drawbacks? Also wonder if the price could go down to $5 / TB in a few years time.

The price of Backblaze WAS $5 a few years ago and they increased it to $6 (and added some free bandwidth).

Re: Colossus for Rapid Storage

#67

Post by immibis »

Lex-2008 wrote: post #57
Dylan16807 wrote: post #47

Earlier quoted context omitted.

Delivering "HDD prices" is a bold claim there. They charge $20/TB/month for basic cloud storage. You can build storage servers for $20/TB flat. If you add 10% for local parity, 15% free space, 5% in spare drives, and $2000/rack/month overhead, then triple everything for redundancy purposes, then over a 3 year period the price of using your own hard drives is $115/TB and google's price is $720. Over 5 years it's $145…

I like your comparison with self-built storage, but comparing $20/TB/month with other CLOUD offerings, we see: * hetzner storage box starts from $4/month for 1TB, and then goes down to $2.4/TB/month if you rent a 10TB box. * mega starts from €10/month for 2TB, and goes down to €2/TB/month if you get a 16TB plan * backblaze costs (starts from?) $6/TB/month I was looking for a cheap cloud storage recently, so have a a…

I get the impression that business has always been about being the best schmoozer more than about having the best product.

BTW at Hetzner you can rent servers with very large (hundred of TB) non-redundant storage for an effective price of about $1.50/TB/month. If you want to build a cloud storage product, that seems like a good starting point - of course, once you take into account redundancy, spare capacity, and paying yourself, the prices you charge to your customers will end up closer to the price of Backblaze at a minimum.

Re: Colossus for Rapid Storage

#68

Post by immibis »

jeffbee wrote: post #30
hiddencost wrote: post #7

Hats off to whoever convinced management that selling Colossus via cloud was Artificial Intelligence. Bravo.

I would pay serious money if they sold CFS as a service but on AWS.

Hi, I'm looking for a job. Are you willing to pay me serious money to set up CFS as a service on your AWS?

Re: Colossus for Rapid Storage

#69

Post by remus »

onlyrealcuzzo wrote: post #65
phonon wrote: post #25

Earlier quoted context omitted.

Supercomputers are measured based on 64 bit floating point operations. Here they (inaptly) compared it to their 8 bit floating point operations (which are only useful for AI workloads).

Why is it inapt? If all you care about is an 8-bit AI workload (there's definitely a market for that), it's nice to have 24x the speed.

It's an apples to oranges comparison.

Re: Colossus for Rapid Storage

#70

Post by lysecret »

davedx wrote: post #52

Terrifyingly complicated and buzzword packed. I really don't know what to make of any of this or what it does, and I work with AI applications in my day job. I'm guessing the $300 of Google Cloud credit offered in this webpage wouldn't go very far using any of this stuff?

You can try out everything for 300 dollars easily. Most expensive thing you can do is get a server with 8 h200s and spend 90 dollars an hour.
Post reply on HN