Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Colossus for Rapid Storage

cloud.google.com

Post reply on HN
Print view View original

101–110 of 129 posts

Re: Colossus for Rapid Storage

#101

Post by CobrastanJorji »

Zvez wrote: post #58
CobrastanJorji wrote: post #40

Earlier quoted context omitted.

They're not saying that it's AI. They're saying it's for customers who do AI. Training means lots and lots of reads from a big data store, and if you're reading from, like, big Parquet files, that probably means lots of random reads. This is for that. Speedier data access, presumably at the cost of durability and availability, which is probably a great trade-off for people doing ML training jobs.

calling everything 'for AI' is the new standard >if you're reading from, like, big Parquet files, that probably means lots of random reads and it also usually means that you shouldn't use s3 in the first place for workloads like this. Because they are usually very inefficient comparing to distributed fs. Unless you have some prefetch/cache layer, you will get both bad timings and higher costs

But a distributed FS is far more expensive than cloud blob storage would be, and I can't imagine most workloads would need the features of a POSIX filesystem.

Re: Colossus for Rapid Storage

#102

Post by jeffbee »

dastbe wrote: post #89
akshayshah wrote: post #81

Very cool! This makes Google the only major cloud that has low-latency single-zone object storage, standard regional object storage, and transparently-replicated dual-region object storage - all with the same API. For infra systems, this is great: code against the GCS API, and let the user choose the cost/latency/durability tradeoffs that make sense for their use case.

? s3: https://aws.amazon.com/pm/serv-s3 s3 express: https://aws.amazon.com/s3/storage-classes/express-one-zone/ cross-region replication: https://docs.aws.amazon.com/AmazonS3/latest/userguide/replic...

The semantics they are offering are very different from S3. In Colossus a writer can make a durable 1-byte append and other observers are able to reason about the commit point. S3 does not offer this property.

Re: Colossus for Rapid Storage

#104

Post by noahl »

immibis wrote: post #84
jsnell wrote: post #74

Earlier quoted context omitted.

Obviously not, since you could not deliver it. It seems that you maybe don't realize what CFS is in this context, and are thinking of something else that you could just "set up"? What jeffbee is talking about is Google's proprietary Colossus File System, and all its transitive dependencies.

I meant it sarcastically, but for "serious money" you can have any software system you can dream of. You have to dream of it, though - that's one of the hard parts. It looks like every other clustered file system. What's special about Google's Colossus?

There are some semantic differences compared to POSIX filesystems. A couple big ones:

  - You can only append to an object, and each object can only have one writer at the time. This is useful for distributed systems - you could have one process adding records to the end of a log, and readers pulling new records from the end.
  - It's also possible to "finalize" an object, meaning that it can't be appended to any more.
(I work on Rapid storage.)

Re: Colossus for Rapid Storage

#105

Post by singhrac »

immibis wrote: post #67
Lex-2008 wrote: post #57

Earlier quoted context omitted.

I like your comparison with self-built storage, but comparing $20/TB/month with other CLOUD offerings, we see: * hetzner storage box starts from $4/month for 1TB, and then goes down to $2.4/TB/month if you rent a 10TB box. * mega starts from €10/month for 2TB, and goes down to €2/TB/month if you get a 16TB plan * backblaze costs (starts from?) $6/TB/month I was looking for a cheap cloud storage recently, so have a a…

I get the impression that business has always been about being the best schmoozer more than about having the best product. BTW at Hetzner you can rent servers with very large (hundred of TB) non-redundant storage for an effective price of about $1.50/TB/month. If you want to build a cloud storage product, that seems like a good starting point - of course, once you take into account redundancy, spare capacity, and pay…

Yes, this is pretty much what Hetzner must have built with their object storage - and they get to 5 EUR/month, so really close to Backblaze pricing.

Re: Colossus for Rapid Storage

#106

Post by korkybuchek »

akshayshah wrote: post #95
korkybuchek wrote: post #92

Earlier quoted context omitted.

> This makes Google the only major cloud that has low-latency single-zone object storage, standard regional object storage, Absurd claim. S3 Express launched last year.

Sure, but AFAIK S3’s multi-region capabilities are quite far behind GCS’s. S3 offers some multi-region replication facilities, but as far as I’ve seen they all come at the cost of inconsistent reads - which greatly complicates application code. GCS dual-region buckets offer strongly consistent metadata reads across multiple regions, transparently fetch data from the source region where necessary, and offer clear SLAs…

> Sure, but AFAIK S3’s multi-region capabilities are quite far behind GCS’s.

Entirely different claim.

Re: Colossus for Rapid Storage

#107

Post by akshayshah »

korkybuchek wrote: post #106
akshayshah wrote: post #95

Earlier quoted context omitted.

Sure, but AFAIK S3’s multi-region capabilities are quite far behind GCS’s. S3 offers some multi-region replication facilities, but as far as I’ve seen they all come at the cost of inconsistent reads - which greatly complicates application code. GCS dual-region buckets offer strongly consistent metadata reads across multiple regions, transparently fetch data from the source region where necessary, and offer clear SLAs…

> Sure, but AFAIK S3’s multi-region capabilities are quite far behind GCS’s. Entirely different claim.

I claimed that Google is the only major cloud provider with all three of:

- single-zone object storage buckets

- regional object storage buckets

- transparently replicated, dual region object storage buckets

I agree that AWS has two of the three. AFAIK AWS does not have multi-region buckets - the closest they have is canned replication between single-region buckets.

Re: Colossus for Rapid Storage

#108

Post by thayne »

korkybuchek wrote: post #106
akshayshah wrote: post #95

Earlier quoted context omitted.

Sure, but AFAIK S3’s multi-region capabilities are quite far behind GCS’s. S3 offers some multi-region replication facilities, but as far as I’ve seen they all come at the cost of inconsistent reads - which greatly complicates application code. GCS dual-region buckets offer strongly consistent metadata reads across multiple regions, transparently fetch data from the source region where necessary, and offer clear SLAs…

> Sure, but AFAIK S3’s multi-region capabilities are quite far behind GCS’s. Entirely different claim.

S3 doesn't have "transparently-replicated dual-region object storage", which was part of the claim.

S3 does have replication, but it is far from transparent and frought with gotchas.

And it certainly doesn't have all of that with a single API.

Re: Colossus for Rapid Storage

#110

Post by Dylan16807 »

ddorian43 wrote: post #61
Lex-2008 wrote: post #57

Earlier quoted context omitted.

I like your comparison with self-built storage, but comparing $20/TB/month with other CLOUD offerings, we see: * hetzner storage box starts from $4/month for 1TB, and then goes down to $2.4/TB/month if you rent a 10TB box. * mega starts from €10/month for 2TB, and goes down to €2/TB/month if you get a 16TB plan * backblaze costs (starts from?) $6/TB/month I was looking for a cheap cloud storage recently, so have a a…

Of what you mentioned, only backblaze is similar (object storage with S3-like API), all others are apples to oranges.

You don't need very many terabytes to cover the labor cost of installing and maintaining an S3-compatible server program.
Post reply on HN