Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing S2

s2.dev

Post reply on HN
Print view View original

41–50 of 206 posts

Re: Introducing S2

#41

Post by agallego »

shikhar wrote: post #17
stogot wrote: post #13

Earlier quoted context omitted.

Could you clarify the Kafka difference further? Or more generally, when is it better to choose S2 vs services like SQS or Kinesis? S2 sounds like an ordered queue to me, but those exist?

(Founder here) Managed cloud offerings for streaming limit ordered throughput pretty low, e.g. Kinesis at 1 MiBps, Redpanda serverless at 1 MiBps, Confluent's even higher-end clusters at 10-20 MiBps IIRC. If you really need ordering, this can indeed be a limit. S2 lets you push 125 MiBps currently, and we may grow that. Another factor is how many ordered streams you can have. Typically a few thousand at most with tho…

Redpanda cloud doesn’t limit tput. Most ppl get a bigger discount at high volumes. We have customers in 10s of GB/s. Confluent has those volumes too.

Re: Introducing S2

#42

Post by shikhar »

masterj wrote: post #30

So is this basically WarpStream except providing a lower-level API instead of jumping straight to Kafka compatibility? An S3-level primitive API for streaming seems really valuable in the long-term if adopted

(Founder) That somewhat summarizes yes :) We take a different approach than WarpStream architecturally too, which allows us to offer much lower latencies. No disks in our system, either.

Re: Introducing S2

#43

Post by 8n4vidtmkvmk »

bushido wrote: post #23

I wish more dev-tools startups would focus on clearly explaining the business use cases, targeting a slightly broader audience beyond highly technical users. I visited several pages on the site before eventually giving up. I can sort of grasp what the S2 team is aiming to achieve, but it feels like I’m forced to perform unnecessary mental gymnastics to connect their platform with the specific problems it can solve fo…

If you ever figure it out, LMK. I don't think I've ever looked at logs more than about 24 hours old. Persistence and durability is not something I care about.

Errors, OTOH, I need a week or two of. But I consider these 2 different things. Logs are kind of a last resort when you really can't figure out what's going on in prod.

Re: Introducing S2

#44

Post by shikhar »

shikhar wrote: post #17
stogot wrote: post #13

Earlier quoted context omitted.

Could you clarify the Kafka difference further? Or more generally, when is it better to choose S2 vs services like SQS or Kinesis? S2 sounds like an ordered queue to me, but those exist?

(Founder here) Managed cloud offerings for streaming limit ordered throughput pretty low, e.g. Kinesis at 1 MiBps, Redpanda serverless at 1 MiBps, Confluent's even higher-end clusters at 10-20 MiBps IIRC. If you really need ordering, this can indeed be a limit. S2 lets you push 125 MiBps currently, and we may grow that. Another factor is how many ordered streams you can have. Typically a few thousand at most with tho…

@agallego Yes in aggregate both Confluent and Redpanda can push GiBps throughputs, and I know Redpanda has amazing perf. I was referring to Redpanda Serverless :) And per-partition i.e. ordered throughput.

ED: for some reason I wasn't seeing the reply link before on your comment, do see it now.

Re: Introducing S2

#45

Post by Scaevolus »

This is a very useful service model, but I'm confused about the value proposition given how every write is persisted to S3 before being acknowledged.

I suppose the writers could batch a group of records before writing them out as a larger blob, with background processes performing compaction, but it's still an object-backed streaming service, right?

AWS has shown their willingness to implement mostly-protocol compatible services (RDS -> Aurora), and I could see them doing the same with a Kafka reimplementation.

Re: Introducing S2

#46

Post by shikhar »

alwa wrote: post #40
shikhar wrote: post #17

Earlier quoted context omitted.

(Founder here) Managed cloud offerings for streaming limit ordered throughput pretty low, e.g. Kinesis at 1 MiBps, Redpanda serverless at 1 MiBps, Confluent's even higher-end clusters at 10-20 MiBps IIRC. If you really need ordering, this can indeed be a limit. S2 lets you push 125 MiBps currently, and we may grow that. Another factor is how many ordered streams you can have. Typically a few thousand at most with tho…

Between your site copy and your early comments on this thread, it was this rundown that made the product click in my mind. I’m sure that in this early preview you’re trying to reach mainly devs with existing domain expertise, but the way that, in this comment, you laid out existing constraints and what possibilities might lie beyond them—it really helped me situate your S2 product in the constellation of cloud primit…

thank you for the feedback!

Re: Introducing S2

#50

Post by anal_reactor »

thaumasiotes wrote: post #21
jsheard wrote: post #19

Earlier quoted context omitted.

How many of these letter-number storage services are there now? S3, B2, R2, S2...

S3 isn't the name of the service - that's "Amazon Simple Storage Service". S3 is a nickname, short for "Simple Storage Service".

When I was a student we had a Facebook group to share information, and one angry guy ranted that the correct shortening of "Mathematical Analysis" is not, in fact, "anal", as we were used to say
Post reply on HN