How Discord Stores Billions of Messages (2017) • Hacker News Board

Re: How Discord Stores Billions of Messages (2017)

#371

Post by MacsHeadroom » Fri, Aug 27, 2021, 9:55 AM UTC

anonymoushn wrote: post #328

enlyth wrote: post #314

Earlier quoted context omitted.

What about cryptocurrency trading which goes on continuously for 24 hours a day?

Cryptocurrency venues handle extremely low throughput and, with maybe one exception, regularly go down for hours.

The decentralized Solana network is currently capable of over 50k TPS 24/7 in the live Beta with a target transaction finality speed of 30ms round trip from your browser/device. Their unofficial moto is "DeFi at Nasdaq Speed." Solana is nascent and will likely reach 10M+ TPS and 10ms finality within a couple years time.

Decentralized Acyclic Graph based networks (e.g. Hashigraph, which are not technically blockchains) can reach effectively infinite TPS but suffer in time to finality.

Solana is a blockchain with zero downtown (and a Turing complete smartchain), mind you-- nota centralized exchange.

Re: How Discord Stores Billions of Messages (2017)

#372

Post by anonymoushn » Fri, Aug 27, 2021, 10:00 AM UTC

MacsHeadroom wrote: post #371

anonymoushn wrote: post #328

Earlier quoted context omitted.

Cryptocurrency venues handle extremely low throughput and, with maybe one exception, regularly go down for hours.

The decentralized Solana network is currently capable of over 50k TPS 24/7 in the live Beta with a target transaction finality speed of 30ms round trip from your browser/device. Their unofficial moto is "DeFi at Nasdaq Speed." Solana is nascent and will likely reach 10M+ TPS and 10ms finality within a couple years time. Decentralized Acyclic Graph based networks (e.g. Hashigraph, which are not technically blockchains…

Solana has 400ms block times, so I don't think it can achieve finality in 30ms, and the whole comment seems a bit off-topic.

Re: How Discord Stores Billions of Messages (2017)

#373

Post by thibauts » Fri, Aug 27, 2021, 2:55 PM UTC

lmilcin wrote: post #228

Well... we have 3 node MongoDB cluster and are processing up to a million trades... per second. And a trade is way more complex than a chat message. Has tens to hundreds of fields, may require enriching with data from multiple external services and then requires to be stored, be searchable with unknown, arbitrary bitemporal queries and may need multiple downstream systems to be notified depending on a lot of factors…

This is my experience too. Millions of persisted and serialized TPS on regular NVMe with ms latency. Though it took a bit of effort to get to these numbers.

https://github.com/dataptive/styx

Re: How Discord Stores Billions of Messages (2017)

#374

Post by auchtopus » Fri, Aug 27, 2021, 6:46 PM UTC

mikesun wrote: post #70

klaussilveira wrote: post #11

Did they ever make it to ScyllaDB?

Yes. And we're hiring storage engineers! https://discord.com/jobs/5411664002 https://discord.com/jobs/5426301002

Does Discord offer undergraduate internships? I use the platform daily, enjoyed reading the thought process that went in to this design, and would love the opportunity to intern there.

If y'all are, I'd like to get in contact! My email is: anthony75025[at]gmail[dot]com, and my resume can be found at https://www.anthonyjiang.com/pages/resume.html

Re: How Discord Stores Billions of Messages (2017)

#375

Post by PeterCorless » Fri, Aug 27, 2021, 7:00 PM UTC

AtlasBarfed wrote: post #363

bsaul wrote: post #173

Earlier quoted context omitted.

what would you recommend now instead ?

Kind of the million dollar question. People like to complain about cassandra, but that just brings up the adage about C++: people complain about the things they use. But let's not pretend that cassandra isn't almost always a bear. The other problem is that cassandra keeps things up (and never gets the credit for it) but that creates a host of edge cases and management headaches (which makes management hate it). Most…

> Scylla did a C++ rewrite to address tail latency due to JVM GC, but after an explosive release cycle, they basically stalled at partial 2.2 compatiblity

Could you explain this more? Because Scylla has had pretty steady major release updates over the past few year. See the timeline of updates here:

https://www.scylladb.com/2021/08/24/apache-cassandra-4-0-vs-...

We have long since passed C* 3.11 compatibility. In fact, if anything, Scylla, while maintaining a tremendous amount of Cassandra compatibility, now offers better implementations of features present in Cassandra (Materialized Views, Secondary Indexes, Change Data Capture, Lightweight Transactions), plus unique features of its own — incremental compaction and workload prioritization.

But if there's something in particular you're thinking of, I'm open to hear more on how you see it.

Re: How Discord Stores Billions of Messages (2017)

#376

Post by naikrovek » Wed, Sep 01, 2021, 4:29 AM UTC

unchar1 wrote: post #320

naikrovek wrote: post #303

Earlier quoted context omitted.

Go and Rust won't help there, but Zig might.

I can understand that Go has a GC so it (might) be slower. But wouldn't Rust and Zig have about the same performance?

Go is slower because it optimizes for compile time.

Rust is slower (usually) because Rust does not revolve around making the most use of the hardware it has.

both are fine choices; nothing wrong with either direction.

Zig performs very well by default because it was designed to be efficient and fast from the start, without compromise. it has memory safety, too, but in a way that few seem to understand, myself included, so it's difficult for me to describe with my rudimentary understanding.

Re: How Discord Stores Billions of Messages (2017)

#377

Post by trashcan » Thu, Sep 02, 2021, 9:19 PM UTC

ZephyrBlu wrote: post #260

trashcan wrote: post #193

Earlier quoted context omitted.

I don't think it would be appropriate for me to say very specifically, but I suspect about double what a software engineer with the same amount of experience would earn.

Software Engineer salaries vary wildly, so this is not particularly helpful. You could easily be talking about anywhere between $200k-$700k.

To elaborate, I meant 2x compared to SWEs at the same company. I'd prefer not to post exact dollar amounts as they are relative based on location, company, and several other things.