Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Discord Stores Billions of Messages (2017)

blog.discord.com

Post reply on HN
Print view View original

251–260 of 377 posts

Re: How Discord Stores Billions of Messages (2017)

#251

Post by mbesto »

Kaze404 wrote: post #225
mbesto wrote: post #189

Earlier quoted context omitted.

What do you prefer? The fact that you can: (1) Find a server with a simple URL (no pw needed, no port or IP, etc.) (2) Find your friends easily with a unique username and use chat as a fallback (3) Create an audio room (that scales!) that has great audio quality (doesn't "drop" calls and (4) Client that auto-updates to provide more and more features All within a few minutes is a HUGE upgrade or Ventrilo. This doesn't…

I think you misread my comment. I definitely prefer Discord over the others, but I still think it has a long way to go before it becomes a chat experience that's not insufferable to use.

Got it, definitely did.

Re: How Discord Stores Billions of Messages (2017)

#252

Post by hardwaregeek »

disillusioned wrote: post #227
hardwaregeek wrote: post #141

Earlier quoted context omitted.

Slack was/is pretty terrible too. Having every workspace require a new user is the pinnacle of idiocy. So annoying and even worse if you have different emails for different workspaces.

There are plenty of reasons to do this, all of which have to do with, say, privacy, and Slack has managed their way into making it dramatically less annoying. 1) They send magic links. Pretty easy. 2) They make all known workspaces you've logged into before discoverable and allow for a one-click "add to desktop Slack" option, which makes dealing with the whole "different users" issue. And to the extent that I use dif…

That's true, but it doesn't change the user story going from "I click a link, I join the workspace" to "I click a link, I fill out yet another registration form, decide which email to use, add another password to my password safe, then join the workspace". Minor differences but friction does matter.

Re: How Discord Stores Billions of Messages (2017)

#253

Post by fao_ »

lmilcin wrote: post #245
kevinsundar wrote: post #235

Earlier quoted context omitted.

Nice that's a pretty decent size, curious on the latency still. Thats the primary problem for a real time chat app. Complexity of data and running business rules on it is not a data store problem though, that's a compute problem. It's highly parallelizable and compute is cheap. For reference, my team runs transformations on about 1 PB of (uncompressed) data per day with 3 spark clusters, each with 50 nodes. We've got…

The latency is a complex topic. In order to be able to run so much stuff on MongoDB, we almost never run single queries to the database. If I fetch or insert trade data, I probably run a query for 10 thousand trades at the same time. So what happens is, as data comes from multiple directions it is being batched (for example 1-10 thousand at a time), split into groups that can be processed together in a roughly simila…

I'm not sure why you're being downvoted when you're a domain expert talking about your craft? People have weird hangups on hacker news, it seems

Re: How Discord Stores Billions of Messages (2017)

#254

Post by cityzen »

imadeamistake wrote: post #206
PeterCorless wrote: post #181

Earlier quoted context omitted.

We just benchmarked Cassandra 4.0, which is brand-spanking new. The good news: C 4.0 is a far better performing database than C 3.11. The new GCs definitely get rid of the long tail latency nightmares: https://www.scylladb.com/2021/08/19/cassandra-4-0-vs-cassand... However, we also compared it to Scylla's latest release, and though C 4 is better*, you can still find other CQL-compatible databases that outperform it.…

I like Scylla. I've been working with it for a while now, and it's a good alternative for transactional loads. It's a hell of a lot faster than Cassandra, and much much much much cheaper than DynamoDB. Cassandra has always felt like improvements came in fits and starts. I work at a Fortune 50 company, and Amazon quoted us ~$2-3 million a year to run our load on Dynamo (we were paying $350k/year for Aurora). With Scyl…

I'd love to hear how Dynamo would end up being $2-3 million a year They sure do a great job of convincing people that it's cheap so I'm curious where the cost seems to blow up?

Re: How Discord Stores Billions of Messages (2017)

#255

Post by fao_ »

phailhaus wrote: post #248
lmilcin wrote: post #237

Earlier quoted context omitted.

As an example, there are bitemporal queries like "for the given population of trades specified by following rules, find the set of trades that met the rules at a particular point in time, based on our knowledge at another given point in time". Also trades are versioned (are a stream of business events from trading system), then have amendments (each event may be amended in the future but the older version must be pre…

How many simultaneous queries of that nature can the system handle?

Are you not sure that financial data "with hundreds of fields" is more complex than chat data which has a relatively linear threading and only a handful of fields?

Re: How Discord Stores Billions of Messages (2017)

#257

Post by Cullinet »

lmilcin wrote: post #233
dpedu wrote: post #230

Earlier quoted context omitted.

Sounds interesting. Does this "we" have any writings about this?

I have a plan to write a book on how to write reactive applications like that. Mostly collection of observations, tips, tricks, patterns for reactive composition, some very MongoDB specific solutions, etc. Not sure how many people would be interested. Reactor has quite steep learning curve but also very little literature on how to use for anything non-trivial. The aim is not just enable good throughput, but also achi…

I'm interested in getting your book published. Career in publishing and specialist media but a lot of it spent on related problems to your subject. Semi retired have risk capital to get to the right distribution maintaining well above industry standard terms. Email in profile.

Re: How Discord Stores Billions of Messages (2017)

#258

Post by throwdbaaway »

lmilcin wrote: post #245
kevinsundar wrote: post #235

Earlier quoted context omitted.

Nice that's a pretty decent size, curious on the latency still. Thats the primary problem for a real time chat app. Complexity of data and running business rules on it is not a data store problem though, that's a compute problem. It's highly parallelizable and compute is cheap. For reference, my team runs transformations on about 1 PB of (uncompressed) data per day with 3 spark clusters, each with 50 nodes. We've got…

The latency is a complex topic. In order to be able to run so much stuff on MongoDB, we almost never run single queries to the database. If I fetch or insert trade data, I probably run a query for 10 thousand trades at the same time. So what happens is, as data comes from multiple directions it is being batched (for example 1-10 thousand at a time), split into groups that can be processed together in a roughly simila…

I thought you are doing millions qps with a 3 nodes mongodb cluster, from the top level comment. That would be impressive.

By batching 1-10 thousands records at a time, your use case is very different from discord, which needs to deliver individual messages as fast as possible.

Re: How Discord Stores Billions of Messages (2017)

#259

Post by kin »

welder wrote: post #226

Two horrible choices of databases... first they used MongoDB then they migrated to Cassandra. I've used tons of databases [1] in production and those two are the worst. [1] I've used RethinkDB, Postgres, MongoDB, MySQL, Cassandra, CockroachDB, TimescaleDB, SSDB, and others

Cassandra has been successfully deployed at many companies. Would you care to provide some insight into your experience and why you consider it one of the worst?

Re: How Discord Stores Billions of Messages (2017)

#260

Post by ZephyrBlu »

trashcan wrote: post #193
objektif wrote: post #183

Earlier quoted context omitted.

How much is very very high pay?

I don't think it would be appropriate for me to say very specifically, but I suspect about double what a software engineer with the same amount of experience would earn.

Software Engineer salaries vary wildly, so this is not particularly helpful. You could easily be talking about anywhere between $200k-$700k.
Post reply on HN