Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Discord Stores Billions of Messages (2017)

blog.discord.com

Post reply on HN
Print view View original

191–200 of 377 posts

Re: How Discord Stores Billions of Messages (2017)

#191

Post by PeterCorless »

yeswecatan wrote: post #20
gregoriol wrote: post #16

Earlier quoted context omitted.

2017 is pretty antique though now: the scaling and the ecosystem change fast

I admittedly know very little about this space. What are some of the newer players?

If you are not familiar, DB-engines.com keeps a listing of who the major players are in the database world.

https://db-engines.com/en/ranking

MongoDB is ranked #5 on the list at present; Cassandra comes in at #11. (And Scylla, which they moved to most of their workload from Cassandra, is currently #88.)

DB-engines also have specific rankings for what are known as 'NoSQL wide column stores' — which is what Cassandra and Scylla are classed as:

https://db-engines.com/en/ranking/wide+column+store

Note that MongoDB is a different class of NoSQL entirely. It is a "document store" — MongoDB is the most popular document store.

https://db-engines.com/en/ranking/document+store

But what this means is that even though both MongoDB, Cassandra and Scylla are all "NoSQL" making this move for Discord required significant data modeling and migration.

(Note that the difference between Cassandra and Scylla is far narrower. Both use the same data model and Cassandra Query Language (CQL).

Hope that helps give you some orientation in the NoSQL database field.

Re: How Discord Stores Billions of Messages (2017)

#192

Post by manigandham »

bcrosby95 wrote: post #120
NotAnOtter wrote: post #7

This is like a masterclass in how to answer system design questions. Maybe a bit verbose. They cover requirements, how to answer those requirements, relevant tech for the problem, implementation, and techniques for maintenance

Are you saying that one person, without consulting other engineers, or having to do any research, made this decision in less than a day? Because I have some bad news for you. The sentiment you express here is why interviewing is so shit these days.

1) Yes, there are people that can do this.

2) But this isn't an interview situation. This was "system design questions" as in how to solve a problem as a company using the whole team (4 backend engineers at that time).

Re: How Discord Stores Billions of Messages (2017)

#193

Post by trashcan »

objektif wrote: post #183
trashcan wrote: post #145

Earlier quoted context omitted.

I've used Cassandra at two companies, and had the exact same experience as you at the first company. At a much bigger company that had some very, very highly paid Cassandra DBAs it was actually a relatively smooth experience.

How much is very very high pay?

I don't think it would be appropriate for me to say very specifically, but I suspect about double what a software engineer with the same amount of experience would earn.

Re: How Discord Stores Billions of Messages (2017)

#194

Post by Dylan16807 »

CoryAlexMartin wrote: post #186
RHSeeger wrote: post #36

Earlier quoted context omitted.

Honestly, I find discord super frustrating. Can't have multiple chats open at the same time, can't close the right rail, etc. It's UX is subpar in almost every way that matters to me. I use it because _everyone_ uses it, not because I want to.

You may enjoy Ripcord if you're not happy with Discord's UI. I've been using it for a few months, and it's made Discord enjoyable to use. I do have to open the official client whenever I do voice calls though, because there's currently an issue that can cause incoming audio to sound terrible. But for text chat, it's great. https://cancel.fm/ripcord/

It would help if discord would stop threatening to ban people for using third party clients.

Re: How Discord Stores Billions of Messages (2017)

#195

Post by gunapologist99 »

sa1 wrote: post #105
derefr wrote: post #79

Earlier quoted context omitted.

-

https://twitter.com/discord/status/1229357198918197248?lang=...

For reference, this tweet from @discord says:

"All 3rd party apps or client modifiers are against our ToS, and the use of them can result in your account being disabled. I don't recommend using them."

Re: How Discord Stores Billions of Messages (2017)

#196

Post by Rd6n6 »

RHSeeger wrote: post #36
tester756 wrote: post #2

Discord had like $300M invested and they created unparalleled piece of software that ate whole market, damn. One of the most impressive softwares that I've seen and use after years of using ventrilo/mumble/teamspeak.

Honestly, I find discord super frustrating. Can't have multiple chats open at the same time, can't close the right rail, etc. It's UX is subpar in almost every way that matters to me. I use it because _everyone_ uses it, not because I want to.

In indie game dev, everyone’s on discord and so are your extremely-important potential players. Whether you enjoy discord or dislike it, you’re going to be on there. It has really strong networks effects

Re: How Discord Stores Billions of Messages (2017)

#197

Post by yeswecatan »

PeterCorless wrote: post #191
yeswecatan wrote: post #20

Earlier quoted context omitted.

I admittedly know very little about this space. What are some of the newer players?

If you are not familiar, DB-engines.com keeps a listing of who the major players are in the database world. https://db-engines.com/en/ranking MongoDB is ranked #5 on the list at present; Cassandra comes in at #11. (And Scylla, which they moved to most of their workload from Cassandra, is currently #88.) DB-engines also have specific rankings for what are known as 'NoSQL wide column stores' — which is what Cassandra a…

This is excellent. Thank you!

Re: How Discord Stores Billions of Messages (2017)

#198

Post by golergka »

bcrosby95 wrote: post #120
NotAnOtter wrote: post #7

This is like a masterclass in how to answer system design questions. Maybe a bit verbose. They cover requirements, how to answer those requirements, relevant tech for the problem, implementation, and techniques for maintenance

Are you saying that one person, without consulting other engineers, or having to do any research, made this decision in less than a day? Because I have some bad news for you. The sentiment you express here is why interviewing is so shit these days.

Well, that's true for any task that you're asked to complete at an interview: you're doing the fast, draft version that is obviously not comparable in quality to what you would do in a real setting. But still, even this draft could be illuminating.

Re: How Discord Stores Billions of Messages (2017)

#199

Post by ThatMedicIsASpy »

Dylan16807 wrote: post #194
CoryAlexMartin wrote: post #186

Earlier quoted context omitted.

You may enjoy Ripcord if you're not happy with Discord's UI. I've been using it for a few months, and it's made Discord enjoyable to use. I do have to open the official client whenever I do voice calls though, because there's currently an issue that can cause incoming audio to sound terrible. But for text chat, it's great. https://cancel.fm/ripcord/

It would help if discord would stop threatening to ban people for using third party clients.

I've ditched the client for web only with custom css. Also allows me to block every kind of tracking (opened programs). I only use it as a chat client and still run a TS3 server because discords audio is just garbage

Re: How Discord Stores Billions of Messages (2017)

#200

Post by imadeamistake »

PretzelPirate wrote: post #167
habibur wrote: post #161

Earlier quoted context omitted.

Reading the article I was right now visiting Cassandra site to figure out what the catch is. Surely there should be a catch. Well, here it is. The partitioning in manual upto the SQL level.

The bigger catch is that when your partition grows too big and your nodes are hit by the OOMKiller, you have very few options other than create a new table and replay data, or use a cli tool to manually partition your data while the node is offline. Using Cassandra tends to mean pushing costs to your developers instead of spending more money on storage resources, and your devs will almost certainly spend a ton of tim…

Cassandra won't try and load a partition into memory. It doesn't work that way. The only way you would get behavior like that is by setting "allow filtering" to on. Allow filtering is a dedicated keyword for "I know I shouldn't do this but I'm going to anyway". If you're trying to run those types of queries, use a different database. If someone is making you use a transactional database without joins for analytical load, get a different job because that's a nightmare.

Also, your partitions should never get that large. If you're designing your tables in such away that the partitions grow unbounded, there's an issue. There are lots of ways to ensure that the cardinality of partitions grows as the dataset grows. And you actually control this behavior by managing the partitioning. It's really easy to grok the distribution of data in on disk if you think about how it's keyed.

You've basically listed a bunch of examples of what happens when you don't use a wide columnar store correctly. If you're constantly fixing downed nodes, you're probably running the cluster on hardware from Goodwill.

This is a pretty good list of what not to do with Cassandra, or any similar database. https://blog.softwaremill.com/7-mistakes-when-using-apache-c...

Post reply on HN