Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Discord Stores Billions of Messages (2017)

blog.discord.com

Post reply on HN
Print view View original

161–170 of 377 posts

Re: How Discord Stores Billions of Messages (2017)

#161

Post by habibur »

kerblang wrote: post #24

I've used cassandra quite a bit and even I had to go back and figure out what this primary key means: ((channel_id, bucket), message_id) The primary key consists of partition key + clustering columns, so this says that channel_id & bucket are the partition key, and message_id is the one and only clustering column (you can have more). They also cite the most common cassandra mistake, which is not understanding that yo…

Reading the article I was right now visiting Cassandra site to figure out what the catch is. Surely there should be a catch.

Well, here it is. The partitioning in manual upto the SQL level.

Re: How Discord Stores Billions of Messages (2017)

#164

Post by tester756 »

nnncdrj wrote: post #47
yelnatz wrote: post #14

Earlier quoted context omitted.

Well it solved a lot of pain points with the target market. I remember my friends and I kept bickering who would pay for this month's bill for the vent/mumble servers. That kept on for years until I had enough and hosted my own in a droplet in digital ocean. None of my friends knew how to do that since they're not very technical. Discord you just had to click a couple buttons and its free.

I don't get it, you can run mumble on any random Linux box in your house, you don't need to pay to have it hosted somewhere. Works find running on any box on your desk. Discord makes you the product. It's gratis in exchange for letting them spy on you. If you don't know why that's bad...

>you can run mumble on any random Linux box in your house, you don't need to pay to have it hosted somewhere.

if you have public IP or use stuff like hamachi (at least that's how we did it decade ago)

Re: How Discord Stores Billions of Messages (2017)

#165

Post by rokhayakebe »

mavhc wrote: post #146
EamonnMR wrote: post #69

Earlier quoted context omitted.

Slack isn't free, they sell you history-in effect, you generate the data that they sell back to you.

Holding your data hostage until you pay up, isn't that ransomware?

Not if you voluntarily provided it in the first place.

Re: How Discord Stores Billions of Messages (2017)

#166

Post by chillacy »

andrewstuart wrote: post #114

I'd first reach for Postgres to do this. Anyone have any idea how Postgres would stack up in a similar challenge?

Hmm... have not tried this myself, but just brainstorming. You could shard based on discord server or chat room, which would give "read before write" consistency since writes can lock, but then you'd have to manage shards to account for varying loads like servers/rooms which grow rapidly, and deal with hot shards which might outgrow the capacity of a single db server.

Given that they said their requirements were "linear scalability, automatic failover, low maintenance, predictable performance", I don't think I'd go that route.

Re: How Discord Stores Billions of Messages (2017)

#167

Post by PretzelPirate »

habibur wrote: post #161
kerblang wrote: post #24

I've used cassandra quite a bit and even I had to go back and figure out what this primary key means: ((channel_id, bucket), message_id) The primary key consists of partition key + clustering columns, so this says that channel_id & bucket are the partition key, and message_id is the one and only clustering column (you can have more). They also cite the most common cassandra mistake, which is not understanding that yo…

Reading the article I was right now visiting Cassandra site to figure out what the catch is. Surely there should be a catch. Well, here it is. The partitioning in manual upto the SQL level.

The bigger catch is that when your partition grows too big and your nodes are hit by the OOMKiller, you have very few options other than create a new table and replay data, or use a cli tool to manually partition your data while the node is offline.

Using Cassandra tends to mean pushing costs to your developers instead of spending more money on storage resources, and your devs will almost certainly spend a ton of time fixing downed nodes.

Apple supplied some of the biggest contributors to Cassandra who were optimizing things like how to read data in a partition without fully reading the partition into memory to avoid the terrible GC cost. They put in a ton of engineering effort that probably could have been better spent elsewhere if they’d used a different database.

Re: How Discord Stores Billions of Messages (2017)

#168

Post by bluecalm »

Kaze404 wrote: post #82
RHSeeger wrote: post #36

Earlier quoted context omitted.

Honestly, I find discord super frustrating. Can't have multiple chats open at the same time, can't close the right rail, etc. It's UX is subpar in almost every way that matters to me. I use it because _everyone_ uses it, not because I want to.

I can't think of many reasons Discord "ate the whole market" besides smart marketing, honestly. It does audio rooms incredibly well, but everything else (even their developer support team) is just terrible.

People love it, marketing has nothing to do with it. I've heard any Discord when I followed open source programming project (Leela Chess Zero) and it was obvious after a few minutes why it's a fantastic fit. I moved my project there shortly after as well and it's fantastic.

Re: How Discord Stores Billions of Messages (2017)

#169

Post by jacurtis »

nnncdrj wrote: post #47
yelnatz wrote: post #14

Earlier quoted context omitted.

Well it solved a lot of pain points with the target market. I remember my friends and I kept bickering who would pay for this month's bill for the vent/mumble servers. That kept on for years until I had enough and hosted my own in a droplet in digital ocean. None of my friends knew how to do that since they're not very technical. Discord you just had to click a couple buttons and its free.

I don't get it, you can run mumble on any random Linux box in your house, you don't need to pay to have it hosted somewhere. Works find running on any box on your desk. Discord makes you the product. It's gratis in exchange for letting them spy on you. If you don't know why that's bad...

> you can run mumble on any random Linux box in your house

That seems easy to you. That would be easy for me too and most likely 90% of the people on HackerNews.

But the average person doesn't have a "random Linux box" in their house. Most people don't even know what Linux is. Most people would be overwhelmed just looking for the terminal emulator on their computer, before they even typed a command into it.

Most people don't want to manage an always-on linux box for a voice server. Most people don't want to manage port-forwarding on their firewall/router. Most people don't have static IPs at their house and wouldn't know how to setup dynamic dns to solve the problem. Most people don't even know what DNS is.

MOST PEOPLE just want a program they can launch when they want to talk to their friends. That is why Discord has been successful.

I'm not saying that's good. I am just saying that its the way the world is.

Re: How Discord Stores Billions of Messages (2017)

#170

Post by chillacy »

legerdemain wrote: post #142
yukinon wrote: post #133

Earlier quoted context omitted.

If you had to do it again, what would you choose instead?

These days? Redshift or B̶i̶g̶Q̶u̶e̶r̶y̶ BigTable (thanks for correcting my think-o!). Back then? Maybe HBase.

Do you mean BigTable? That's Google's "HBase" in sofar as hbase is based on the BT paper.

From what I recall from using it a few years ago, it's pretty damn fast, very low latency. HBase had speedy p50s as well but tended to get quite slow at p99 due to GC.

Post reply on HN