Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Discord Stores Billions of Messages (2017)

blog.discord.com

Post reply on HN
Print view View original

311–320 of 377 posts

Re: How Discord Stores Billions of Messages (2017)

#311

Post by streamofdigits »

jacurtis wrote: post #169
nnncdrj wrote: post #47

Earlier quoted context omitted.

I don't get it, you can run mumble on any random Linux box in your house, you don't need to pay to have it hosted somewhere. Works find running on any box on your desk. Discord makes you the product. It's gratis in exchange for letting them spy on you. If you don't know why that's bad...

> you can run mumble on any random Linux box in your house That seems easy to you. That would be easy for me too and most likely 90% of the people on HackerNews. But the average person doesn't have a "random Linux box" in their house. Most people don't even know what Linux is. Most people would be overwhelmed just looking for the terminal emulator on their computer, before they even typed a command into it. Most peop…

somehow the only real challenge for the tech community is how to wrap all that self-hosting complexity so that most people could just use it with the click of a button.

its not a technical challenge, its a moral challenge. it means doing what is good for the users even if they don't really know it

Re: How Discord Stores Billions of Messages (2017)

#312

Post by t0mas88 »

bob1029 wrote: post #272
lmilcin wrote: post #228

Well... we have 3 node MongoDB cluster and are processing up to a million trades... per second. And a trade is way more complex than a chat message. Has tens to hundreds of fields, may require enriching with data from multiple external services and then requires to be stored, be searchable with unknown, arbitrary bitemporal queries and may need multiple downstream systems to be notified depending on a lot of factors…

A million sounds impressive, but this is clearly not serialized throughput based on other comments here. Getting a million of anything to fast NVMe is trivial if there is no contention and you are a little clever with IO. I have written experimental datastores that can hit in excess of 2 million writes per second on a samsung 980 pro. 1k object size, fully serialized throughput (~2 gigabytes/second, saturates the dis…

Trading is for multiple reasons ideal for this, one is that total global ordering is a key feature (and requirement) of the domain so this "1 fast big server" thing is good. It is also quite widely known that several of the big exchange operate this model, a single sequencer application and then using multicast to transmit the outcomes of what it sees.

The other thing that is helping a lot here compared to Discord: Trading is very neatly organized in trading days and shuts down for hours between each trading day. So you don't have the issue that Discord had where some channels have low message volumes and others have high, leading to having scattered data all over the place. You can naturally partition data by day and you know at query time which data you want to have.

Re: How Discord Stores Billions of Messages (2017)

#313

Post by combyn8tor »

bottled_poe wrote: post #310
Aeolun wrote: post #298

Earlier quoted context omitted.

Something about the tone of the messages rubs me entirely the wrong way.

It reads like justified opinions from experience. Not seeing much emotional tone in there.

Well on one hand you've got engineers at a billion dollar company explaining how they've solved a problem. On the other hand you've got some random commentor on HN over-simplifying a complex engineering solution.

Re: How Discord Stores Billions of Messages (2017)

#314

Post by enlyth »

t0mas88 wrote: post #312
bob1029 wrote: post #272

Earlier quoted context omitted.

A million sounds impressive, but this is clearly not serialized throughput based on other comments here. Getting a million of anything to fast NVMe is trivial if there is no contention and you are a little clever with IO. I have written experimental datastores that can hit in excess of 2 million writes per second on a samsung 980 pro. 1k object size, fully serialized throughput (~2 gigabytes/second, saturates the dis…

Trading is for multiple reasons ideal for this, one is that total global ordering is a key feature (and requirement) of the domain so this "1 fast big server" thing is good. It is also quite widely known that several of the big exchange operate this model, a single sequencer application and then using multicast to transmit the outcomes of what it sees. The other thing that is helping a lot here compared to Discord: T…

What about cryptocurrency trading which goes on continuously for 24 hours a day?

Re: How Discord Stores Billions of Messages (2017)

#315

Post by geenat »

dragonfax wrote: post #217

KKV databases (Cassandra and DynamoDB are good examples) have a common problem with hotspots or "hot partitions". The most common mistake is to use a timestamp of any kind in the range (cluster) column. Then, whatever partition represents "today" or "this hour" ends up being the hot partition. The article mentions hot partitions becomming a problem with max partition size, but they're also a problem with scalability.…

Cockroach DB recently addressed hotspots on sequence/timestamp workloads with: https://www.cockroachlabs.com/blog/hash-sharded-indexes-unlo...

Does what it says on the tin for the primary key.

That said, hotspots are 100% the reason why Cockroach encourages UUID primary keys. The disadvantage to UUID is you want sequential data, you then need a secondary index which you'll have to bucket anyway.

Re: How Discord Stores Billions of Messages (2017)

#316

Post by yobbo »

caymanjim wrote: post #60
yelnatz wrote: post #14

Earlier quoted context omitted.

Well it solved a lot of pain points with the target market. I remember my friends and I kept bickering who would pay for this month's bill for the vent/mumble servers. That kept on for years until I had enough and hosted my own in a droplet in digital ocean. None of my friends knew how to do that since they're not very technical. Discord you just had to click a couple buttons and its free.

It's free for the same reason everything is free these days. VC funds anything that will attract a lot of users to mine data from so they can sell the data. Discord didn't do anything that was groundbreaking or even solve a problem that had no solution; they just came along during a time when investors are willing to fund a company operating at a loss for a decade until FANG buys them. Discord's a pretty good product…

> Same for Slack

The ux of Slack is essentially screen+irc implemented in JS with emotes. It enabled technical and non-technical people to use the same tool. The key to success is not technical, it's that they tailored the product to a specific group that would then lock itself in.

I didn't understand Discord's success, but comments here point that gamers couldn't find free group-voice apps at a critical time. Here again, they tailored the product to a group that would then voluntarily lock itself in.

Later, they sell the companies with valuations based on the captured user bases.

Re: How Discord Stores Billions of Messages (2017)

#317

Post by BigJono »

combyn8tor wrote: post #313
bottled_poe wrote: post #310

Earlier quoted context omitted.

It reads like justified opinions from experience. Not seeing much emotional tone in there.

Well on one hand you've got engineers at a billion dollar company explaining how they've solved a problem. On the other hand you've got some random commentor on HN over-simplifying a complex engineering solution.

Sounds to me like it's some "random commentor" who has solved a similar problem at a similar scale with a solution that's much simpler.

Re: How Discord Stores Billions of Messages (2017)

#318

Post by coldblues »

c7DJTLrn wrote: post #84

They wouldn't need to store so many if they actually let people delete their messages on account deletion. Instead, they ban many people who attempt to do so via automated scripts.

One of the big reasons I refuse to use Discord. Deleting your messages is a right every user should have. Whether it be individually or in bulk. The way it's done now just makes it more susceptible for users to be open to malicious attacks. Whether someone archives your content before you delete it, that's not of importance, that can happen on any internet medium.

Re: How Discord Stores Billions of Messages (2017)

#319

Post by jokoon »

RHSeeger wrote: post #36
tester756 wrote: post #2

Discord had like $300M invested and they created unparalleled piece of software that ate whole market, damn. One of the most impressive softwares that I've seen and use after years of using ventrilo/mumble/teamspeak.

Honestly, I find discord super frustrating. Can't have multiple chats open at the same time, can't close the right rail, etc. It's UX is subpar in almost every way that matters to me. I use it because _everyone_ uses it, not because I want to.

The top problem I have with discord is that you cannot have a list of favorite channels.

I really prefer IRC.

Re: How Discord Stores Billions of Messages (2017)

#320

Post by unchar1 »

naikrovek wrote: post #303
giancarlostoro wrote: post #281

Earlier quoted context omitted.

> -- if you are willing to spend a little bit of learning effort, it is easily possible to run millions of non trivial transactions per second on a single server, I got into programming through the Private Server (gaming) scene. You learn that the more you optimize and refactor your code to be more efficient, the more you can handle on less hardware, including embedded systems. So yeah, it's amazing how much is waste…

Go and Rust won't help there, but Zig might.

I can understand that Go has a GC so it (might) be slower. But wouldn't Rust and Zig have about the same performance?
Post reply on HN