How Discord Stores Billions of Messages (2017)

#1

Post by ibraheemdev » Tue, Aug 24, 2021, 5:43 PM UTC

How Discord Stores Billions of Messages (2017)
blog.discord.com

Re: How Discord Stores Billions of Messages (2017)

#2

Post by tester756 » Tue, Aug 24, 2021, 6:29 PM UTC

Discord had like $300M invested and they created unparalleled piece of software that ate whole market, damn.

One of the most impressive softwares that I've seen and use after years of using ventrilo/mumble/teamspeak.

Re: How Discord Stores Billions of Messages (2017)

#3

Post by paulryanrogers » Tue, Aug 24, 2021, 6:30 PM UTC

TLDR MongoDB then Cassandra

Re: How Discord Stores Billions of Messages (2017)

#4

Post by jamesdwilson » Tue, Aug 24, 2021, 6:31 PM UTC

With privacy concerns, companies should be shamed for storing billions of messages.

Re: How Discord Stores Billions of Messages (2017)

#5

Post by typon » Tue, Aug 24, 2021, 6:32 PM UTC

paulryanrogers wrote: post #3

TLDR MongoDB then Cassandra

Did they make the move to ScyllaDB, as mentioned in their "Future work" section?

Re: How Discord Stores Billions of Messages (2017)

#6

Post by yeswecatan » Tue, Aug 24, 2021, 6:33 PM UTC

paulryanrogers wrote: post #3

TLDR MongoDB then Cassandra

Still a good read if you're curious about Cassandra

Re: How Discord Stores Billions of Messages (2017)

#7

Post by NotAnOtter » Tue, Aug 24, 2021, 6:36 PM UTC

This is like a masterclass in how to answer system design questions. Maybe a bit verbose. They cover requirements, how to answer those requirements, relevant tech for the problem, implementation, and techniques for maintenance

Re: How Discord Stores Billions of Messages (2017)

#8

Post by vortico » Tue, Aug 24, 2021, 6:37 PM UTC

I'm curious of the 2021 measure of total disk space that Discord consumes. Servers that I'm in share images every few minutes, which must add up pretty quick.

Re: How Discord Stores Billions of Messages (2017)

#9

Post by losteric » Tue, Aug 24, 2021, 6:37 PM UTC

jamesdwilson wrote: post #4

With privacy concerns, companies should be shamed for storing billions of messages.

That is a fair concern. However, as a customer - searching my message history is a desirable feature. I would rather see meaningful individual and corporate accountability for privacy breaches. The threat of jail and/or 100MM's in fines should motivate better data handling.

Re: How Discord Stores Billions of Messages (2017)

#10

Post by dvt » Tue, Aug 24, 2021, 6:37 PM UTC

tester756 wrote: post #2

Discord had like $300M invested and they created unparalleled piece of software that ate whole market, damn. One of the most impressive softwares that I've seen and use after years of using ventrilo/mumble/teamspeak.

> ventrilo/mumble/teamspeak

To be fair, Mumble is FOSS, and Ventrilo and Teamspeak have literally not iterated since 2005. Discord is pretty mediocre software (remember when they accidentally allowed iframe XSS RCE attacks? A very amateurish mistake), but the incumbents were an absolute dumpster fire.