Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Discord Stores Billions of Messages (2017)

blog.discord.com

Post reply on HN
Print view View original

331–340 of 377 posts

Re: How Discord Stores Billions of Messages (2017)

#331

Post by bob1029 »

lmilcin wrote: post #324
jfoutz wrote: post #321

Earlier quoted context omitted.

you can handle the load or not, right? A built in maintenance window is super nice, but servers crash all the time. So, that's a problem, or you've got a system in place. if you can handle failover, you've got free maintenance windows anyway, so it seems not any more difficult?

> but servers crash all the time This is both wise and stupid at the same time. It is wise if you mean "be ready for servers to crash at any time by thinking they are going to crash at the worst possible moment". But it is stupid, because people think they need massive parallel deployments just because servers will be constantly crashing and it is just not true. The cost they pay is in having couple of times more nod…

> The reality is, servers do not crash

This is the regime we operate in as well. For our business, a failure, while really bad, is not catastrophic (we still maintain non-repudiation). We look at it like any other risk model in the market.

For many in our industry, the cost of not engineering this way and eating the occasional super rare bag of shit is orders of magnitude higher than otherwise tolerable. One well-managed server forged of the highest binned silicon is usually the cheapest and most effective engineering solution over the long term.

Re: How Discord Stores Billions of Messages (2017)

#332

Post by masklinn »

atom_arranger wrote: post #302
masklinn wrote: post #127

Earlier quoted context omitted.

> Discord uses proper markdown Not even remotely. Discord supports: * fenced code blocks (but not indentation) * quoting, a single level (nested quotes don't work, properly replying to other comments is painful) * inline decorations (italics, bold, underline, strikethrough, code) * inline spoilers (an extension) * disabling autolinking (an other extension) It doesn't support: headings, paragraphs, lists (ordered or n…

I hadn't noticed some of those. I don't use Discord a ton, just enough to know it works better than Slack I guess, at least what they do support uses markdown syntax. No bulleted lists... that's disappointing. Maybe worth upvoting: https://support.discord.com/hc/en-us/community/posts/3600400...

> Maybe worth upvoting:

My experience of the community forums is it’s only worth if you like venting or seeing people vent, the folks in charge of discord really don’t care.

Re: How Discord Stores Billions of Messages (2017)

#333

Post by Ekaros »

SilverRed wrote: post #244
jacurtis wrote: post #169

Earlier quoted context omitted.

> you can run mumble on any random Linux box in your house That seems easy to you. That would be easy for me too and most likely 90% of the people on HackerNews. But the average person doesn't have a "random Linux box" in their house. Most people don't even know what Linux is. Most people would be overwhelmed just looking for the terminal emulator on their computer, before they even typed a command into it. Most peop…

I find it astounding that people here can not even grasp the concept of why Discord is popular. I am perfectly capable of hosting my own server and doing everything manually. But it is clear as day why discord wiped out the competition while most of the comments here seem dazed by the fact and are left wondering why people don't just use IRC. It's no wonder so many projects and FOSS tools fail to gain large userbases…

Best comparison for Discord is to think it as social media site, Facebook or Reddit. Find or get invite to server, everything is there trivially. Creating your own space is simple, easy and fast. Everyone is already there.

IRC is pretty similar, but much more fragmented and not really very user-friendly.

Re: How Discord Stores Billions of Messages (2017)

#334

Post by Ekaros »

nonbirithm wrote: post #74
kccqzy wrote: post #30

Earlier quoted context omitted.

I've also noticed that in a lot of tech-related social circles people are increasingly choosing Discord over Slack. That's a trend I totally didn't expect: at least until a few years ago it was clear that Discord was for gamers and Slack was for work and everyone else. That changed quickly. Impressive indeed!

They also pivoted their marketing message away from being "for gamers" and towards anyone who wanted "a place to hang out," like developer groups or high schoolers.

Kinda makes perfect sense, Discord is very good addition to subreddit or Facebook group. And barrier of entry is low, just like those two. It fills a niche for audio and chat of large communities.

Re: How Discord Stores Billions of Messages (2017)

#335

Post by Ekaros »

vortico wrote: post #222
jjice wrote: post #203

Earlier quoted context omitted.

I don't know much about image de-deuplication, but maybe they can get some sort of fingerprint/hash for an image, see if they already have it, and then serve that already existing image. I'd imagine a hash like SHA256 would be tricky because if that image was compressed an additional time at all throughout it's internet journey, then we'd get a different resulting hash, but maybe there is an effective way to fingerpr…

I'd imagine Discord uses deduplication, but I bet it doesn't save them 5% storage space.

I think it might, spamming same meme images over and over is quite common in some servers. On other hand the bigger pictures might overhelm these just in size.

Re: How Discord Stores Billions of Messages (2017)

#336

Post by onei »

anonymoushn wrote: post #330
unchar1 wrote: post #320

Earlier quoted context omitted.

I can understand that Go has a GC so it (might) be slower. But wouldn't Rust and Zig have about the same performance?

Maybe if you use nostd. I evaluated a bunch of Rust libraries for some server software, but I could not use any of them because they pervasively assume that it is ok to make syscalls to allocate memory. If you'd like to write software that makes few syscalls in the steady state, you can do it in rust, but you can't use libraries. Or String or Vec, I guess.

Why is memory allocation via syscalls bad? I get it for embedded (which was mentioned above so perhaps that's what this targets), but I kind of assumed malloc was a syscall underneath on an actual OS and that was fine.

Re: How Discord Stores Billions of Messages (2017)

#337

Post by t0mas88 »

jfoutz wrote: post #321
enlyth wrote: post #314

Earlier quoted context omitted.

What about cryptocurrency trading which goes on continuously for 24 hours a day?

you can handle the load or not, right? A built in maintenance window is super nice, but servers crash all the time. So, that's a problem, or you've got a system in place. if you can handle failover, you've got free maintenance windows anyway, so it seems not any more difficult?

In regular trading (not crypto, see the other comment about the volume differences) it is common to tune Java for example to run GC outside of trading hours. That works if you don't allocate new heap memory in every transaction/message but instead only use the stack + pre-allocated pools.

Re: How Discord Stores Billions of Messages (2017)

#338

Post by Aeolun »

bottled_poe wrote: post #310
Aeolun wrote: post #298

Earlier quoted context omitted.

Something about the tone of the messages rubs me entirely the wrong way.

It reads like justified opinions from experience. Not seeing much emotional tone in there.

I dunno, sounds very ‘I am very smart’ to me. They may be right or they may not, but both solutions sound workable to me.

Don’t let perfect be the enemy of good, and all that. There’s enough utter garbage around to shit on.

Re: How Discord Stores Billions of Messages (2017)

#339

Post by michael_j_ward »

lmilcin wrote: post #237
phailhaus wrote: post #234

Earlier quoted context omitted.

Again, the article is not about throughput. How fast can you search across all historical trades?

As an example, there are bitemporal queries like "for the given population of trades specified by following rules, find the set of trades that met the rules at a particular point in time, based on our knowledge at another given point in time". Also trades are versioned (are a stream of business events from trading system), then have amendments (each event may be amended in the future but the older version must be pre…

Thanks for all the great insight.

Did you consider an explicitly bitemporal database like crux[0]?

[0] https://github.com/juxt/crux?

Re: How Discord Stores Billions of Messages (2017)

#340

Post by lmilcin »

jfoutz wrote: post #329
lmilcin wrote: post #324

Earlier quoted context omitted.

> but servers crash all the time This is both wise and stupid at the same time. It is wise if you mean "be ready for servers to crash at any time by thinking they are going to crash at the worst possible moment". But it is stupid, because people think they need massive parallel deployments just because servers will be constantly crashing and it is just not true. The cost they pay is in having couple of times more nod…

> 3 years in one of my past jobs and during that time we haven't lost a single packet. I. uh. I. That's amazing. Fuck, that's incredible. I'm trying to envision how I could track the success of each packet ever sent over the last 3 years. Just the metrics around that seems overwhelming. Jesus, that makes me question a lot about my personal skills. Fucking 'A man! Not losing a single packet in 3 years is. Shit I don't…

We know because that communication happens on UDP and each packet on app layer has sequence number. It is used on receiving side to rebuild sequence of packets (events from the exchange can only be understood correctly when processed in same order as generated and only if you have complete stream -- you can't process a packet until you processed the one preceding it). It is trivial to detect that we haven't missed a packet.

We had a special alert for a missing packet. To my knowledge that has never activated in production except for exchange-mandated tests (the exchange runs tests regularly to ensure every brokerage house can handle faults in communications, maximum loads, etc.)

If a packed was missed, the same data is replicated on another, independent link through another carrier.

And if this wasn't enough, if your system is down (which it shouldn't happen during trading) you can contact exchange's TCP service and request for the missing sequences. But that never happened, either.

As we really liked this pattern, we built a small framework and used it for internal communication as well including data flowing to traders' workstations.

Mind, that neither the carrier link, the networking devices or people who maintain it are cheap.

Post reply on HN