Partition Key selection aside, if you want a better Cassandra, look at ScyllaDB. Much, much better engine, imho.
/disclaimer/ I used to work at Scylla.
How Discord Stores Billions of Messages (2017)
301–310 of 377 posts
Re: How Discord Stores Billions of Messages (2017)
#302Earlier quoted context omitted.
Slack uses a language that is like markdown, but is not markdown, which is annoying for engineers. Slack does not support syntax highlighting of code blocks. Discord uses proper markdown and supports syntax highlighting. These are two things that make me think Discord is better specifically for engineers, aside from it just being generally way better.
> Discord uses proper markdown Not even remotely. Discord supports: * fenced code blocks (but not indentation) * quoting, a single level (nested quotes don't work, properly replying to other comments is painful) * inline decorations (italics, bold, underline, strikethrough, code) * inline spoilers (an extension) * disabling autolinking (an other extension) It doesn't support: headings, paragraphs, lists (ordered or n…
I hadn't noticed some of those. I don't use Discord a ton, just enough to know it works better than Slack I guess, at least what they do support uses markdown syntax.
No bulleted lists... that's disappointing.
Maybe worth upvoting:
https://support.discord.com/hc/en-us/community/posts/3600400...
Re: How Discord Stores Billions of Messages (2017)
#303Well... we have 3 node MongoDB cluster and are processing up to a million trades... per second. And a trade is way more complex than a chat message. Has tens to hundreds of fields, may require enriching with data from multiple external services and then requires to be stored, be searchable with unknown, arbitrary bitemporal queries and may need multiple downstream systems to be notified depending on a lot of factors…
> -- if you are willing to spend a little bit of learning effort, it is easily possible to run millions of non trivial transactions per second on a single server, I got into programming through the Private Server (gaming) scene. You learn that the more you optimize and refactor your code to be more efficient, the more you can handle on less hardware, including embedded systems. So yeah, it's amazing how much is waste…
Go and Rust won't help there, but Zig might.
Re: How Discord Stores Billions of Messages (2017)
#304If you can, use Scylla over Cassandra. The performance difference is tremendous in my experience and replacing can be trivial (easier if you start with Scylla on day 0)
Re: How Discord Stores Billions of Messages (2017)
#305Earlier quoted context omitted.
I don't think it would be appropriate for me to say very specifically, but I suspect about double what a software engineer with the same amount of experience would earn.
Data is big $$$. Slap a couple of NoSQL databases and Spark on your resume and watch the money roll in. DBAs are disappearing with managed services, though.
Yeah, I don't know about "slap." We want you to have deep production experience with these systems. Designing them, deploying them at significant scale, predicting their pitfalls and avoiding them proactively. Diagnosing systemic problems and finding reasonable solutions.
If you can't magically put out production fires, on huge high-throughput systems, potentially in the dead of night, we are unlikely to pay you $300-400K.
Re: How Discord Stores Billions of Messages (2017)
#306Earlier quoted context omitted.
Well yeah. Though Vent, Teamspeak and Mumble never had these issues (if you could trust the server admin). Skype (at the time, no idea now) was a very shoddily written piece of software. It was trivial to query the IP of any online user, even if they were not on your contact list or appearing offline. You had to use a VPN or carefully conceal your Skype ID, I did work with a somewhat popular live streamer back then (…
Skype wasn't shoddy per se, but it was extremely aggressively peer-to-peer, which resulted in exactly this kind of exposure of connection information.
I'd say exposing the info of any user, even if they're not on your contact list, is pretty shoddy.
Re: How Discord Stores Billions of Messages (2017)
#307Earlier quoted context omitted.
Honestly, I find discord super frustrating. Can't have multiple chats open at the same time, can't close the right rail, etc. It's UX is subpar in almost every way that matters to me. I use it because _everyone_ uses it, not because I want to.
Samesies. It's not bad per se but there's plenty of crap in there. The shortcuts situation is absolutely dreadful for one, I don't understand how gamers can cope with it: * there are all of 5 actions you can bind to custom shortcuts * discord defines dozens of built-in shortcuts you can not rebind or disable, if any of those conflicts with something you need you better hope the OS has a way of overriding it Large cha…
Slack sold for approximately $30B and it has a lot of the same limitations (or maybe I'm just missing something extremely simple). I think Discord is doing pretty great job all things considered. By focusing on gamers, they unintuitively created a great product for casual and business users.
Re: How Discord Stores Billions of Messages (2017)
#308Discord had like $300M invested and they created unparalleled piece of software that ate whole market, damn. One of the most impressive softwares that I've seen and use after years of using ventrilo/mumble/teamspeak.
Honestly, I find discord super frustrating. Can't have multiple chats open at the same time, can't close the right rail, etc. It's UX is subpar in almost every way that matters to me. I use it because _everyone_ uses it, not because I want to.
I'm no fan of discord or Slack...but the fact that companies are using slack, and the slack 'threading' is sooo bad, drive me bonkers...
Re: How Discord Stores Billions of Messages (2017)
#309Earlier quoted context omitted.
So what do you think happened? That people were manipulated in to using discord? Or that they don't know what the alternatives are? Everyone I have spoken to loves discord and thinks it is one of the best programs they have. It's only a select group of hacker news style users who complain about minute details the average person does not care about. I know it's hard for most people on this site to understand but the a…
What I think happened, as someone who's been using Discord daily since 2015, is that they came up with a slightly better product than the alternatives, spent enough in marketing (to gamers specifically) to convince investors that it was a platform worth investing in, and only then slowly started improving their faulty software. To say people were manipulated into using Discord is obviously not true, but it's also dis…
The last part of the comment was probably a reply to this: "I don't see how that's relevant? I don't even prefer those over Discord, but I don't think it's enough of an improvement to warrant the market share it has now."
You don't see how those valuable those improvements are but the average person does and that's why it has a massive market share.
Re: How Discord Stores Billions of Messages (2017)
#310Earlier quoted context omitted.
I'm not sure why you're being downvoted when you're a domain expert talking about your craft? People have weird hangups on hacker news, it seems
Something about the tone of the messages rubs me entirely the wrong way.
It reads like justified opinions from experience. Not seeing much emotional tone in there.