221–230 of 377 posts
Earlier quoted context omitted.
Data is big $$$. Slap a couple of NoSQL databases and Spark on your resume and watch the money roll in. DBAs are disappearing with managed services, though.
If you have been branding yourself as a DBA, time to lift and shift to "DevOps."
Distributed databases are pretty difficult to manage, they deserve every dollar.
Earlier quoted context omitted.
How does that matter? Do they keep track of all images on the internet?
I don't know much about image de-deuplication, but maybe they can get some sort of fingerprint/hash for an image, see if they already have it, and then serve that already existing image. I'd imagine a hash like SHA256 would be tricky because if that image was compressed an additional time at all throughout it's internet journey, then we'd get a different resulting hash, but maybe there is an effective way to fingerpr…
Earlier quoted context omitted.
I can't think of many reasons Discord "ate the whole market" besides smart marketing, honestly. It does audio rooms incredibly well, but everything else (even their developer support team) is just terrible.
People love it, marketing has nothing to do with it. I've heard any Discord when I followed open source programming project (Leela Chess Zero) and it was obvious after a few minutes why it's a fantastic fit. I moved my project there shortly after as well and it's fantastic.
Earlier quoted context omitted.
> With privacy concerns, companies should be shamed for storing billions of messages. should we shame ycombinator for storing the messages, accounts and comments on hacker news then? I am still unable to delete my account here even though the CCPA and the GDPR exists. But here we are.
yes. whataboutism.
Earlier quoted context omitted.
I don't see how that's relevant? I don't even prefer those over Discord, but I don't think it's enough of an improvement to warrant the market share it has now.
What do you prefer? The fact that you can: (1) Find a server with a simple URL (no pw needed, no port or IP, etc.) (2) Find your friends easily with a unique username and use chat as a fallback (3) Create an audio room (that scales!) that has great audio quality (doesn't "drop" calls and (4) Client that auto-updates to provide more and more features All within a few minutes is a HUGE upgrade or Ventrilo. This doesn't…
[1] I've used RethinkDB, Postgres, MongoDB, MySQL, Cassandra, CockroachDB, TimescaleDB, SSDB, and others
Earlier quoted context omitted.
I can't think of many reasons Discord "ate the whole market" besides smart marketing, honestly. It does audio rooms incredibly well, but everything else (even their developer support team) is just terrible.
Slack was/is pretty terrible too. Having every workspace require a new user is the pinnacle of idiocy. So annoying and even worse if you have different emails for different workspaces.
1) They send magic links. Pretty easy.
2) They make all known workspaces you've logged into before discoverable and allow for a one-click "add to desktop Slack" option, which makes dealing with the whole "different users" issue. And to the extent that I use different emails for different workspaces, Slack accommodates that and allows me to do so within the same desktop instance, so not really sure what the concern is there.
All this happens on the aforementioned MongoDB cluster and just two server nodes. And the two server nodes are really only for redundancy, a single node easily fits the load.
What I want to say is:
-- processing a hundred million simple transactions per day is nothing difficult on modern hardware.
-- modern servers have stupendous potential to process transactions which is 99.99% wasted by "modern" application stacks,
-- if you are willing to spend a little bit of learning effort, it is easily possible to run millions of non trivial transactions per second on a single server,
-- most databases (even as bad as MongoDB is) have a potential to handle much more load than people think they can. You just need to kind of understand how it works and what its strengths are and play into rather than against them.
And if you think we are running Rust on bare metal and some super large servers -- you would be wrong. It is a normal Java reactive application running on OpenJDK on an 8 core server with couple hundred GB of memory. And the last time I needed to look at the profiler was about a year ago.
Well... we have 3 node MongoDB cluster and are processing up to a million trades... per second. And a trade is way more complex than a chat message. Has tens to hundreds of fields, may require enriching with data from multiple external services and then requires to be stored, be searchable with unknown, arbitrary bitemporal queries and may need multiple downstream systems to be notified depending on a lot of factors…
In the big data world the "complexity" of the data doesn't really mean much. It's just bytes.
Well... we have 3 node MongoDB cluster and are processing up to a million trades... per second. And a trade is way more complex than a chat message. Has tens to hundreds of fields, may require enriching with data from multiple external services and then requires to be stored, be searchable with unknown, arbitrary bitemporal queries and may need multiple downstream systems to be notified depending on a lot of factors…