171–180 of 311 posts
My recent project - a replacement for CodeMaster's RaceNet, runs on flat files! https://dirtforever.net/ Just have to use locks to be careful with writes. I figured I'd migrate it to a database after maybe 10k users or so.
Neat, what happened to the original system? Last I checked multiplayer was working in DR2.
Sadly no solution for non-rooted consoles.
In order to ask this question it's important to understand the lifecycle of the data in question. If it is constantly being updated and requires "liveness" (updates are reflected in queries immediately), the simple answer is: yes, you need a database. But if you have data that is static or effectively static (data that is updated occasionally or batched), then serving via custom file handling can have its place. If t…
But then you'll get two files to join?
At some point, don't you just end up making a low-quality, poorly-tested reinvention of SQLite by doing this and adding features?
“You Aren’t Gonna Need It” - one of the most important software principles. Wait until you actually need it.
My recent project - a replacement for CodeMaster's RaceNet, runs on flat files! https://dirtforever.net/ Just have to use locks to be careful with writes. I figured I'd migrate it to a database after maybe 10k users or so.
And crashes you can exclude? Good luck!
Earlier quoted context omitted.
This always confuses me because we have decades of SQL and all its issues as well. Hundreds of experienced devs talking about all the issues in SQL and the quirks of queries when your data is not trivial. One would think that for a startup of sorts, where things changes fast and are unpredictable, NoSQL is the correct answer. And when things are stable and the shape of entities are known, going for SQL becomes a natu…
Disclaimer: I work part time on the DB team. You could also consider renting an Oracle DB. Yep! Consider some unintuitive facts: • It can be cheaper to use Oracle than MongoDB. There are companies that have migrated away from Mongo to Oracle to save money. This idea violates some of HN's most sacred memes, but there you go. Cloud databases are things you always pay for, even if they're based on open source code. • Or…
Never!
Earlier quoted context omitted.
Autonomous DB can run on-premises or in any cloud, not just Oracle's cloud. So it's not quite the same. I think PG doesn't have most of the features I named, I'm pretty sure it doesn't have integrated queues for example (SELECT FOR UPDATE SKIP LOCKED isn't an MQ system), but also, bear in mind the "postgres" cloud vendors sell is often not actually Postgres. They've forked it and are exploiting the weak trademark pro…
In the spirit of helpfulness (not pedantry) FYI "knocked up" means "impregnated". Maybe "put together"?
Yes I meant it in this sense: "If you knock something up, you make it or build it very quickly, using whatever materials are available."
https://www.collinsdictionary.com/dictionary/english/knock-u...
Earlier quoted context omitted.
I think part of it is the scale in terms of the past decade and a half... The hardware and vertical scale you could get in 2010 is dramatically different than today. A lot of the bespoke no-sql data stores really started to come to the forefront around 2010 or so. At that time, having 8 cpu cores and 10k rpm SAS spinning drives was a high end server. Today, we have well over 100 cores, with TBs of RAM and PCIe Gen 4/…
What about the microservices/serverless functions world? This was another common topic over the years, that using SQL with this type of system was not optimal, I believe the issue being the connections to the SQL database and stuff.
That said, I've leaned into avoiding breaking up a lot of microservices unless/until you need them... I'm also not opposed to combining CQRS style workflows if/when you do need micro services. Usually if you need them, you're either breaking off certain compute/logic workflows first where the async/queued nature lends itself to your needs. My limited experience with a heavy micro-service application combined with GraphQL was somewhat painful in that the infrastructure and orchestration weren't appropriately backed by dedicated teams leading to excess complexity and job duties for a project that would have scaled just fine in a more monolithic approach.
YMMV depending on your specific needs, of course. You can also have microservices call natural services that have better connection sharing heuristics depending again on your infrastructure and needs... I've got worker pools that mostly operate of a queue, perform heavy compute loads then interact with the same API service(s) as everything else.
Earlier quoted context omitted.
As soon as you need to do a JOIN, you're either rewriting a database or replatforming on Sqlite.
a) Just heard today: JOINs are bad for performance b) How many columns can (an Excel) table have: no need for JOINs
Earlier quoted context omitted.
No, when things change fast and unpredictably, NoSQL is worse than when they are well-known and stable. NoSQL gains you no speed at all in redesigning your system. Instead, you trade a few hard to do tasks in data migration into an unsurmountable mess of data inconsistency bugs that you'll never actually get into the end of. > is mostly bad design decisions and poor domain knowledge Yes, using NoSQL to avoid data mig…
If the argument for NoSQL is, “we don’t know what our schema is going to be”, stop. Stop and go ask more questions until you have a better understanding of the problem.
You need databases if you need any kind of atomicity. Doing atomic writes is extremely fragile if you are just on top of the filesystem. This is also why many databases have persistence issues and can easily corrupt on-disk data on crash. Rocksdb on windows is a very simple example a couple years back. It was regularly having corruption issues when doing development with it.
For the simple case, it isn't necessarily that fragile. Write the entire database to a temp file, then after flushing, move the temp file to overwrite the old file. All Unix filesystems will ensure the move operation is atomic. Lots of "we dump a bunch of JSON to the disk" use cases could be much more stable if they just did this. Doesn't scale at all, though - all of the data that needs to be self-consistent needs t…