141–150 of 187 posts
Earlier quoted context omitted.
redis and mongo are the type of things i will yak shave to no ends so i don't have to deploy them in production
I’m honestly not sure what you are talking about. In my experience, Redis is super easy to run and manage in production.
Earlier quoted context omitted.
I don’t want to go ad personam on the blog author - but checking his socials he is not really experienced person. I don’t think we have anything to discuss here. He seems just to want to do cool stuff and his drop of databases seems to be because he just doesn’t know a lot of stuff there is to know. I applaud attempt and might be that his needs will be covered by what he is doing. But for everyone else yes, pick bori…
> I don’t want to go ad personam on the blog author - but checking his socials he is not really experienced person. According to LinkedIn: - Masters in CS from UPenn - 1 year as SWE at Google - 6 years as SWE at FB/Meta - 6 years running his own company When I hear "not really experienced," I think recent college grad, not someone with a Master's and 15 years of industry experience.
I once saw a project in the wild where the "database" was implemented using filesystem directories as "tables" with JSON files inside as "rows". When I asked people working on it if they considered Redis or Mongo or Postgres with jsonb columns, they just said they considered all of those things but decided to roll out their own db anyway because "they understood it better". This article gives off the same energy. I r…
I did this sort of thing recently. I felt bad doing it, I still objectively hate it, because I do know enough to know that basically I'm re-implementing what years of hardworking O/S developers have done, piecemeal. But at least I'm going in with my eyes open which feels better.
The only real mitigating factor I have is that the application is largely 'never-read' and then when reading is done, it's sequential batches. Which is not normally something databases optimise for and works okay for file-storage.
(If someone does know a lightweight database architecture that performs like this, let me know).
Decades ago, PG wrote that he didn't use a database for Viaweb, and that it seemed odd for web apps to be frontends to databases when desktop apps were not[0]. HN also doesn't use a database. That's no longer true, with modern desktop and mobile apps often using a database (usually SQLite) because relational data storage and queries turn out to be pretty useful in a wide range of applications. [0] https://www.paulgra…
HN does not use a database?! Can you expand on that? It's very surprising to me.
I think that is just written to disk as something like file41207393 when you click reply.
When the system needs an item it sees if it's cached in memeory and otherwise reads it from disk and I think that is pretty much the whole memory system. Some other stuff like user id that works in the same sort of way.
Earlier quoted context omitted.
I think it's important to understand that every startup goes through three phases: Explore, Expand, Extract. What's simple in one phase isn't simple in the other. A transactional database is simple in Expand and Extract, but adds additional overhead during the Explore phase, because you're focusing on infrastructure issues rather than product. Data reliability isn't critical in the Explore phase either, because you j…
Having Explored with a transactional database: I really can't agree. Just change your database, migrations are easy and should be something you're comfortable doing at any time, or you'll get stuck working around it for 100x more effort in the future.
I think sql db make the most sense during the explore phase and you switch off of them once you know you need an improvement somewhere (like latency or horizontal scalability).
We didn’t want to build something complicated, so we implemented our own raft consensus layer. Have you considered just using Redis?
Redis is/was basically just Tcl-typed which are persisted to disk using snapshots (Tcl commands) and append-only Tcl commands, that had a network protocol for non-Tcl applications to talk to
I once saw a project in the wild where the "database" was implemented using filesystem directories as "tables" with JSON files inside as "rows". When I asked people working on it if they considered Redis or Mongo or Postgres with jsonb columns, they just said they considered all of those things but decided to roll out their own db anyway because "they understood it better". This article gives off the same energy. I r…
> I once saw a project in the wild where the "database" was implemented using filesystem directories as "tables" with JSON files inside as "rows". I did this sort of thing recently. I felt bad doing it, I still objectively hate it, because I do know enough to know that basically I'm re-implementing what years of hardworking O/S developers have done, piecemeal. But at least I'm going in with my eyes open which feels b…
Earlier quoted context omitted.
Well he is not Kent Beck or Jon Skeet, Martin Fowler - that is what I call experienced to take seriously a blog post. Just working at Meta/Google doesn’t impress me much just like Shania Twain would sing.
> he is not Kent Beck or Jon Skeet, Martin Fowler Just FYI, you are (perhaps unintentionally) showing your lack of experience. There are many thousands of brilliant engineers for every brilliant engineer who also is a author/speaker/publisher. These are very different skills. Also, perhaps the author _is_ the next Martin Fowler? You never know...
I am also not claiming I have a way to run every other new software compan.
Earlier quoted context omitted.
They basically only save on serialization & deserialization at query time, which I would consider an infinitesimal saving in the vast majority of use cases. They claim to be able to build some magical index that's not possible with existing disk-based databases (I didn't read the linked blog post). They lose access to a nice query language and entire ecosystems of tools and domain knowledge. I fail to see how this li…
You should probably RTFA before making broad assumptions on their solution and how it works. Most of what you wrote is both incorrect and addressed in the article.
The only thing in my comment that’s not directly based on the article is a handwavy 1ms/request saving estimate, and since they don’t provide any measurement, it’s anyone’s guess.
Seems weird to start with “not talking about using something like SQLite where your data is still serialized”, then end up with a home grown transaction log that requires serialization and needs to be replicated, which is how databases are replicated anyway. If your load fits entirely on one server, then just run the database on that damn server and forget about “special architectures to reduce round-trips to your da…
I think it's important to understand that every startup goes through three phases: Explore, Expand, Extract. What's simple in one phase isn't simple in the other. A transactional database is simple in Expand and Extract, but adds additional overhead during the Explore phase, because you're focusing on infrastructure issues rather than product. Data reliability isn't critical in the Explore phase either, because you j…