Live data from Hacker News

Json-Base – Database built as JSON files

github.com

141–150 of 189 posts

Re: Json-Base – Database built as JSON files

#141
post #29
post #10

It's all fun and games until this ends up in somebody's production environment.

And there's never a serious consequence to the ones who did it. By then they switched to a new position somewhere else. Like most prima donnas. And this shows that deep, deep down they know they are fake.

That sounds like it comes from some specific experience you had... but it's pretty uncalled for to apply it so confidently to someone you don't know. Don't be mean, right?

Actually, another view is that there's nothing wrong with tinkering and DIY. Perl, JS, Redis all came from people hacking their own solutions (as far as I know).

Also, many big software orgs build extensive internal tools themselves.

Plus, making your own stuff is a lot of fun. You should try it sometime (if you haven't already) :)

Re: Json-Base – Database built as JSON files

#142
post #65
post #24

So much pessimism! Not sure if the author is here, but it would be interesting to hear what makes this different from, say, Lowdb. Also, the writing in the README feels sloppy, which doesn’t inspire confidence. For example, you might want to decide if it’s called jsonbase, JSON-base, JSONBASe, Json-Base, JSON-Base, json-base or jsonDB.

If the author doesn’t provide any explanation of why this exists or what motivated them to create it, what am I supposed to assume? They’ve called it a database. They have said explicitly “ You can use this as a backend for your ReST APIs.” But it doesn’t meet the table stakes for a database and encouraging folks to use it in a production environment is actively harmful. I wish more folks were up front with the trade…

I think it's wrong to try to push the responsibility for other people's choices to use something or not to the person who creates that thing. And it's also very contradictory to another aspect of hacker or engineering culture which is like... someone can create a amazing free security service and then that service can be used in deplorable ways by criminals but almost nobody in this scene will admonish the creator because they realize that the creator is not responsible for how people choose to use that creation. not to mention that exact sentiments is basically universally expressed in every license that exists. so I really think it's embarrassing how such supposed criticism passes on these forums without being you know immediately dismissed as ridiculous.

It is also impractical to expect the Creator to anticipate all the use cases and potential benefits and pitfalls that people might find in those different use cases and express them.

Second it's fundamentally a violation of a boundary about choices. The people who make the choice to adopt software or not are the ones who are responsible for the technical debt or credit they allocate by making that choice.

Instead of criticizing creators for not adequately disclaiming their new products because of a hypothetical or real harm that is incurred because people choosing that, you should criticize the people selecting things for being irresponsible with the projects they are responsible for.

If your evaluation of a project is simply based on reading the readme at a superficial level then it's nobody else's fault but yours if you end up with problems with the tech that you choose.

I'm not saying you're being mean here I think this is just a misguided attempt to try to avoid technical debt but it doesn't focus on an effective way to do that. What I feel is disappointing is how this sort of criticism is often leveled at new projects as a way to dismiss or I think unfairly criticize these creations and their authors, maybe as form of "concern trolling." if I understand that term correctly.

Like, "don't use this new project in production" is sort of a tautology of "be careful about any tech that you choose that it's suitable for your use case", which is pretty obvious and I think low value thing to say, but it's often said about new projects in a way that suggests "this project is terrible and the author is bad for suggesting that people even think about using this". which I think is very toxic to a culture of creation, invention and tinkering and it's disrespectful of people who put in the effort to make something. it also encourages something which I think is harmful which is the need to think "I need to make this project perfect and bulletproof before I even think of releasing it" which I think means there's a lot of projects that could have benefited if they were appreciated at the small flame level, but maybe people are discouraged from putting them out there because of this sort of misused criticism.

even though I'm not really a fan of his I think Paul Graham said something about this point regarding startups that's like a startup is like an idea that's just being born and it's very fragile so you have to kind of protect it but it can grow into something really amazing.

Re: Json-Base – Database built as JSON files

#143
post #16

Someone at my old company basically did this and put it into production. The first problem he encountered was that multiple connections couldn't both be using the database at a time without clobbering each other. "No problem," he thought, this is a good use case for micro services. A service sitting on top would ensure that there was only one operation being performed at a time. Next, his problem was that the databas…

In every paragraph I'm expecting CTO/engineering lead/senior dev to appear and punch the guy.

:(

Re: Json-Base – Database built as JSON files

#144

Earlier quoted context omitted.

1. A cosmic ray storm turned all ASCII charactees into ECBDIC 2. Lightning struck the 12 V feed and upped the voltage to 10 MV, turning all 0 and 1’s into 6’s 3. Someone spilled a New England Pale Ale on the server 4. The process was assinated by the mysterious killer only known from his modus operandi of leaving OOM written in blood across the syslog 5. Birds nested within the server and fed all the SATA cables to t…

File renames are atomic. This is a solved problem: 1. Write your updates to a copy of the file. 2. Do an atomic rename of that copy to the original.

1. Write your updates to a copy of the file.

1a. fsync() the file to ensure the contents are durable.

2. Do an atomic rename of that copy to the original.

2a. fsync() the directory to ensure the rename is durable.

Re: Json-Base – Database built as JSON files

#145

You definitely shouldn't use this in production. Looking at the code there's: - race conditions everywhere. - bad and inconsistent formatting, which doesn't help with the - huge if-else monstrosities. - Also uses synchronous IO and asynchronous IO randomly. - Uses try-catch liberally, doesn't check the caught errors, and just re-tries blindly forever in some cases. If you do any parallel updates/inserts/removals with…

Yes, this feels very much like a weekend hackathon project rather than a finished product, even for its intended use case.

Re: Json-Base – Database built as JSON files

#146
post #137

Earlier quoted context omitted.

File renames are atomic. This is a solved problem: 1. Write your updates to a copy of the file. 2. Do an atomic rename of that copy to the original.

Renames aren't atomic on crash. https://danluu.com/file-consistency/

They are if you perform the correct sequence of fsync operations on both the file and the directories, and use a file system which is correctly implemented.

Re: Json-Base – Database built as JSON files

#147
post #109

Earlier quoted context omitted.

Yeah, commit's weren't going to trunk/master without the extra eyes/PR. The commits I was told to review if I wanted to stop thecraziness were the personal ones going to the bugfix/feature branch.

This might be a separate issue! :) Many good companies enforce a no-origin-branches policy, with rare and well-justified exceptions. Because, used as you describe, a "feature branch" is just a future massive diff in disguise (when it's eventually merged), and massive diffs are a big no-no because they're a huge pain to iterate on via code review.

Doesn't every git repo have an origin branch? What is the alternative to creating a feature branch for developing something you don't want in production until it's ready?

Re: Json-Base – Database built as JSON files

#148
post #16

Someone at my old company basically did this and put it into production. The first problem he encountered was that multiple connections couldn't both be using the database at a time without clobbering each other. "No problem," he thought, this is a good use case for micro services. A service sitting on top would ensure that there was only one operation being performed at a time. Next, his problem was that the databas…

How was he hired for that position?

Re: Json-Base – Database built as JSON files

#149
post #147
post #109

Earlier quoted context omitted.

This might be a separate issue! :) Many good companies enforce a no-origin-branches policy, with rare and well-justified exceptions. Because, used as you describe, a "feature branch" is just a future massive diff in disguise (when it's eventually merged), and massive diffs are a big no-no because they're a huge pain to iterate on via code review.

Doesn't every git repo have an origin branch? What is the alternative to creating a feature branch for developing something you don't want in production until it's ready?

Yep! Sorry, I meant that the only developed branch on origin is `master` (or whatever it’s called at your org). You can create branches locally, but pushing a local branch is strongly discouraged.

The workflow looks something like this.

git pull master; git checkout -b my-feature; ... ; git add -A; git commit

At this point, you submit the code for review, and upon approval the branch is merged into master and pushed. It’s not possible to push a commit hash to master that has not been reviewed.

If you have a feature that’s composed of many steps, you can “stack” multiple commits, and review/merge them in order.

If you want to develop the entire stack at once, you’re most likely doing something wrong (according to this culture). You can incrementally merge pieces of code to master in such a way that’s impossible for it to be deployed, and your final diff can be what makes it deployable.

Re: Json-Base – Database built as JSON files

#150
post #144

Earlier quoted context omitted.

File renames are atomic. This is a solved problem: 1. Write your updates to a copy of the file. 2. Do an atomic rename of that copy to the original.

1. Write your updates to a copy of the file. 1a. fsync() the file to ensure the contents are durable. 2. Do an atomic rename of that copy to the original. 2a. fsync() the directory to ensure the rename is durable.

Or just use a database like SQLite or Postgres that has developers dedicated to solving this problem.
Post reply on HN