Json-Base – Database built as JSON files • Hacker News Board

Re: Json-Base – Database built as JSON files

#81

Post by zachrip » Thu, Jul 02, 2020, 8:46 PM UTC

teej wrote: post #65

tobr wrote: post #24

So much pessimism! Not sure if the author is here, but it would be interesting to hear what makes this different from, say, Lowdb. Also, the writing in the README feels sloppy, which doesn’t inspire confidence. For example, you might want to decide if it’s called jsonbase, JSON-base, JSONBASe, Json-Base, JSON-Base, json-base or jsonDB.

If the author doesn’t provide any explanation of why this exists or what motivated them to create it, what am I supposed to assume? They’ve called it a database. They have said explicitly “ You can use this as a backend for your ReST APIs.” But it doesn’t meet the table stakes for a database and encouraging folks to use it in a production environment is actively harmful. I wish more folks were up front with the trade…

I don't believe the OP of this post are the same person as the author of the library. Someone publishing a project isn't harmful, you don't have to use it. If someone uses it and gets burned that is their fault, not the author's. If you're making this project a dep, it's your job to vet it, especially if it's a database. Just because something is OSS doesn't mean it needs to be some polished stone that meets your standards.

Re: Json-Base – Database built as JSON files

#82

Post by jack_h » Thu, Jul 02, 2020, 8:51 PM UTC

mkhalil wrote: post #75

musingsole wrote: post #53

Earlier quoted context omitted.

I can sympathize but it seems hard to argue with this developer's approach then. If it met the needs of the company, particularly to the desired level at the times these features were requested, I don't think there's a valid critique of the developer's architecture beyond iT's NoT DoNe CoRrEcTlY. And still, there's a lot to be said for keeping your developer's entertained so they stick around.

I left my last company because one of my co-devs would always do crazy hack-job things, and when I complained to them or higher-ups, the excuse was: "When?" I asked, considering she had just put up the (big) PR's and PR's ARE the time to review... My jaw dropped. Especially since I was hired on as "Lead" and had all the accountability but no actual power.

Yeah, I'm in a similar situation at the moment.

It's incredibly frustrating because during code reviews I will request changes so it's not such a broken hack job, and the response will basically be "No, it's not worth changing". At which point I'm the one "holding up development". We wasted hundreds of development hours during the last project because of this persons "inventive" code, and nobody seems to understand what's going on.

Shame the job market is a bit crap right now.

Re: Json-Base – Database built as JSON files

#83

Post by russnewcomer » Thu, Jul 02, 2020, 8:55 PM UTC

marci wrote: post #71

russnewcomer wrote: post #61

Earlier quoted context omitted.

Nope, that's too much complexity. Because then someone has to run and manage a webserver, and there is no guarantee that ServiceWorkers will work like they do in 5 years or on an ancient Windows7 laptop running IE7. I want this to be able to run for years without my intervention. :)

something that will work in 5 years on ancient windows7 and ie7? couchdb there's nothing to manage. On Windows, you install couchdb.msi or whatever, installed as a windows service, it automatically boots at startup time. Start IE7 go to localhost:5984/_utils, you get the DB's UI. At that point, all you did was installation. One click later, you created the first db called 'somedb', a click later, you created the firs…

That is a little more complex than, "Here's a .exe, it will save files into your My Documents. Click Export to make a zip, import to read someone else's zip." Plus generating instructions on how to do that would have been tough, and this way should be much easier to spread the app around.

Basically, you don't have a bad idea, and if I were a couch expert or were not on the other side of the world, I might have chosen that. But since I know C#/WinForms well enough, and if we went with the browser I would have to support mobile phones and I don't want to support mobile phones for this use case for a lot of other reasons.

Re: Json-Base – Database built as JSON files

#84

Post by kbenson » Thu, Jul 02, 2020, 8:58 PM UTC

musingsole wrote: post #53

bvinc wrote: post #49

Earlier quoted context omitted.

I tried. I held a meeting to talk about the code. I found the problems hard to predict and hard to describe. It was decided that after the meeting he would work more on making his code less hacky and more production ready. But the real answer is that our team was very siloed. No one knew what anyone else was doing. The other problem was that he was actually solving real world problems, and he was a very high performe…

I can sympathize but it seems hard to argue with this developer's approach then. If it met the needs of the company, particularly to the desired level at the times these features were requested, I don't think there's a valid critique of the developer's architecture beyond iT's NoT DoNe CoRrEcTlY. And still, there's a lot to be said for keeping your developer's entertained so they stick around.

> If it met the needs of the company, particularly to the desired level at the times these features were requested, I don't think there's a valid critique of the developer's architecture beyond iT's NoT DoNe CoRrEcTlY.

I'm pretty sure the cascading series of "his next problem" sentences implies that there were plenty of problems with the architecture that weren't identified ahead of time, and they had to encounter and then fix as a series of bugs.

> And still, there's a lot to be said for keeping your developer's entertained so they stick around.

There's a difference between keeping your developers entertained and letting them infect production with ill-conceived projects that cause problems for all those that interact with them.

This project is reimplmenting something already solved multiple times. There are many document stores, and JSON interfaces and addon to traditional RDBMS, so what was being solved here, other than letting someone scratch an itch at the expense of the division he's working in. You're better off giving him 20% time for his own projects and calling it a day if you really think entertaining your developers is important enough to warrant it.

There are times when rolling your own is useful. Generally when there's some extreme requirements for space or performance, but even that becomes rare when the area is mature and explored thoroughly. A database, even a JSON document store of some sort, is so mature that to make it worth while for one person to roll their own when it seems to need all the common features (locking, remote access, different clients), that to actually recoup the cost of building our own (much less the future cost of troubleshooting and bug fixing) is almost impossible unless you're somehow hired a genius workaholic for peanuts.

Re: Json-Base – Database built as JSON files

#85

Post by teej » Thu, Jul 02, 2020, 8:58 PM UTC

zachrip wrote: post #81

teej wrote: post #65

Earlier quoted context omitted.

If the author doesn’t provide any explanation of why this exists or what motivated them to create it, what am I supposed to assume? They’ve called it a database. They have said explicitly “ You can use this as a backend for your ReST APIs.” But it doesn’t meet the table stakes for a database and encouraging folks to use it in a production environment is actively harmful. I wish more folks were up front with the trade…

I don't believe the OP of this post are the same person as the author of the library. Someone publishing a project isn't harmful, you don't have to use it. If someone uses it and gets burned that is their fault, not the author's. If you're making this project a dep, it's your job to vet it, especially if it's a database. Just because something is OSS doesn't mean it needs to be some polished stone that meets your sta…

I agree with you. For those same reasons it’s reasonable for HN commenters to be “pessimistic” about a library with no track record and no discernible take on why it deserves to be production ready.

Re: Json-Base – Database built as JSON files

#86

Post by idclip » Thu, Jul 02, 2020, 8:59 PM UTC

bvinc wrote: post #16

Someone at my old company basically did this and put it into production. The first problem he encountered was that multiple connections couldn't both be using the database at a time without clobbering each other. "No problem," he thought, this is a good use case for micro services. A service sitting on top would ensure that there was only one operation being performed at a time. Next, his problem was that the databas…

Bless him. I love it

Re: Json-Base – Database built as JSON files

#87

Post by runawaybottle » Thu, Jul 02, 2020, 9:04 PM UTC

slantyyz wrote: post #17

This reminds me a lot of LokiJS[1], which, if I recall correctly, could optionally save the data as JSON file(s). [1] https://github.com/techfort/LokiJS

I’ve used Loki as more of an in memory db in the Browser, never with the assumption of using it as real storage.

Re: Json-Base – Database built as JSON files

#88

Post by zachrip » Thu, Jul 02, 2020, 9:07 PM UTC

teej wrote: post #85

zachrip wrote: post #81

Earlier quoted context omitted.

I don't believe the OP of this post are the same person as the author of the library. Someone publishing a project isn't harmful, you don't have to use it. If someone uses it and gets burned that is their fault, not the author's. If you're making this project a dep, it's your job to vet it, especially if it's a database. Just because something is OSS doesn't mean it needs to be some polished stone that meets your sta…

I agree with you. For those same reasons it’s reasonable for HN commenters to be “pessimistic” about a library with no track record and no discernible take on why it deserves to be production ready.

The project is 2 months old and they say nothing about its readiness for production, simply that it _could_ be used for a REST backend or similar. They also say that it could be used for a quick PoC. I'm not quite understanding how either of those claims are wrong. Why are people torching a young project that someone is releasing publicly for free? Again, if you don't want to use the project, nobody is forcing you to.

Re: Json-Base – Database built as JSON files

#89

Post by munk-a » Thu, Jul 02, 2020, 9:22 PM UTC

mkhalil wrote: post #75

musingsole wrote: post #53

Earlier quoted context omitted.

I can sympathize but it seems hard to argue with this developer's approach then. If it met the needs of the company, particularly to the desired level at the times these features were requested, I don't think there's a valid critique of the developer's architecture beyond iT's NoT DoNe CoRrEcTlY. And still, there's a lot to be said for keeping your developer's entertained so they stick around.

I left my last company because one of my co-devs would always do crazy hack-job things, and when I complained to them or higher-ups, the excuse was: "When?" I asked, considering she had just put up the (big) PR's and PR's ARE the time to review... My jaw dropped. Especially since I was hired on as "Lead" and had all the accountability but no actual power.

At our company no commits get into the trunk without going by another set of eyes. We're probably creeping up to mid-sized right now so those eyes can vary in stringency and reliability more so than they would have when it was just a handful of devs, but I think mandatory code reviews are a good habit to get into - so long as you empower every reviewer to be critical and make it clear that both the reviewer and dev are owning the code and must ensure it is acceptable during the process.

We've had that process on for quite a while, and while there are some big weaknesses and holes in it we've also adopted a principle to keep PRs as small as possible[1] with those two tools we've had some pretty reasonable success with a lot of our biggest incidents being related to times when we've made large changes or a review was skimped on.

1. Even if that isn't measure in LoC - moving a dependency and updating references to it is something I'd count as a single action - but one I'd want isolated from any logic changes.

Re: Json-Base – Database built as JSON files

#90

Post by kaiby » Thu, Jul 02, 2020, 9:27 PM UTC

bvinc wrote: post #16

Someone at my old company basically did this and put it into production. The first problem he encountered was that multiple connections couldn't both be using the database at a time without clobbering each other. "No problem," he thought, this is a good use case for micro services. A service sitting on top would ensure that there was only one operation being performed at a time. Next, his problem was that the databas…

As others have mentioned, there are a ton of off-the-shelf solutions that would have been more than adequate for this.

My question is, why didn't he go for any of the existing solutions when setting them up would've still been faster than rolling his own DB-in-a-JSON-file solution?