Live data from Hacker News

Json-Base – Database built as JSON files

github.com

41–50 of 189 posts

Re: Json-Base – Database built as JSON files

#41
post #16

Someone at my old company basically did this and put it into production. The first problem he encountered was that multiple connections couldn't both be using the database at a time without clobbering each other. "No problem," he thought, this is a good use case for micro services. A service sitting on top would ensure that there was only one operation being performed at a time. Next, his problem was that the databas…

But what color was it? Mauve?

Re: Json-Base – Database built as JSON files

#42
post #16

Someone at my old company basically did this and put it into production. The first problem he encountered was that multiple connections couldn't both be using the database at a time without clobbering each other. "No problem," he thought, this is a good use case for micro services. A service sitting on top would ensure that there was only one operation being performed at a time. Next, his problem was that the databas…

But.... why?

Maybe because it was tempting: JSON is fairly easy to handle, very portable, and when you look at a JSON document, it's straightforward to think about querying it, and thus DB, although JSON is structured, and DBs are relational.

Re: Json-Base – Database built as JSON files

#44
post #18

I work on a system with a JSONB column with 11k unique paths with no schema. I created some tools to generate jsdoc which is a start, but please no.

Are there tools which will scan a body of JSON and and generate all the unique paths? Is it common to do this?

jq can do this. I had to do something similar recently with a firebase database I inherited. We had nowhere near 11k unique paths thhough.

Re: Json-Base – Database built as JSON files

#45
post #39

I did something vaguely similar to this recently, and I still maintain it was a good choice. I volunteered to write a medical visit recording app for an NGO in a developing country (a friend works with the NGO and asked me if I would help), and they have almost no budget, no guarantees of internet connectivity when their folks are in the field, and the likelihood that they may be using this software for years. So I w…

I strongly approve of this sort of thing. Using dead-simple and human-readable formats is a big win for things like this, even if it isn't architecturally "correct". It sounds like your decision was a good one for the use case you were looking at.

I made a weekly zoom quiz that stores questions: text, audio, image, video links as json files and it works fine when parsed with Gatsby.

Re: Json-Base – Database built as JSON files

#46
post #11

I don't mean this negatively, but I'm having trouble thinking of a real-world use case for this. Can anyone add some?

Looks a bit like https://github.com/typicode/lowdb , which I've used for a few production-ish usecases in the past. Specifically for storing the results/state of a set of manually-executed management scripts. The scripts needed to query the data from previous executions, do some stuff, and store the output. Think poor mans version of terraform. Everything was dumped in a git repo that was shared across a few people.…

Yeah lowdb or even https://github.com/LokiJS-Forge/LokiDB will probably be much more performant/more featureful.

Re: Json-Base – Database built as JSON files

#47
post #39

I did something vaguely similar to this recently, and I still maintain it was a good choice. I volunteered to write a medical visit recording app for an NGO in a developing country (a friend works with the NGO and asked me if I would help), and they have almost no budget, no guarantees of internet connectivity when their folks are in the field, and the likelihood that they may be using this software for years. So I w…

I strongly approve of this sort of thing. Using dead-simple and human-readable formats is a big win for things like this, even if it isn't architecturally "correct". It sounds like your decision was a good one for the use case you were looking at.

It is great until your charity gets acquihired by a big think tank/bigger charity/international aid group and most doctors/charity operators are not known for their talent at scaling software.

Re: Json-Base – Database built as JSON files

#48

I did something vaguely similar to this recently, and I still maintain it was a good choice. I volunteered to write a medical visit recording app for an NGO in a developing country (a friend works with the NGO and asked me if I would help), and they have almost no budget, no guarantees of internet connectivity when their folks are in the field, and the likelihood that they may be using this software for years. So I w…

The difference is that your DB is the FS, and each JSON file is an individual record. You're not storing each table in a single JSON document.

Re: Json-Base – Database built as JSON files

#49
post #36
post #16

Someone at my old company basically did this and put it into production. The first problem he encountered was that multiple connections couldn't both be using the database at a time without clobbering each other. "No problem," he thought, this is a good use case for micro services. A service sitting on top would ensure that there was only one operation being performed at a time. Next, his problem was that the databas…

Funny read, why did nobody stop him?

I tried. I held a meeting to talk about the code. I found the problems hard to predict and hard to describe. It was decided that after the meeting he would work more on making his code less hacky and more production ready.

But the real answer is that our team was very siloed. No one knew what anyone else was doing. The other problem was that he was actually solving real world problems, and he was a very high performer. He got stuff done. Arguing to start over a project that's already working is a difficult position to hold when talking to management.

Re: Json-Base – Database built as JSON files

#50
post #35

Earlier quoted context omitted.

I think that, oddly enough, the point of that project is to use the JSON format as the DB storage format, not as an export option. Just from the look of it (I don't know either projects), LokiJS will very likely be always faster.

Sorry for the confusion, by "save as", I didn't mean export. LokiJS has multiple persistence options, with JSON files in the filesystem being just one of them. Alternatively, you could also just use it in-memory or with IndexedDB.

Oh, I understand. I suppose it makes sense, if for instance one needs to store a bunch of parameters somewhere, it might as well be a JSON file.
Post reply on HN