Earlier quoted context omitted.
I would say operator friendliness is actually the best reason to roll your own (was clearly not the case here). If you have a system that is less complex, because it meets your use case only and not the competing use cases of every damn engineering outfit that can pay overpaid and underqualified devs to commit to an open sourced codebase, and as a result requires less labor to manage (for example, not using kubernete…
Sure, but there are different levels of "roll your own". Mysql or Pestresql + a text field and a microservice front end for access control and JSON validation (if you don't want to use the included components from those respective projects that handle those for you) is easier and friendlier most of the time than a microservice on top of sqlite on a local disk, which is friendlier than replacing sqlite with BDB, which…
Json-Base – Database built as JSON files
121–130 of 189 posts
Re: Json-Base – Database built as JSON files
#122Re: Json-Base – Database built as JSON files
#123Someone at my old company basically did this and put it into production. The first problem he encountered was that multiple connections couldn't both be using the database at a time without clobbering each other. "No problem," he thought, this is a good use case for micro services. A service sitting on top would ensure that there was only one operation being performed at a time. Next, his problem was that the databas…
https://github.com/skorokithakis/goatfish/
It's actually quite good as a quick-and-dirty datastore. I do need to move everything to SQLite's JSON field, though.
Re: Json-Base – Database built as JSON files
#124You definitely shouldn't use this in production. Looking at the code there's: - race conditions everywhere. - bad and inconsistent formatting, which doesn't help with the - huge if-else monstrosities. - Also uses synchronous IO and asynchronous IO randomly. - Uses try-catch liberally, doesn't check the caught errors, and just re-tries blindly forever in some cases. If you do any parallel updates/inserts/removals with…
Re: Json-Base – Database built as JSON files
#125Earlier quoted context omitted.
> Next, his problem was that the database would get corrupt sometimes when something bad happened in the middle of writing the file. I'm not sure i understand how this can happen... unless you try to update JSON in-place (which is a very bad idea for any text-based format), what you do is encode/write the entire JSON from scratch. So either the file is written properly or it isn't written. Honestly from the entire me…
1. A cosmic ray storm turned all ASCII charactees into ECBDIC 2. Lightning struck the 12 V feed and upped the voltage to 10 MV, turning all 0 and 1’s into 6’s 3. Someone spilled a New England Pale Ale on the server 4. The process was assinated by the mysterious killer only known from his modus operandi of leaving OOM written in blood across the syslog 5. Birds nested within the server and fed all the SATA cables to t…
1. Write your updates to a copy of the file.
2. Do an atomic rename of that copy to the original.
Re: Json-Base – Database built as JSON files
#126Someone at my old company basically did this and put it into production. The first problem he encountered was that multiple connections couldn't both be using the database at a time without clobbering each other. "No problem," he thought, this is a good use case for micro services. A service sitting on top would ensure that there was only one operation being performed at a time. Next, his problem was that the databas…
Re: Json-Base – Database built as JSON files
#127I did something vaguely similar to this recently, and I still maintain it was a good choice. I volunteered to write a medical visit recording app for an NGO in a developing country (a friend works with the NGO and asked me if I would help), and they have almost no budget, no guarantees of internet connectivity when their folks are in the field, and the likelihood that they may be using this software for years. So I w…
How did you get involved with this kind of work?
If you want the long version, or you are interested in ways you could also be involved in that kind of project (or know C# and WinForms and want to help??? :) ) my email is my hn username at gmail.
Re: Json-Base – Database built as JSON files
#128Earlier quoted context omitted.
> Next, his problem was that the database would get corrupt sometimes when something bad happened in the middle of writing the file. I'm not sure i understand how this can happen... unless you try to update JSON in-place (which is a very bad idea for any text-based format), what you do is encode/write the entire JSON from scratch. So either the file is written properly or it isn't written. Honestly from the entire me…
Here, have a read. Warning: if you haven’t done low-level development, you might walk away wondering if your entire life has been a lie. https://danluu.com/file-consistency/
Re: Json-Base – Database built as JSON files
#129I did something vaguely similar to this recently, and I still maintain it was a good choice. I volunteered to write a medical visit recording app for an NGO in a developing country (a friend works with the NGO and asked me if I would help), and they have almost no budget, no guarantees of internet connectivity when their folks are in the field, and the likelihood that they may be using this software for years. So I w…
https://github.com/cris691/stubdb
The criticism of this project is helping me refine mine, for example,
lalaland1125 15 minutes ago
File renames are atomic. This is a solved problem:
1. Write your updates to a copy of the file.
2. Do an atomic rename of that copy to the original.
Even tho I already use sync file access everywhere, I think this idea could help.
I really enjoy this project.
Re: Json-Base – Database built as JSON files
#130Earlier quoted context omitted.
> Next, his problem was that the database would get corrupt sometimes when something bad happened in the middle of writing the file. I'm not sure i understand how this can happen... unless you try to update JSON in-place (which is a very bad idea for any text-based format), what you do is encode/write the entire JSON from scratch. So either the file is written properly or it isn't written. Honestly from the entire me…
Here, have a read. Warning: if you haven’t done low-level development, you might walk away wondering if your entire life has been a lie. https://danluu.com/file-consistency/