Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: JSONlite – A simple, serverless, zero-configuration JSON document store

github.com

Post reply on HN
Print view View original

51–60 of 66 posts

Re: Show HN: JSONlite – A simple, serverless, zero-configuration JSON document store

#51

Post by camus2 »

dchest wrote: post #37
camus2 wrote: post #10

Serverless means anything really, a cli app is now serverless. I didn't know I was writing serverless applications for 20 years then. Serverless was already a bad buzz word, it gets more and more meaningless by the hour. By the way sqlite just got support for JSON : https://www.sqlite.org/json1.html off-topic : I'd like to see more books on database implementation for beginners. Of all the crap load of CS books that…

No, serverless database is not a recent buzz word, but a term frequently used to describe DB that doesn't need a separate server running to which clients connect: compare MySQL, PostgreSQL against BDB, LMDB, SQLite.

"serverless" means so many things it really means nothing, that's like "cloud computing" and that's the definition of a buzzword. Sqlite is an embedded database and has nothing to do with your favorite buzzword.

Re: Show HN: JSONlite – A simple, serverless, zero-configuration JSON document store

#53

Post by dchest »

camus2 wrote: post #51
dchest wrote: post #37

Earlier quoted context omitted.

No, serverless database is not a recent buzz word, but a term frequently used to describe DB that doesn't need a separate server running to which clients connect: compare MySQL, PostgreSQL against BDB, LMDB, SQLite.

"serverless" means so many things it really means nothing, that's like "cloud computing" and that's the definition of a buzzword. Sqlite is an embedded database and has nothing to do with your favorite buzzword.

Richard Hipp called SQLite "serverless" long before you started trying to be cool by hating buzzwords https://sqlite.org/serverless.html

Re: Show HN: JSONlite – A simple, serverless, zero-configuration JSON document store

#55

Post by sciurus »

nodesocket wrote: post #45
camus2 wrote: post #16

Earlier quoted context omitted.

I thought it was "serverless"? what does "sharding" have to do with anything? I'm sorry but it seems like you throw around some vocabulary that doesn't describe your project properly, to make it look bigger than it really is. That's nice marketing but it does not feel like there is much effort put into it, when I looked at your bash script.

"The word shard means a small part of a whole." The idea is breakup uuid keys into sub directories based on the first couple of letters to prevent file system performance issues. Seems like the right usage of the word from a dictionary perspective.

According to your logic, my home directory is also a sharded, serverless, zero-configuration document store. I should apply for some VC funding. ;-)

Re: Show HN: JSONlite – A simple, serverless, zero-configuration JSON document store

#56

Post by cocktailpeanuts »

edraferi wrote: post #11
mtolan wrote: post #7

I think your title might not be the best; * Serverless typically implies something accessible from more than the host you're on * It's not zero configuration, there's at least 1 configurable parameter (albeit with a sensible default) * I'm actually not even clear why this is a document store limited to JSON, other than you're piping it though a JSON python module. Having done some things like this in the past, as you…

I agree that "serverless" now mainly means "Function as a Service." The goal here is convey that there is no server component of the database, similar to SQLite and unlike PostgreSQL. What is the correct terminology to highlight this architectural distinction?

Of course there is no server, since we are talking about local. I agree with others that the title is misleading. I looked at the project for a while to realize that it's just a local command line interface to storing objects in sqlite.

While I am sure there are use cases for this, it is not correct to say this is "serverless".

By that standard you can call literally ANYTHING "serverless".

For starters, this coffee I'm drinking right now is "serverless". It does its job well, completely without a server!

Re: Show HN: JSONlite – A simple, serverless, zero-configuration JSON document store

#57

Post by tyingq »

e12e wrote: post #48
tyingq wrote: post #36

Earlier quoted context omitted.

Even if it's running on a filesystem that deals with it well, standard utilities like 'ls' don't deal with it well, as the default behavior is to sort. You end up having to look up obscure options, like 'ls -U' to disable sorting. You can also run into issues like "Argument list too long". ARG_MAX is larger on linux than it used to be, but it's pretty short on older kernels. I assume similar issues might exist on oth…

I'm not sure that's so much of a problem, I just tried: mkdir uuid; cd uuid uuid -v 4 -n 1000000 |\ while read uuid; do touch $uuid; done And ran out of inodes, but: time ls uuid|wc -l 425621 real 0m1.796s user 0m1.552s sys 0m0.240s Sure, it's not exactly stellar performance for a linear scan of ~400k keys - but it's not terrible (for various values and expectations of terrible). This is in a hyper-v vm on a Surface…

Depends on the system, and 400k is short of where things tend to go bad on mine. At 800k files I see it taking 7 seconds.

Even at 400k files, you see longer wait times if you've aliased ls to ls --color (pretty common), or use something like ls -F. Either runs stat() on every file.

Then, somewhere in the 1m+ range, it gets unusable.

Re: Show HN: JSONlite – A simple, serverless, zero-configuration JSON document store

#58

Post by e12e »

tyingq wrote: post #57
e12e wrote: post #48

Earlier quoted context omitted.

I'm not sure that's so much of a problem, I just tried: mkdir uuid; cd uuid uuid -v 4 -n 1000000 |\ while read uuid; do touch $uuid; done And ran out of inodes, but: time ls uuid|wc -l 425621 real 0m1.796s user 0m1.552s sys 0m0.240s Sure, it's not exactly stellar performance for a linear scan of ~400k keys - but it's not terrible (for various values and expectations of terrible). This is in a hyper-v vm on a Surface…

Depends on the system, and 400k is short of where things tend to go bad on mine. At 800k files I see it taking 7 seconds. Even at 400k files, you see longer wait times if you've aliased ls to ls --color (pretty common), or use something like ls -F. Either runs stat() on every file. Then, somewhere in the 1m+ range, it gets unusable.

Outputting 1m lines to the console is likely to be slow - and isn't really ls' fault?

Re: Show HN: JSONlite – A simple, serverless, zero-configuration JSON document store

#59

Post by tyingq »

e12e wrote: post #58
tyingq wrote: post #57

Earlier quoted context omitted.

Depends on the system, and 400k is short of where things tend to go bad on mine. At 800k files I see it taking 7 seconds. Even at 400k files, you see longer wait times if you've aliased ls to ls --color (pretty common), or use something like ls -F. Either runs stat() on every file. Then, somewhere in the 1m+ range, it gets unusable.

Outputting 1m lines to the console is likely to be slow - and isn't really ls' fault?

For troubleshooting or statistical reasons. It's serving as the backing data store. So piping through grep, or to wc, etc. Specifically not sending the output to the screen.

Re: Show HN: JSONlite – A simple, serverless, zero-configuration JSON document store

#60

Post by nodesocket »

CJefferson wrote: post #3

If you are going to invoke the sacred name of 'sqlite', then I expect to see rock-solid code and quality testing. Instead, there are two tiny tests, and I found a couple of issues in 30 seconds of looking at the code. EDIT: I should prove my point. 1) Race condition on calculating uuid (obviously won't be too serious for a good uuid implementation) 2) No check that file is written successfully. 3) No check when json…

Heads up, just fixed the bug where providing invalid json to set would write an empty file with a uuid filename to the data directory. Now providing invalid json to set errors with no file written to the data directory.

New version is 1.1.0 (https://github.com/nodesocket/jsonlite/releases/tag/1.1.0) if you care.

Post reply on HN