Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

UnQLite - An Embeddable NoSQL Database Engine

unqlite.org

Post reply on HN
Print view View original

71–80 of 88 posts

Re: UnQLite - An Embeddable NoSQL Database Engine

#71

Post by symisc_devel »

ccera wrote: post #58
mmcclellan wrote: post #35

Earlier quoted context omitted.

No. Dr Hipp and Damien Katz were working on UnQL ( http://www.unqlspec.org/display/UnQL/Home ), which is unfortunately quite similar in name to this new UnQLite project.

Actually, Hipp has said publicly that he intended to created a new embedded database called UnQLite. It seems this developer just blatantly ripped off the exact name Hipp was planning to use. He also ripped off some of the core SQLite code (the VFS, etc.), which is legal to do since SQLite is in the public domain, but still... Not cool.

OK, actually I've emailed D. Richard Hipp a few months ago asking for permission to use the name UnQLite in a future open source project, here is a copy of Hipp's reply:

It would be good if you can make it clear on your website, somehow, that yours is an unaffiliated project. Otherwise, people might go complaining to me when they find bugs in your code. (Don't laugh - that sort of thing happens a lot.)

Other than that, you are welcomed to use the name.

You might want to have a look at the LSM storage engine that Dan Kennedy is working on for SQLite4. It is faster than the clunky and dated B-Tree used by SQLite3. It is also faster than LevelDB. And it supports nested transactions, with rollback. And concurrency. And it is more NAND-flash friendly. See http://www.sqlite.org/src4/timeline for the latest code.

Re: UnQLite - An Embeddable NoSQL Database Engine

#72

Post by symisc_devel »

jessaustin wrote: post #60
symisc_devel wrote: post #51

Earlier quoted context omitted.

From UnQLite core developer: No, Jx9 (the standalone library) is under SPL, while UnQLite is 2-clause BSD (including the Jx9 core), so there is no worry about that since the two software are developed by the same company. In other words, use UnQLite without restrictions

Care to respond to this complaint? https://news.ycombinator.com/item?id=5751928

Yep, sorry for the delay, We're quite busy.

I think this post will clarify the license issues http://unqlite.org/forum/note-on-the-licensing-situation

Re: UnQLite - An Embeddable NoSQL Database Engine

#73

Post by tracker1 »

benjamincburns wrote: post #22

As a hacker/tinkerer type programmer I can understand why it might be fun to build something like this. All of the data structure fun with none of the distributed difficulty! NoSQL! I can even build in that cool new language I've been designing in my head! But like everyone else I'm straining to see a use case. "Serverless" NoSQL? If you've got something small enough that you want an embedded datastore, why use a NoS…

I'd have to agree with you here... I mean, there are lots of interfaces for reading/writing json, and simply outputting a JSON as a UTF-8 .json.gz file seems more appropriate... especially if your data can fit into memory.

I would be more inclined to simply have my object structure in memory, using a more convenient high-level language, and do a load/dump from .json.gz files as needed. It's fast enough, and worst case scenario, I can backup/extract and read/write in any number of programming languages.

If you need multiple records, that may be easier, having an index, and using line delimited flat json structures can work as well... (breaking on \n)

This really seems like an also ran, where you could have abstracted out an SQLite db with a single table of (key VARCHAR(100) PRIMARY KEY, value VARCHAR(4000)) or something similar.

Re: UnQLite - An Embeddable NoSQL Database Engine

#74

Post by tracker1 »

brokenparser wrote: post #66

This seems backwards to me. Sure, use the right tool for the job, but still. SQL server -> noSQL / embedded SQL -> embedded noSQL. At every step there's someone going "X is hard, let's not do X" and we end up with a serverless, configuration-free, noSQL transactional database. The next logical steps are presumably the removal of transactions (who needs them, anyway?) and volatile storage (RAM is fast, let's use that)…

Well, you can do MVCC over straight blocking transactions. As for using volatile storage, see MemcacheD and Redis.

Re: UnQLite - An Embeddable NoSQL Database Engine

#75

Post by ricardobeat »

benjamincburns wrote: post #22

As a hacker/tinkerer type programmer I can understand why it might be fun to build something like this. All of the data structure fun with none of the distributed difficulty! NoSQL! I can even build in that cool new language I've been designing in my head! But like everyone else I'm straining to see a use case. "Serverless" NoSQL? If you've got something small enough that you want an embedded datastore, why use a NoS…

"serverless" databases are usually called storage engines: LevelDB, BerkeleyDB, BitCask, InnoDB. There is a (good) trend towards modular software, separating the storage engine from the server is a natural consequence.

But yeah, requiring a new obscure language pretty much obliterates any reason you'd want to use this. Lua is fast, popular, made for this exact use case and already used by Redis.

Re: UnQLite - An Embeddable NoSQL Database Engine

#77

Post by shuzchen »

tracker1 wrote: post #73
benjamincburns wrote: post #22

As a hacker/tinkerer type programmer I can understand why it might be fun to build something like this. All of the data structure fun with none of the distributed difficulty! NoSQL! I can even build in that cool new language I've been designing in my head! But like everyone else I'm straining to see a use case. "Serverless" NoSQL? If you've got something small enough that you want an embedded datastore, why use a NoS…

I'd have to agree with you here... I mean, there are lots of interfaces for reading/writing json, and simply outputting a JSON as a UTF-8 .json.gz file seems more appropriate... especially if your data can fit into memory. I would be more inclined to simply have my object structure in memory, using a more convenient high-level language, and do a load/dump from .json.gz files as needed. It's fast enough, and worst cas…

Following the thread of building upon sqlite, Yelp (along with the fabulous mrjob library) also open-sourced sqlite3dbm (http://pythonhosted.org/sqlite3dbm/), which is a document store on top of sqlite. Python only (meant to give random-access data to your EMR jobs running with mrjob), but the same approach could be applied to any other language.

Re: UnQLite - An Embeddable NoSQL Database Engine

#78

Post by benjamincburns »

continuations wrote: post #65
benjamincburns wrote: post #22

As a hacker/tinkerer type programmer I can understand why it might be fun to build something like this. All of the data structure fun with none of the distributed difficulty! NoSQL! I can even build in that cool new language I've been designing in my head! But like everyone else I'm straining to see a use case. "Serverless" NoSQL? If you've got something small enough that you want an embedded datastore, why use a NoS…

> I'm straining to see a use case. "Serverless" NoSQL? If you've got something small enough that you want an embedded datastore, why use a NoSQL variant? Serverless NoSQL embedded datastore aren't really that unusual. BerkeleyDB, Tokyo Cabinet, LevelDB are all commonly used serverless NoSQL.

I think what's happening here is that NoSQL is just an anti-category.

When I think of NoSQL, I think of something like MongoDB or Cassandra which are designed to handle tons of data under heavy load. When I think of BerkeleyDB and friends, I think local persistence. It's a true statement that these too are NoSQL stores, but, at least for me, they're not of the variety that first comes to mind.

Re: UnQLite - An Embeddable NoSQL Database Engine

#79

Post by benjamincburns »

ricardobeat wrote: post #75
benjamincburns wrote: post #22

As a hacker/tinkerer type programmer I can understand why it might be fun to build something like this. All of the data structure fun with none of the distributed difficulty! NoSQL! I can even build in that cool new language I've been designing in my head! But like everyone else I'm straining to see a use case. "Serverless" NoSQL? If you've got something small enough that you want an embedded datastore, why use a NoS…

"serverless" databases are usually called storage engines: LevelDB, BerkeleyDB, BitCask, InnoDB. There is a (good) trend towards modular software, separating the storage engine from the server is a natural consequence. But yeah, requiring a new obscure language pretty much obliterates any reason you'd want to use this. Lua is fast, popular, made for this exact use case and already used by Redis.

I probably came off too harsh. I'm quite familiar with the idea of embedded datastores, SQL and otherwise, and I didn't mean to attack the concept. I was more trying to illustrate (albeit weakly) that the marketing seems a bit off.

Like I said elsewhere, to me NoSQL tends to mean "big, distributed storage thing that isn't RDBMS, usually with an emphasis on horizontal scalability." Others have said correctly that some also take it first to mean schemaless and relax the "big" requirement. However I'm still confused on what value a general-purpose "schemaless" datastore provides in an embedded context. I write schemaless in quotes there because there's still an impedance mismatch between the format in which your application works with and the format in which the datastore works.

All of these rely on some general-purpose abstraction. But your application, if designed properly, probably makes use of a number of varying data structures. What are you gaining by pushing all of these through some abstraction? Assuming that all you're after is simple persistence, I have to ask - is writing data to disk really that hard?

That said, I've used local persistence libraries plenty, usually as a persistent local cache, and/or as a tool to enforce a schema which allows for data migration between constantly updating versions of software.

Re: UnQLite - An Embeddable NoSQL Database Engine

#80

Post by est »

DonnyV wrote: post #36

I would love an embeddable nosql database engine but this isn't it. I'm looking for something like MongoDB but embeddable. RaptorDB comes close but it doesn't have full support for Mono yet. https://raptordb.codeplex.com/ Can anyone recommend one?

https://github.com/Softmotions/ejdb

exactly what you wanted.

Post reply on HN