Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Badger – A fast key-value store written natively in Go

open.dgraph.io

Post reply on HN
Print view View original

41–50 of 101 posts

Re: Badger – A fast key-value store written natively in Go

#41

Post by monocasa »

mirekrusin wrote: post #39

Silly question - if SSDs or even motherboard had persistent storage of (couple of) 4 KB blocks where you'd be able to fsync It seems that databases often want to persist/flush data in unfinished (4KB or 8KB) pages when they're being built, once they are full, they don't change much - once full they could be normally persisted. Another kind of pages are those that change very frequently - ie. single "root" page which…

A lot of higher end HBAs have battery backed DRAM write buffers for this reason.

Re: Badger – A fast key-value store written natively in Go

#42

Post by jkarneges »

timClicks wrote: post #29
fortytw2 wrote: post #11

off topic: flipping through the dgraph code, I noticed their licensing switch from Apache 2 to AGPLv3, anyone involved around to comment? Adding a draconic open source license is an unwise decision for an early stage database product imo ( https://open.dgraph.io/licensing is a dead link)

The AGPL for database products isn't unheard of. See also Neo4J. It makes sense from the host business's point of view. If you are the sole contributor, then you're entitled to do what you like. Moreover, you're also free to charge for commercial licences. This licencing model wouldn't be appropriate for a community-centric database, such as PostgreSQL. With many contributors to core, no one would be able to arbitrag…

> The AGPL for database products isn't unheard of. See also Neo4J.

And Mongo. Also RethinkDB until the parent company folded.

> This licencing model wouldn't be appropriate for a community-centric database, such as PostgreSQL.

I don't disagree.

It's interesting, though, how counterintuitive this is. I would think that GPL wouldn't be a problem for individual contributors (the types of participants I imagine when I think of a "community"), but for business contributors who don't want competitors to take advantage of their modifications. And yet, anything a business contributes back to an MIT/Apache project is actually less protected than a contribution to a GPL project.

Re: Badger – A fast key-value store written natively in Go

#44

Post by [deleted] »

dkarapetyan wrote: post #40
blaisio wrote: post #14

There is so much negativity in these comments! This project is really cool! I really appreciate the trend to rewrite C/C++ libraries in Go. It has always been really frustrating that hacky library wrappers for other languages leave performance and features on the table because they're either incomplete or just too hard to implement in the host language. For most of the languages out there, it is always better to have…

Was with you until you said you don't need SQL.

[deleted]

Re: Badger – A fast key-value store written natively in Go

#45

Post by bogomipz »

detaro wrote: post #34
bogomipz wrote: post #33

Earlier quoted context omitted.

Can you say what exactly is an "embedded workload"? I have seen this a few times now and tried googling but only ever come up with references to embedded systems and I'm guessing that this is not the same context. I know rocksdb is LSM-based and was built by FB to address write amplification on SSDs.

It has no server that runs independently and applications connect to, instead you integrate ( embed ) it into your program as a library.

Thanks

Re: Badger – A fast key-value store written natively in Go

#46

Post by rakoo »

marcrosoft wrote: post #9

Why not boltdb?

There are basically two usecases for LSM trees vs B+trees:

- Either you have a lot of random new writes and not so many updates, in which case LSM trees will ingest new data as fast as the disk can store them

- Or you care more about read performance, and LMDB and BoltDB will have more predictable (and arguably better) performance

InfluxDB is a timeseries database, so they write a lot of stuff and don't even read all of it. As data gets older it can be pruned efficiently with an LSM-based design, not so easily with B+tree. Dgraph on the other hand seems to sit right in the middle as it wants to be a general purpose database so there's no easy winner here. Hopefully the choice was correct for most use cases.

Re: Badger – A fast key-value store written natively in Go

#47

Post by wolco »

jkarneges wrote: post #42
timClicks wrote: post #29

Earlier quoted context omitted.

The AGPL for database products isn't unheard of. See also Neo4J. It makes sense from the host business's point of view. If you are the sole contributor, then you're entitled to do what you like. Moreover, you're also free to charge for commercial licences. This licencing model wouldn't be appropriate for a community-centric database, such as PostgreSQL. With many contributors to core, no one would be able to arbitrag…

> The AGPL for database products isn't unheard of. See also Neo4J. And Mongo. Also RethinkDB until the parent company folded. > This licencing model wouldn't be appropriate for a community-centric database, such as PostgreSQL. I don't disagree. It's interesting, though, how counterintuitive this is. I would think that GPL wouldn't be a problem for individual contributors (the types of participants I imagine when I th…

I think the problem is they never will contribute back any modifications upstream. Pick the right license for the project. Apache is for marketshare, gpl is perfect framework for community involvement, agpl is best for control

Re: Badger – A fast key-value store written natively in Go

#48

Post by ckaygusu »

dkarapetyan wrote: post #40
blaisio wrote: post #14

There is so much negativity in these comments! This project is really cool! I really appreciate the trend to rewrite C/C++ libraries in Go. It has always been really frustrating that hacky library wrappers for other languages leave performance and features on the table because they're either incomplete or just too hard to implement in the host language. For most of the languages out there, it is always better to have…

Was with you until you said you don't need SQL.

For really simplistic applications, omitting SQL is a fine choice, especially in golang you don't have the comfort of a well established ORM.

There exists libraries that do provide an ORM, but since the declarative side of golang is limited compared to languages, say Python, I find them unwieldy. Also, the current trend shuns the usage of an ORM in golang and encourages directly or indirectly writing SQL queries and interacting with the database through database/sql and its extending libraries, like jmoiron/sqlx.

Re: Badger – A fast key-value store written natively in Go

#49

Post by solidsnack9000 »

blaisio wrote: post #14

There is so much negativity in these comments! This project is really cool! I really appreciate the trend to rewrite C/C++ libraries in Go. It has always been really frustrating that hacky library wrappers for other languages leave performance and features on the table because they're either incomplete or just too hard to implement in the host language. For most of the languages out there, it is always better to have…

What negativity are you referring to? Most comments seem to be positive or something to the effect of "how do B+ and LSM compare for this use case?".

> For most of the languages out there, it is always better to have a native implemention.

I'm not sure that's true...imagine a K/V store written natively in Ruby, Python, JS... People have done what DGraph did before, in other languages -- writing log-structured databases in Haskell, &c -- but without a very large community you didn't get the level of improvement and testing that you see with something written in C and used across many languages. The JVM is an example of an environment where the "native" approach has worked out well; but is that "most of the languages"?

Re: Badger – A fast key-value store written natively in Go

#50

Post by anarazel »

jkarneges wrote: post #42
timClicks wrote: post #29

Earlier quoted context omitted.

The AGPL for database products isn't unheard of. See also Neo4J. It makes sense from the host business's point of view. If you are the sole contributor, then you're entitled to do what you like. Moreover, you're also free to charge for commercial licences. This licencing model wouldn't be appropriate for a community-centric database, such as PostgreSQL. With many contributors to core, no one would be able to arbitrag…

> The AGPL for database products isn't unheard of. See also Neo4J. And Mongo. Also RethinkDB until the parent company folded. > This licencing model wouldn't be appropriate for a community-centric database, such as PostgreSQL. I don't disagree. It's interesting, though, how counterintuitive this is. I would think that GPL wouldn't be a problem for individual contributors (the types of participants I imagine when I th…

> > This licencing model wouldn't be appropriate for a community-centric database, such as PostgreSQL.

> I don't disagree.

> It's interesting, though, how counterintuitive this is. I would think that GPL wouldn't be a problem for individual contributors (the types of participants I imagine when I think of a "community"), but for business contributors who don't want competitors to take advantage of their modifications. And yet, anything a business contributes back to an MIT/Apache project is actually less protected than a contribution to a GPL project.

Well, a lot of them want to, at least temporarily, distribute some features without releasing them. And that simply doesn't work for GPL projects, unless there's a sole owner and all external contributions are made under some form of CLA. There's a lot of open-core type projects, but in my experience they're on average less healthy than projects with multiple contributing entities.

For PostgreSQL there've been a lot of closed source forks, but a lot of them folded and/or couldn't keep up with the amount of changes and thus are based on some super old version (hello Redshift, hello Greenplum). The only ones that appear to be able to keep up are ones 1) that move more invasive changes upstream after a while and religiously rebase after every release, never delaying, or 2) move their modifications into extensions, possibly adding the necessary extension APIs to core PostgreSQL.

Post reply on HN