Unfortunate name, excellent project :)
To elaborate on what this comment may be referring to. A Ripple (cryptocurrency) fork was recently released with the name of Stellar.
It's also the phonetical-match to an iPhone app:
Show HN: Stellar – Git for PostreSQL and MySQL
31–40 of 79 posts
Re: Show HN: Stellar – Git for PostreSQL and MySQL
#32The implications for this extend beyond backing up your database.
Imagine a world where daily time-series data can be stored efficiently: This is a lesser known use case, but it works like this: I'm a financial company and I want to store 1000 metrics about a potential customer. Maybe the number of transactions in the past year, the number of defaults, the number of credit cards, etc.
Normally I would have to duplicate this row in the database every day/week/month/year for every potential customer. With some kind of git-like storing of diffs between the row today and the row yesterday, I could easily have access to time series information without duplicating unchanged information. This would accomplish MASSIVE storage savings.
FWIW efficiently storing time series data is big problem at my company. No off the shelf solution makes this easy for us right now, and we would rather throw cheap hard disk at the problem rather than expensive engineers.
Re: Show HN: Stellar – Git for PostreSQL and MySQL
#33How does it work? Where does it breakdown? Why are these things not in the README?
Re: Show HN: Stellar – Git for PostreSQL and MySQL
#34Earlier quoted context omitted.
Suppose I'm working on a new feature branch and I run a bunch of migrations and add new data. Then I need to go make a bug fix on the master branch. Using Stellar, I could just take a snapshot before starting my feature branch, restore the DB state, do my bug fix, then go back to the DB as it was in my feature branch. I haven't tried Stellar yet, but this seems like a scenario where it would be useful. (And definitel…
You could use schemas or multiple databases for that.
True. Although then you would have to worry about changing your DB configuration whenever you change branches. You'd also have to make a copy of your database manually any time you want to start modifying it. Stellar seems like an easier solution.
Re: Show HN: Stellar – Git for PostreSQL and MySQL
#35The implications for this extend beyond backing up your database. Imagine a world where daily time-series data can be stored efficiently: This is a lesser known use case, but it works like this: I'm a financial company and I want to store 1000 metrics about a potential customer. Maybe the number of transactions in the past year, the number of defaults, the number of credit cards, etc. Normally I would have to duplica…
There are a lot of existing compression algorithms for time series data that do just this. I'm not sure how well any of these are implemented however. I think the problem is not necessarily how the data is stored, since that's fairly easy to fix with a bit of engineering effort if you're willing to write your own system. The harder part is rewriting query engines to take advantage of this sort of compression. Although ideally this could just be abstracted away by the storage layer.
Re: Show HN: Stellar – Git for PostreSQL and MySQL
#36If anyone's interested in git-like storage systems then it's work checking out Irmin [1]. Previous discussion is at [2]. Excerpt: "Irmin is a library to persist and synchronize distributed data structures both on-disk and in-memory. It enables a style of programming very similar to the Git workflow, where distributed nodes fork, fetch, merge and push data between each other. The general idea is that you want every ac…
I've spent a day looking at Irmin, trying to figure out the JSON CRUD interface, but there's just no documentation, and the irc channel is pretty much dead. The code looks good, but it's my first introduction to OCaml, coming from Clojure. Would love to see some more documentation on it.
Re: Show HN: Stellar – Git for PostreSQL and MySQL
#37How does it work? Where does it breakdown? Why are these things not in the README?
Because its not doing anything magic whatsoever.
https://github.com/fastmonkeys/stellar/blob/master/stellar/o...
Re: Show HN: Stellar – Git for PostreSQL and MySQL
#38This sort of thing is useful, but already supported by Postgres through transactional DDL. Migrations that fail will have their transaction reverted.
What if you're writing a migration that includes DML, and it fails partway through? Wouldn't it be good to be able to restore the DB state automatically and try again?
Postgres has transactional DDL and DML. If anything fails - everything is rolled back.
Re: Show HN: Stellar – Git for PostreSQL and MySQL
#39The implications for this extend beyond backing up your database. Imagine a world where daily time-series data can be stored efficiently: This is a lesser known use case, but it works like this: I'm a financial company and I want to store 1000 metrics about a potential customer. Maybe the number of transactions in the past year, the number of defaults, the number of credit cards, etc. Normally I would have to duplica…
Column-oriented databases virtually all feature this in the form of column compression (e.g. "repeat this value for the next 1000 rows"). And if you don't want column compression, they have sparse data filling/interpolation -- e.g. use the last available value from a time series. This is pretty much their bread and butter. Interpolation is essentially making the query engine smarter, so you don't end up in the situation you're apparently facing where you have to insert duplicate records purely to satisfy a simplistic join.
Back to this product (which appears to simply wholesale copy databases?), I use LVM for exactly what it is doing -- I create and rollback and access and update LVM snapshots of databases. The snapshots are instant, and in most situations the data duplications is very limited. LVM is one of the coolest, most under-appreciated facets of most Linux installs -- http://goo.gl/J2mIvG
Re: Show HN: Stellar – Git for PostreSQL and MySQL
#40Earlier quoted context omitted.
What if you're writing a migration that includes DML, and it fails partway through? Wouldn't it be good to be able to restore the DB state automatically and try again?
Postgres has transactional DDL and DML. If anything fails - everything is rolled back.
"Transnational" => "Transactional"