Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Immutability, MVCC, and garbage collection

xaprb.com

Post reply on HN
Print view View original

21–30 of 48 posts

Re: Immutability, MVCC, and garbage collection

#21

Post by greatsuccess »

catnaroek wrote: post #13
greatsuccess wrote: post #11

Id like to keep immutability and databases separate in this comment. Immutability the supposed "big advantage" of functional languages, is a discussion which is one-way. No one ever discusses the implications of a system which is constantly, needlessly, insanely doing nothing but MAKING COPIES OF DATA. This is not how systems are/should be designed and Im sure as hell not going to use a functional language until func…

> Immutability the supposed "big advantage" of functional languages No, immutability is just the means. The objective is writing programs that are easier to reason about. > No one ever discusses the implications of a system which is constantly, needlessly, insanely doing nothing but MAKING COPIES OF DATA. Just because copy assignment requires, well, copying in languages like C++, it does not mean things work the same…

You are correct, thats easy to reason about unless you are writing the garbage collector, in which case its pointless to write a garbage collector because there will never be anything to collect and memory will expand infinitely.

As for the persistent data structure comments, that has nothing to do with the fact that every function in a functional program has to copy data at such a ridiculously fined grained level that all you are doing is writing a garbage creator.

This data that every function creates in the name of immutability is meaningless, void of any purpose and is a bug.

Re: Immutability, MVCC, and garbage collection

#22

Post by [deleted] »

talaketu wrote: post #14
greatsuccess wrote: post #11

Id like to keep immutability and databases separate in this comment. Immutability the supposed "big advantage" of functional languages, is a discussion which is one-way. No one ever discusses the implications of a system which is constantly, needlessly, insanely doing nothing but MAKING COPIES OF DATA. This is not how systems are/should be designed and Im sure as hell not going to use a functional language until func…

> No one ever discusses the implications of a system which is constantly, needlessly, insanely doing nothing but MAKING COPIES OF DATA. see persistent data structure

[deleted]

Re: Immutability, MVCC, and garbage collection

#23

Post by greatsuccess »

talaketu wrote: post #14
greatsuccess wrote: post #11

Id like to keep immutability and databases separate in this comment. Immutability the supposed "big advantage" of functional languages, is a discussion which is one-way. No one ever discusses the implications of a system which is constantly, needlessly, insanely doing nothing but MAKING COPIES OF DATA. This is not how systems are/should be designed and Im sure as hell not going to use a functional language until func…

> No one ever discusses the implications of a system which is constantly, needlessly, insanely doing nothing but MAKING COPIES OF DATA. see persistent data structure

I don't believe in this data structure. Its ill informed Rich Hickey nonsense and it doesnt scale and has no purpose but to seem clever.

Re: Immutability, MVCC, and garbage collection

#24

Post by DigitalJack »

jeffdavis wrote: post #7
gfodor wrote: post #5

I'm going to go out on a limb here and use a logical fallacy, but I have a feeling that Rich Hickey probably has some understanding of the history of database theory and didn't just build Datomic out of ignorance.

But he also does not have infinite engineering resources at his disposal. I expect that a lot of the designs are probably somewhat simplistic compared to the battle-hardened implementations in common use today. It doesn't mean Rich Hickey was wrong; merely that we are seeing only the ultra-clean design now. After Datomic takes off and has a lot of production users in demanding environments for 5-10 years, I doubt the…

I thought he codesigned it with a company called Relevance

Re: Immutability, MVCC, and garbage collection

#25

Post by TheHydroImpulse »

greatsuccess wrote: post #23
talaketu wrote: post #14

Earlier quoted context omitted.

> No one ever discusses the implications of a system which is constantly, needlessly, insanely doing nothing but MAKING COPIES OF DATA. see persistent data structure

I don't believe in this data structure. Its ill informed Rich Hickey nonsense and it doesnt scale and has no purpose but to seem clever.

> Its ill informed Rich Hickey nonsense

How so?

> it doesnt scale

I take it that if many of Clojure's primitives are built using this data structure, it doesn't scale either? Yet some else mentioned how performant it actually is, given that it doesn't copy everything, but merely shares commonality.

Given your past comments, this isn't the first time you say something without knowing much about it.

Re: Immutability, MVCC, and garbage collection

#26

Post by noelwelsh »

The author ends with "Also, if I’ve gone too far, missed something important, gotten anything wrong, or otherwise need some education myself, please let me know so I can a) learn and b) correct my error." so allow me to offer the following:

When you spend a substantial portion of your post sniping at the competition you come across as an arrogant arse. For example, second sentence in the post is "My overall impressions of Datomic were pretty negative, but this blog post isn’t about that." Then remove this line! It adds nothing to the point you claim you want to make. Same thing in the next section regarding append-only B-trees, and then we have snark directed at RethinkDB, and so on. I got about half-way through the post before I just skimmed to the end as I was sick of the arrogance and insults, and I was beginning to doubt your authority to comment on this area as a result -- if you have substantial points to make you don't need to cover them up with this crap.

Re: Immutability, MVCC, and garbage collection

#27

Post by jasonwatkinspdx »

jeltz wrote: post #12
homerowilson wrote: post #9

Earlier quoted context omitted.

"By way of simple argument: would anyone advise using a source control system that only kept the last few hundreds of commits? Why do we treat our data differently?" Well said!

Git is not append only though. It has a stop the world GC which compacts data and removes objects not referenced by any current branch, tag or similar object. So even if you want to keep the history you may not want to keep everything.

It's important to make a distinction between semantic and physical datasets. Git is append only (or more properly, strictly accumulative in an information sense) from a semantic perspective. From a user perspective we don't really care what the packfiles etc look like. Git also will forget timelines (transitive closure of a reference) you don't care about if you tell it to.

So it's important to distinguish purely persistent data structures in the sense of Okasaki et all, and append only datasets in the semantic sense. Git still qualifies as the latter even though it's implementation uses mutability as appropriate.

This is a good example of where the OP's critique is painting with too broad a brush. If we think from first principles: indexes are derived state, not primary state. The value log is the ultimate authority, the index can be regenerated at will. There is a huge design space of indexing over append only logs that remains largely unexplored.

I personally strongly believe databases will move emphatically in this direction because the benefits are so great. Given the typical tiered web application, wouldn't you like it if when you get an exception notification instead of just a text blob of a stack trace you can resume a continuation of the state of the system as the user saw it at the time of the error? Navigate forward and backward in time at will, even with nonlinear jumps? Edit the source code of the past and then replay toward the future to see if it resolves or reproduces the exception? There are huge opportunities here.

I will admit it's not immediately clear how to implement a database that reflects these ideas. It's going to take some experimentation and learning. That'll involve some failed experiments. What bugged me most about the OP was the attitude of "we're done, shut up with this other stuff." We are emphatically not done. It's ironic to note that at the moment MySQL was birthed it faced the same FUD style criticism.

Re: Immutability, MVCC, and garbage collection

#28

Post by jasonwatkinspdx »

RyanZAG wrote: post #16
homerowilson wrote: post #9

Earlier quoted context omitted.

"By way of simple argument: would anyone advise using a source control system that only kept the last few hundreds of commits? Why do we treat our data differently?" Well said!

What? Come on, really? My database - only the latest version of all the data that gets updated frequently - has TB of data. My whole git repo with full history is about 500mb and that's excluding large assets. It's a completely different ball game. Sure, if I could buy 100TB disks cheaply and set it up so I could access data across hundreds of them in real time then there wouldn't be a problem. Obviously, I can't do…

When talking about architecture at this level, you need to take a longer view. Once upon a time, a gigabyte database was mammoth beyond thought. Also, you're taking it as given that your data has its present storage footprint even though alternative architectures might have very different properties. It's worth noting that a purely additive database can be very aggressive about compression.

But also, petabyte datasets are downright cheap to deal with these days. They'll look more like that half gigabyte source code repo sooner than I think you're considering.

Re: Immutability, MVCC, and garbage collection

#29

Post by jasonwatkinspdx »

jeffdavis wrote: post #6
jasonwatkinspdx wrote: post #2

Pretty disappointing critique. There are manifold differences between couchdb, datomic, rethinkdb and more traditional sql databases, but the author can't see past his pet issue. He doesn't seem to understand the use cases, or the infrastructure differences. In terms of use cases, there are plenty of analytically datasets that are strictly monotonic. There is no opportunity to reclaim overwritten storage in this case…

I think you are making a mistake by reading it as a critique of immutability in general. Rather, I think the author is criticizing an overly-simplisitic approach to using immutability. When he describes traditional databases: "What’s happening in such a database is a combination of short-term immutability, read and write optimizations to save and/or coalesce redundant work, and continuous “compaction” and reuse of di…

I strongly disagree with the idea that temporal databases will be forced to revise themselves to the implementation designs of Postgres, MySQL, etc. In fact, I'd make the firm prediction that the paged window CoW strategy of LMDB will become dominant for these more traditional database designs, and that a new class of temporal databases will improve upon that architecture.

Re: Immutability, MVCC, and garbage collection

#30

Post by moomin »

DigitalJack wrote: post #24
jeffdavis wrote: post #7

Earlier quoted context omitted.

But he also does not have infinite engineering resources at his disposal. I expect that a lot of the designs are probably somewhat simplistic compared to the battle-hardened implementations in common use today. It doesn't mean Rich Hickey was wrong; merely that we are seeing only the ultra-clean design now. After Datomic takes off and has a lot of production users in demanding environments for 5-10 years, I doubt the…

I thought he codesigned it with a company called Relevance

Well, there used to a firm called Datomic, and one called Relevance, but they merged to Cognitect.
Post reply on HN