Live data from Hacker News

Clarification on “Call Me Maybe: MariaDB Galera Cluster”

percona.com

31–40 of 73 posts

Re: Clarification on “Call Me Maybe: MariaDB Galera Cluster”

#31
post #5

Earlier quoted context omitted.

Good point, but imho anyone that relies on transactionnal properties to ensure validity of its operation should really not be using ORMs ( at least not for the sensitive operations). You shouldn't need to read an ORM documentation to understand what kind of locking is happening at a given time. It should all be there right in your code.

This approach only works if your logic is codable in SQL format in the way this trivial balance change can be. If the change is happening in app business logic, this is always going to be a problem. The fact the balance+=25 type approach works is kind of a hack, and I don't think Aphyr is wrong to call it "corrupted" (in this specific example, in one transaction, you could say that it is inconsistent - but you only n…

Sure, my remark wasn't meant to dismiss aphyr blog. I completely agree that there is a big problem, and calling it corrupted seems valid to me as well.

It was more of a sidenote about orms and transactions in general.

Re: Clarification on “Call Me Maybe: MariaDB Galera Cluster”

#32

Am I missing something or does this not address the main issue the original article raised: The documentation is simply incorrect. It claims to support SNAPSHOT ISOLATION but does not. The company knows this and even this article says the behaviour "is totally expected". Seems like the first response should be to fix the docs and not claim capabilities beyond what's implemented. (Also it was pretty clear from the ori…

Yeah, it's pretty clear that OP missed the point, especially in that his recommended solution is to use lock hints. They really should just update their documentation.

Re: Clarification on “Call Me Maybe: MariaDB Galera Cluster”

#33
post #6
post #3

I do not quite like the usage of the word “corrupted” here. For me, the more correct word be to use is “inconsistent”. Aren't we talking about situations in which a database tracking account balances creates money out of thin air, or vaporizes it unexpectedly? I feel like Aphyr is always at pains to talk about the real-world implications of these findings --- not just how bad they are in sensitive applications, but a…

A database maker would say it isn't the database that created the money out of thin air, but the faulty application code that didn't select the right isolation level. Of course, whether it's wise for a database to default to anything except Serializable isolation is another matter.

> SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE

doesn't get much clearer?

Re: Clarification on “Call Me Maybe: MariaDB Galera Cluster”

#34
Besides being a super well-written and interesting series technically, the Call Me Maybe blogs have been very revealing as far as different organizations' response to criticism. Especially considering all of the target applications are open source, the project maintainers should be profusely thankful someone has taken the time for such thorough analysis, presumably much deeper than the maintainers themselves appear to have done at least on consistency behavior, to reveal bugs which should ultimately make it that much stronger.

Re: Clarification on “Call Me Maybe: MariaDB Galera Cluster”

#35

Edit: ignore this, it's addressing a completely different situation, and I clearly didn't read the article well enough. The code in the article writes all of the locations it reads, so true SI ought to keep you safe. My apologies! Yet another edit: huh, it seems that InnoDB in RR doesn't rollback when you write to a row that's been written since you started the transaction. TIL. -------- It's worth noting here that (…

it doesn't guarantee that the data in the DB itself has stayed constant By definition, snapshot isolation is supposed to guarantee that all reads are the consistent, committed data as of the begin transaction, and the commit will fail and rollback if any data altered within a snapshot isolation transaction was already changed. The response by Percona, unless I am reading it wrong, actually agrees that they are not tr…

EDIT: per grandparent comment edit, this comment is based on a misreading of the article and should be ignored. My apologies!

-------

> By definition, snapshot isolation is supposed to guarantee that all reads are the consistent, committed data as of the begin transaction, and the commit will fail and rollback if any data altered within a snapshot isolation transaction was already changed

Absolutely. What it doesn't guarantee is that data that you read and then use to update a different location has remained constant - which is what write skew is in the first place. What InnoDB and PostgreSQL call 'REPEATABLE READ' and Oracle calls SERIALIZABLE are in fact snapshot isolation.

edit: to clarify, Aphyr's original article describes RR as preventing basic write skew, which mainstream MVCC-based RR implementations simply don't do. Postgres does prevent it when using SSI (the SERIALIZABLE level), and lock-based implementations (RR on DB2 and SQL Server) do also.

You can argue this both ways: The MVCC-based RR implementations do conform to the (fuzzy) letter of the ANSI SQL standard law. They don't conform to Adya's formal definitions, but in fairness many of them existed before those formalisations did :-).

Re: Clarification on “Call Me Maybe: MariaDB Galera Cluster”

#36

Am I missing something or does this not address the main issue the original article raised: The documentation is simply incorrect. It claims to support SNAPSHOT ISOLATION but does not. The company knows this and even this article says the behaviour "is totally expected". Seems like the first response should be to fix the docs and not claim capabilities beyond what's implemented. (Also it was pretty clear from the ori…

Disclaimer: I work for Percona.

The docs are on the galeracluster.com page, which is owned and maintained by another company, so there's no way we could fix those. A staff member from this company (And one of the Galera authors) replied on the original 'Call me maybe' post indicating they would fix the docs, though.

I think the 'corruption vs inconsistency' debate could seem as nitpicking, but anybody who has been working long enough on databases has a very specific concept for each word, and, given transaction processing (distributed or otherwise) is such a complex topic, it does not help to use the wrong terminology.

Re: Clarification on “Call Me Maybe: MariaDB Galera Cluster”

#37

Besides being a super well-written and interesting series technically, the Call Me Maybe blogs have been very revealing as far as different organizations' response to criticism. Especially considering all of the target applications are open source, the project maintainers should be profusely thankful someone has taken the time for such thorough analysis, presumably much deeper than the maintainers themselves appear t…

I really like jerf's take on it [1]:

    How a project performs today tells you the zeroth derivative of its location.
    Looking at the commit log tells you about the first derivative. How people react
    to Call Me Maybe when its about their product gives you a lot of information
    about the second derivative.
[1] https://news.ycombinator.com/item?id=10082099

Edit: username

Re: Clarification on “Call Me Maybe: MariaDB Galera Cluster”

#38
There are many approaches to data consistency, including for example Eventual consistency, not only in databases but in any parallel programming. Different CPU architectures for example for years provided different memory consistency models with trade-offs of performance and being usable for application programmers.

It is important however behavior in this case is clearly documented.

I think there is decent documentation about Innodb describing how transactions work in Innodb http://dev.mysql.com/doc/refman/5.7/en/innodb-transaction-mo...

Galera would benefit having more clear documentation about what data consistency model exactly it provides.

At the same time Percona XtraDB Cluster, MariaDB Cluster can be used to built reliable applications assuming you're writing to their consistency model correctly.

Re: Clarification on “Call Me Maybe: MariaDB Galera Cluster”

#39

Besides being a super well-written and interesting series technically, the Call Me Maybe blogs have been very revealing as far as different organizations' response to criticism. Especially considering all of the target applications are open source, the project maintainers should be profusely thankful someone has taken the time for such thorough analysis, presumably much deeper than the maintainers themselves appear t…

I really like jerf's take on it [1]: How a project performs today tells you the zeroth derivative of its location. Looking at the commit log tells you about the first derivative. How people react to Call Me Maybe when its about their product gives you a lot of information about the second derivative. [1] https://news.ycombinator.com/item?id=10082099 Edit: username

A common response to these has been: "But you're using it wrong!!!!"

If your product is so complex or so ill-specified that full time testers can't make it work correctly, it's probably difficult for your developers to even understand or fix the problems.

We ignore how much of software development has become "it works for me under all of my default assumptions as the developer of the product—ship it," especially when faced with business deadlines and business management focused around business objectives (and maybe not so much around software quality or correctness).

Re: Clarification on “Call Me Maybe: MariaDB Galera Cluster”

#40
post #3

I do not quite like the usage of the word “corrupted” here. For me, the more correct word be to use is “inconsistent”. Aren't we talking about situations in which a database tracking account balances creates money out of thin air, or vaporizes it unexpectedly? I feel like Aphyr is always at pains to talk about the real-world implications of these findings --- not just how bad they are in sensitive applications, but a…

I think the Percona writer is focusing on "corruption" in terms of how I think most database folks imagine "corruption" - where a series of commands will erase a block of data, or make a block of data unrecoverable. I agree that "inconsistent" is probably more mechanically accurate, but since in this case the side effect of the inconsistency is that you can't trust the contents of a block of data, it seems like a dif…

Yup, inconsistency is a kind of data corruption, provided you expect it to be in a consistent state.

The emotional value of the latter is much stronger, though, and it sounds worse to the uninitiated, which is probably why Percona folks are trying to spin it that way.

Post reply on HN