Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Spanner's Most Surprising Revelation: NoSQL is Out and NewSQL is In

highscalability.com

Post reply on HN
Print view View original

41–50 of 82 posts

Re: Google Spanner's Most Surprising Revelation: NoSQL is Out and NewSQL is In

#41

Post by ckuijjer »

sigil wrote: post #15

Buzzword headline aside, the Spanner paper is great and worth your time. As is the BigTable paper, the Dremel paper, and the Paxos Made Live paper. I read the Google whitepapers and wonder, is there anywhere else one can go to work on real solutions to distributed systems problems? At smaller scales you can cheat -- you don't need Paxos, you can get away with non-consensus-based master / slave failover. You can play…

I've downloaded them all, now I should really start reading them. Here are some direct pdf links of the papers:

- Spanner: http://research.google.com/archive/spanner-osdi2012.pdf

- BigTable: http://research.google.com/archive/bigtable-osdi06.pdf

- Dremel: http://research.google.com/pubs/archive/36632.pdf

- Paxos made live: http://www.eecs.harvard.edu/cs262/Readings/paxosmadelive.pdf

Re: Google Spanner's Most Surprising Revelation: NoSQL is Out and NewSQL is In

#42

Post by rbranson »

jbigelow76 wrote: post #40
exabrial wrote: post #27

So... transactions are cool again? Thank goodness!

Transactions have always been cool, it's just that business that don't need to them have always been cooler in the in the world of tech journalism so they don't get much mention. If something goes wrong at Instagram nobody really cares if you they don't roll back posting the picture of your pastrami sandwich just because the #lolcatz tag got applied by mistake.

I'd love to have actual distributed transactions that could scale indefinitely and not create availability issues. We actually get a steady stream of user complaints about inconsistencies between counter caches and what appears in results. Worse is the inconsistencies that can happen between graph edges that you want to partition in two different manners (eg. following vs followers).

Re: Google Spanner's Most Surprising Revelation: NoSQL is Out and NewSQL is In

#45

Post by josephcooney »

efuquen wrote: post #19
cgs1019 wrote: post #13

This article comes across as really cynical and entirely lacking in the kind of rigor and detail I have previously found on highscalability. Spanner is really mind-blowingly cool tech. I thought this article was much more informative and worth the time to read: http://news.ycombinator.com/item?id=4562546

"Another complicating factor is that as Masters of Disk it’s not surprising Google ..." Masters of Disk? He seriously that that was a good line? One of the many things that annoyed me about that post. Thanks for the links.

"He seriously that that was a good line?"

Ironic.

Re: Google Spanner's Most Surprising Revelation: NoSQL is Out and NewSQL is In

#46

Post by apu »

josephcooney wrote: post #45
efuquen wrote: post #19

Earlier quoted context omitted.

"Another complicating factor is that as Masters of Disk it’s not surprising Google ..." Masters of Disk? He seriously that that was a good line? One of the many things that annoyed me about that post. Thanks for the links.

"He seriously that that was a good line?" Ironic.

Muphry's law

Re: Google Spanner's Most Surprising Revelation: NoSQL is Out and NewSQL is In

#47

Post by espeed »

sigil wrote: post #15

Buzzword headline aside, the Spanner paper is great and worth your time. As is the BigTable paper, the Dremel paper, and the Paxos Made Live paper. I read the Google whitepapers and wonder, is there anywhere else one can go to work on real solutions to distributed systems problems? At smaller scales you can cheat -- you don't need Paxos, you can get away with non-consensus-based master / slave failover. You can play…

Look at Titan (http://thinkaurelius.github.com/titan), a new distributed OLTP graph database that has a storage layer that adds distributed transactions to pluggable backends, such as HBase and Cassandra. It's by the team the created Tinkerpop Blueprints and Gremlin, the graph traversal language.

You can read more about it in Matthias's PhD dissertation (http://www.knowledgefrominformation.com/category/publication...).

Also see Calvin:

"Calvin can run 500,000 transactions per second on 100 EC2 instances in Amazon’s US East (Virginia) data center, it can maintain strongly-consistent, up-to-date 100-node replicas in Amazon’s Europe (Ireland) and US West (California) data centers---at no cost to throughput."

"Calvin is designed to run alongside a non-transactional storage system, transforming it into a shared-nothing (near-)linearly scalable database system that provides high availability and full ACID transactions. These transactions can potentially span multiple partitions spread across the shared-nothing cluster. Calvin accomplishes this by providing a layer above the storage system that handles the scheduling of distributed transactions, as well as replication and network communication in the system. The key technical feature that allows for scalability in the face of distributed transactions is a deterministic locking mechanism that enables the elimination of distributed commit protocols."

http://cs.yale.edu/homes/thomson/publications/calvin-sigmod1...

And Omid (https://github.com/yahoo/omid) is another somewhat similar system, but it only works with HBase. Here's a comparison from the Omid team: https://groups.google.com/d/msg/omid-project/BTue2jAH1iQ/ZP3...

Post reply on HN