Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Riak and the demise of Basho

lists.basho.com

Post reply on HN
Print view View original

71–80 of 102 posts

Re: Riak and the demise of Basho

#71

Post by wjossey »

kstrauser wrote: post #69
wjossey wrote: post #59

In 2012, I worked on a nine month project to migrate a fairly large hot dataset (10s of TBs) off of SimpleDB and onto Riak. At the time, we were experiencing tens of hours of downtime a quarter, much of which could be attributed to SDB. My favorite story about that migration is that four months into the project, we were moving along nicely and ready to start cutting over to Riak after the Christmas holiday. My office…

> I received a call from a SDB support team member informing me that we either needed to move off their platform by Christmas Day or they'd have to shut us off Wat?! That's an excellent reason to burn fields and salt them.

I was / am pretty sympathetic to their situation. At that point, SDB was already deprecated in favor of dynamo, and they had been asking us to move off of SDB for over a year (before I had joined the company via acquisition). If someone ever does an oral history of AWS, I'm sure that SDB would be the "grand mistake" they made in their first few years. It was never ready for prime time in the way it needed to be.

Re: Riak and the demise of Basho

#72

Post by kstrauser »

jchrisa wrote: post #35
cies wrote: post #26

Earlier quoted context omitted.

Riak builds on theoretical foundation, laid out in Amazon's Dynamo paper. The other NoSQLs did not have this theoretical underpinning, and so they just offer a "best effort".

Dynamo is not exactly a performant or efficient model. It's the equivalent of pulling all the distributed systems guts out and handing them to the user to deal with. And the resulting toll is quantifiable: http://damienkatz.net/2013/05/dynamo_sure_works_hard.html

Damien's a very smart guy, but I don't think I agree with him here:

> Within a datacenter, the Mean Time To Failure (MTTF) for a network switch is one to two orders of magnitude higher than servers, depending on the quality of the switch.

A switch is highly unlikely to fail. They seem to be bulletproof. But having worked with a datacenter (on the engineering team of an early AWS competitor), switch _misconfiguration_ was all too common. Maybe a tech accidentally plugs in the wrong ethernet cable and forms a switching loop. Maybe someone fat-fingers a tag and a broken VLAN gets automatically deployed to 10,000 nodes. Either way, the _switch_ is alive, well, and pushing packets - but they're the _wrong_ packets and the result is indistinguishable from hardware failure to the end user.

At datacenter scales, these things happen... not infrequently. If you engineer your database to expect that netsplits are rare, you're going to have a bad time.

Re: Riak and the demise of Basho

#73

Post by elcritch »

andrew_deane_ wrote: post #17

Today bet365 have signed the agreement to purchase all Basho IP. We expect the agreement to be ratified in the US courts next week. Once cleared, our intention is to open source all code, help rebuild the community and collaboratively take the development of RIAK forward. In the coming weeks we will hopefully answer the questions people have and will be calling on the community to help forge the initial RIAK Roadmap.…

Fantastic news! I just used Riak Core for making a multi-master service. It was impressively straightforward! Glad to hear there are people intents to continue the project as OSS.

Will this include the Basho technical documents? Much of the better technical documentation lives on basho.com and was returning 404's.

[edit: and a +1 for Apache V2 (or MPL)]

Re: Riak and the demise of Basho

#74

Post by mirashii »

pron wrote: post #65
atonse wrote: post #48

Earlier quoted context omitted.

Please don't do GPL – It's a no-go and too restrictive for a lot of organizations. Apache 2 or MIT would be the most permissive.

If your organization uses any of: Linux, Android, emacs, gcc or any GNU utility, OpenJDK, or MongoDB, then GPL is clearly not a no-go. GPL only poses issues when the product is a library. For standalone products or ones with linking/access exceptions like OpenJDK, GPL poses no issues, and organizations aren't afraid to use it, especially when they don't intend to modify it. GPL is a great license that ensures that im…

> If your organization uses any of: Linux, Android, emacs, gcc or any GNU utility,

Two things wrong with this right off the bat. First, not all of these are GPLv3, which is the version with the problematic patent clauses that scare companies away. Second, it is not just in libraries that the GPLv3 is generally accepted to pose potential issues. You need look no further than Apple and the great investment they went through to avoid ever shipping GPLv3 software. The GPL kicks in as soon as you distribute the software.

Re: Riak and the demise of Basho

#75

Post by elcritch »

kstrauser wrote: post #72
jchrisa wrote: post #35

Earlier quoted context omitted.

Dynamo is not exactly a performant or efficient model. It's the equivalent of pulling all the distributed systems guts out and handing them to the user to deal with. And the resulting toll is quantifiable: http://damienkatz.net/2013/05/dynamo_sure_works_hard.html

Damien's a very smart guy, but I don't think I agree with him here: > Within a datacenter, the Mean Time To Failure (MTTF) for a network switch is one to two orders of magnitude higher than servers, depending on the quality of the switch. A switch is highly unlikely to fail. They seem to be bulletproof. But having worked with a datacenter (on the engineering team of an early AWS competitor), switch _misconfiguration_…

Good points. Weren't the last couple of AWS outages partly due to misconfigured network configs? Depending on your problem the replicated reads makes sense given those kinda of outages. Though I'm new-ish to riak's "core" design, when you get the APL with the servers, isn't it feasible to create a design similar to what Damien's proposing using a preferred master for a given vnode?

Re: Riak and the demise of Basho

#76

Post by sitkack »

russelldb74 wrote: post #41
simonvc wrote: post #8

More importantly, they acquired Russel Brown ( http://basho.com/posts/author/russell-brown/ )

Ah they didn't. I went to work for them after I was let go by Basho in January, but have since moved on to general contract work on Riak. I am lucky enough to remain in contact with the people at bet365. My take, for what it is worth, is that this is a very generous gesture from bet365 and it helps the whole community. bet365 really hide their light under a bushel in terms of how big a UK technology success story the…

You could make a killing getting MDC to work between Riak and BigTable [0]

[0] https://cloud.google.com/bigtable/

Re: Riak and the demise of Basho

#77

Post by ex_amazon_sde »

andrew_deane_ wrote: post #17

Today bet365 have signed the agreement to purchase all Basho IP. We expect the agreement to be ratified in the US courts next week. Once cleared, our intention is to open source all code, help rebuild the community and collaboratively take the development of RIAK forward. In the coming weeks we will hopefully answer the questions people have and will be calling on the community to help forge the initial RIAK Roadmap.…

GPLv3, as it protects from patent trolls and also encourages creating an better ecosystem.

There are companies that refuse to use GPLed code. They want to modify and then sell closed forks of your product, without paying you, and by doing so they fragment the userbase. GPL prevents this.

If you want to allow them to do so you can always switch to BSD or dual-license - perhaps after a private agreement and if you find the companies reputable.

Re: Riak and the demise of Basho

#78

Post by kyledrake »

kstrauser wrote: post #72
jchrisa wrote: post #35

Earlier quoted context omitted.

Dynamo is not exactly a performant or efficient model. It's the equivalent of pulling all the distributed systems guts out and handing them to the user to deal with. And the resulting toll is quantifiable: http://damienkatz.net/2013/05/dynamo_sure_works_hard.html

Damien's a very smart guy, but I don't think I agree with him here: > Within a datacenter, the Mean Time To Failure (MTTF) for a network switch is one to two orders of magnitude higher than servers, depending on the quality of the switch. A switch is highly unlikely to fail. They seem to be bulletproof. But having worked with a datacenter (on the engineering team of an early AWS competitor), switch _misconfiguration_…

VLANs were the bane of my existence when I had to figure out how to deal with them. I don't envy anyone whose job is to manage them on switches for a lot of servers.

Re: Riak and the demise of Basho

#79

Post by zokier »

If you are going to open source Riak, then maybe consider donating it to Apache Software Foundation? Maybe not immediately, but in the long term it feels like it would make sense.

Re: Riak and the demise of Basho

#80

Post by ams6110 »

Wow that one slipped by me totally. I had no idea Basho was in trouble (though I'm not a Riak user, I have used some of their other erlang projects including webmachine and lager).

Glad to hear their code will be open-sourced.

Post reply on HN