Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Call me maybe: Aerospike

aphyr.com

Post reply on HN
Print view View original

101–110 of 132 posts

Re: Call me maybe: Aerospike

#101

Post by annnnd »

bluef00t wrote: post #66
Ankhers wrote: post #54

Earlier quoted context omitted.

We are first evaluating MongoDB. I believe the main reason behind this is we are already using Mongo in other parts of our application, so there is no additional setup when converting. Note that nothing is set in stone. The decision to begin migrations only happened today. It is possible that we will end up using some other technology altogether, or even we find out the issues we are having with Aerospike and continu…

Try HBase. At another AdTech company, we have been very happy with it. https://eng.yammer.com/call-me-maybe-hbase

Nooooooooooooooooooooo! Seriously, no! Use mongoDB, PostgreSQL, even flat files if you must, but HBase?

We used it in production about 3-4 years ago and it was a nightmare from both usage and especially maintenance point. Fortunately we had a flat-files based backup system so we were able to rescue data every! Single! Time! the damn thing crashed and took (part of) data with it.

Of course, this is anecdotal evidence, and things might have changed from then, but I wouldn't touch it. Life is too short.

EDIT: Also, I am curious how the results in the above link would compare to aphyr's if he performed the test on HBase?

Re: Call me maybe: Aerospike

#102

Post by vruiz »

manigandham wrote: post #94
vruiz wrote: post #92

Earlier quoted context omitted.

aphyr is a well respected expert in the matter. You can reproduce the test yourself https://github.com/aphyr/jepsen Consider that if you have not experienced any issues yet it might say more about your network stability than aerospike. Of course you can chose to ignore it, but network partitions eventually do happen, and when it does I hope your data is not mission critical.

Are all the other companies and engineers using it useless then? One guy is not enough to make any opinion either way. And yes, we put a lot of effort into making sure the hardware and network are good, because when that works right everything else works well too. Trying to solve hardware issues with software is bound to lead to misery. We have network partitions all the time, that's how we upgrade. On average each n…

> Are all the other companies and engineers using it useless then?

Of course not, software is about trade-offs, and every company has different use cases. Is it their primary datastore? if they lose data do they lose some data samples for a recommendation engine? or someone's money? I wouldn't assume "if it works for them it works for me".

> Trying to solve hardware issues with software is bound to lead to misery.

I strongly disagree with that, I believe exactly the oposite.

> If aphyr's post is the ultimate rating, why would anyone use anything thing else he's written about?

He does not judge the system's usefulness, throughput, etc. But he's a good benchmark for distributed system's reliability. While he might not test every possible scenario, if he says software X loses data on conditions Y, I do believe him. It's still up to me to decide if that matters for my use case or not.

Re: Call me maybe: Aerospike

#103

Post by annnnd »

masklinn wrote: post #89
jbergens wrote: post #86

Earlier quoted context omitted.

I think FoundationDb got great scores from Aphyr, sadly it is now owned by Apple. For the others the big problem is when they promise some kind of ACID and cannot achieve it, if they were explicit with what the supported all customers could make informed decisions. Relations systems are generally bad ad horizontal scaling and can get very slow with full ACID over many servers.

> I think FoundationDb got great scores from Aphyr, FoundationDB ran Jepsen internally and reported stuff[0], Kyle never worked with it. [0] http://blog.foundationdb.com/call-me-maybe-foundationdb-vs-j... half broken now, none of the images load for me. @aphyr seems to have taken them at their word wrt testing though: https://twitter.com/aphyr/status/405017101804396546

Curious: how difficult is it to replicate aphyr's tests? Can we beleive independent entities posting jepsen reports on some random DB?

Re: Call me maybe: Aerospike

#104

Post by acqq »

beberlei wrote: post #99
acqq wrote: post #98

I was wondering what is going on, with the titles of this form appearing on HN and now browsing the blog, it appears that the author is the fan of the "Call me maybe" titled posts (a lot of them there, and then also here!). From what I understand, this phrase, as used in the titles by him, seem to mean to him something like the "review of" something or the "comment on" something. For what it's worth.

It is a pun on this song https://www.youtube.com/watch?v=fWNaR-rxAic

Apart from author being obviously inspired by the song I admit I don't see any connection or anything worth naming that "the pun." Maybe it's just me.

Re: Call me maybe: Aerospike

#105

Post by pjc50 »

manigandham wrote: post #73

We use Aerospike heavily. It works just fine. I'm constantly surprised by the general tone of comments on posts like these as if it's some crazy revelation that this software still obeys the fundamental laws of distributed systems. There is no perfect database out there, all of them will fail with network partitions. Aerospike was designed to work in clusters that are very close together, often the same rack. It has…

Genuine question: how do you know you've never lost a single write?

Re: Call me maybe: Aerospike

#106

Post by chisleu »

teraflop wrote: post #76
manigandham wrote: post #73

We use Aerospike heavily. It works just fine. I'm constantly surprised by the general tone of comments on posts like these as if it's some crazy revelation that this software still obeys the fundamental laws of distributed systems. There is no perfect database out there, all of them will fail with network partitions. Aerospike was designed to work in clusters that are very close together, often the same rack. It has…

> There is no perfect database out there, all of them will fail with network partitions. Some of them will fail in a way that keeps your data safe, others will fail in a way that preserves uptime but gives you temporarily inconsistent data. Aerospike apparently does neither. Why is it unreasonable to expect them not to falsely claim otherwise? The "crazy revelation" for me was not that Aerospike's software is, like e…

Most high end DB support is very expensive. Cassandra, Oracle, etc, etc.

What bothers me more than anything was Aerospike felt the need to compare a benchmark done on an in-memory dataset directly with a Cassandra benchmark on dataset that was many fold larger into disk. They made this comparison and said "look! Aerospike is x times faster than Cassandra!"

That was the end of giving a shit for me. When people feel they need to lie to convince you of something, that is when I know that I don't want what they want.

Incidentally, that is also how I stopped caring about politics.

Re: Call me maybe: Aerospike

#107

Post by chipsy »

acqq wrote: post #104
beberlei wrote: post #99

Earlier quoted context omitted.

It is a pun on this song https://www.youtube.com/watch?v=fWNaR-rxAic

Apart from author being obviously inspired by the song I admit I don't see any connection or anything worth naming that "the pun." Maybe it's just me.

Making a phone call is an asynchronous event, and as the song suggests, sometimes you give someone your number but they never call back. With any distributed system in real world conditions, a similar situation arises where a request doesn't get handled or is lost along the way.

Re: Call me maybe: Aerospike

#108

Post by gmagnusson »

ploxiln wrote: post #64
gmagnusson wrote: post #42

I've used Aerospike at scale (approx 1MM tx per second) in private network, and smaller loads in cloud. I have always found it to be fast, reliable and extremely easy to operate (upgrade, modify cluster members, etc) w/o any downtime or interruption. It is a critical tool in my toolbox. I also have found their support and engineering team to be excellent. I admire the work that Aphyr does - though at the end of the d…

"though at the end of the day" "nothing is perfect" ... Aerospike makes blatantly ridiculous promises in their high-level descriptions of their database. That makes our jobs harder (before Aphyr makes them easier) because we don't know what Aerospike is actually good at or exactly what kinds of data loss potential we need to architect our systems around. Isn't it kind of annoying that some technical projects bolster…

I can't and won't defend Aerospike's descriptions on website or white paper. And yes, "Thanks Aphyr".

I came across Aerospike technology via a pre-existing system at a previous employer, and watched that system scale up and perform in a serious way. It wasn't all unicorns and roses all the time as real life never is, but in the context of the real world, it was great. The software is rock solid in a way I've rarely come across, and support was spectacular. (I forget my current production clusters are even running sometimes they are so stable, reliable and self-operating)

And at the end of the day, there was no other solution out there remotely competitive that we could find. And I looked - not because we were dissatisfied, but because that was our fiduciary responsibility to the company, to ensure that we were deploying the most cost-effective systems that met our feature and performance requirements.

Ultimately yes - I think that as an engineer, you need to understand what your tools are really capable of and avoid doing what I call "BDD" (Blog-Driven Design). That isn't the ideal answer - it would be nice to have a reliable understanding of the capabilities of the materials we use to build systems (like civil engineers can reason about materials like steel and concrete in repeatable ways) but what we call "software engineering and architecture" is still a very young discipline, very often with unrealistic expectations about our ability to deliver in given budgetary and temporal constraints, so we do what we can.

Re: Call me maybe: Aerospike

#109

Post by rsynnott »

threeseed wrote: post #70
electrum wrote: post #27

Earlier quoted context omitted.

He tested PostgreSQL: https://aphyr.com/posts/282-call-me-maybe-postgres

I don't understand how this is comparable though. All of other databases were tested in clustered mode. Why not PostgreSQL as well ?

I don't think postgres has a first-party clustering system.

Re: Call me maybe: Aerospike

#110

Post by lmm »

sans-culottes wrote: post #65
misframer wrote: post #60

Earlier quoted context omitted.

You should take a look at "Call me maybe: MongoDB" [0] [0] https://aphyr.com/posts/284-call-me-maybe-mongodb

Not to mention "Call me maybe: Redis" [0]. Rinse and repeat for every NoSQL database out there. Point is, @aphyr skewers everybody. Aerospike is just the flavor of the month. [0] https://aphyr.com/posts/283-call-me-maybe-redis

I think that's a false equivalence. Aphyr is pretty positive about Riak (with the correct configuration) and Cassandra (when used for appropriate scenarios). If I was choosing a new system those are the two I'd be looking at.
Post reply on HN