Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Call me maybe: Aerospike

aphyr.com

Post reply on HN
Print view View original

121–130 of 132 posts

Re: Call me maybe: Aerospike

#121

Post by hurin »

phamilton wrote: post #80
hurin wrote: post #61

Earlier quoted context omitted.

> A better example is frequency capping. Ever watch something on Hulu and see the same ad 4 times in a twenty-minute commercial? Or even, worse, back to back? Yeah, but when that happens I usually don't think, oh hey they are lacking an optimal in memory distributed database solution. I think, well... their engineers suck. Or they don't care. Pick one. edit: His point is vague, so there is nothing technical to respon…

Volume and latency requirements make it more difficult to track individuals on the web. It's an easier problem to solve in 50ms. It's also much easier to solve when it's only a million individuals rather than a couple hundred million individuals. Like most problems, scale makes it hard.

It's simply not a difficult problem: there are no consensus requirements between individuals, so scaling can't be made any harder by increasing N.

Re: Call me maybe: Aerospike

#122

Post by jhugg »

linc01n wrote: post #85
Ankhers wrote: post #22

Earlier quoted context omitted.

I'm current working in AdTech. We are using counters to keep track of a lot of things that are important to us (e.g., reasons for not bidding on a given campaign, money spent within a given transaction, number of bid requests we get by exchange, etc). I personally have found two different, yet very similar, data fluctuations. The first I found when debugging an issue. I noticed the counters going up, dip down, then c…

I work in AdTech too. I'm still looking for a perfect counter solution. The counter we are using always overrun (which is better than up/down/up/down). We still manage to hack it by patching the number periodically. p.s. I am working in an Ad Network but not plugging into exchange. Our system is not capable for that.

VoltDB is pretty fantastic at counters. (VoltDB Engineer)

Re: Call me maybe: Aerospike

#123

Post by yellowapple »

manigandham wrote: post #95
teraflop wrote: post #76

Earlier quoted context omitted.

> There is no perfect database out there, all of them will fail with network partitions. Some of them will fail in a way that keeps your data safe, others will fail in a way that preserves uptime but gives you temporarily inconsistent data. Aerospike apparently does neither. Why is it unreasonable to expect them not to falsely claim otherwise? The "crazy revelation" for me was not that Aerospike's software is, like e…

1) Aerospike is open-source and has a free community edition if you need it. 2) Yes, marketing claims are BS. If this was a reason to not use something, we'd have to stop using pretty much every other commercial piece of software we have. That's why we test and run software in our environment, and there... aerospike works. Really well. Even with network partitions. So I can understand kyle's tests in this post and th…

> Yes, marketing claims are BS. If this was a reason to not use something, we'd have to stop using pretty much every other commercial piece of software we have.

Which is what I at least have indeed opted to do; I avoid commercial software like the plague for this very reason, using it only when there isn't an alternative (like when it's a legacy system that has to be interfaced with). There are plenty of free software projects that don't make outrageous marketing claims and - therefore - aren't nearly as susceptible to disappointment and wasted money.

Aerospike's claims border on the realm of false advertising (if they don't actually classify as false advertising, which is a big "if"; the claim of 100% uptime is dubious at best and more likely to be an outright-malicious lie). Why should they get my money?

Re: Call me maybe: Aerospike

#124

Post by themartorana »

manigandham wrote: post #83
obstinate wrote: post #82

Earlier quoted context omitted.

> We trust Aerospike because it's been hardened by lots of much much larger companies with very high production usage Why would you trust something after its untrustworthiness is demonstrated before your eyes? Just because some other companies use it and haven't yet publicized dissatisfaction? That is not how you make sound engineering decisions.

Demonstrated? It's just a single post, we would have to replicate these results ourselves and our specific environment. We use this 24/7 in a production system and have not encountered any issues and it matches actual data and experience from real conversations and meetings with other companies. We don't make decisions from blog posts.

Sad that this was downvoted. I'll add an upvote because I think it's relevant.

Re: Call me maybe: Aerospike

#125

Post by DrJosiah »

jboggan wrote: post #39
Ankhers wrote: post #16

I currently work for a company that uses Aerospike quite heavily. In the past couple weeks, we have begun to notice data inconsistencies in our counters. We are seeing fluctuations in the data, despite having no decrement operations. We have the enterprise edition of Aerospike, allowing us to be in constant contact with their support team and developers. A couple weeks later, and we still have no idea why this is hap…

What is the rationale for storing monetary values in this sort of system? Not being snarky, just legitimately curious what scale of service could possibly necessitate that and what solutions didn't work beforehand.

Having stored money values in Redis several times in the past (sometimes without replicas at all!), the answer is knowing how much you can trust the system.

I trust enough to get the job done, but not enough to get bitten when these systems drop data. Because here's the truth: they all drop data.

Re: Call me maybe: Aerospike

#126

Post by drvsrinivasan »

Ankhers wrote: post #16

I currently work for a company that uses Aerospike quite heavily. In the past couple weeks, we have begun to notice data inconsistencies in our counters. We are seeing fluctuations in the data, despite having no decrement operations. We have the enterprise edition of Aerospike, allowing us to be in constant contact with their support team and developers. A couple weeks later, and we still have no idea why this is hap…

Today, the inconsistencies here were diagnosed by company and Aerospike to be caused by two clusters connected with XDR concurrently writing data to the same counter and shipping to each other and intentionally overwriting some data (bad design that somehow slipped through the cracks). So, this issue is unrelated to the Jepsen network partitioning tests that is the subject of the original article. The work that @Aphyr is doing is very valuable and much appreciated. (Aerospike Founder)

Re: Call me maybe: Aerospike

#127

Post by acqq »

chipsy wrote: post #107
acqq wrote: post #104

Earlier quoted context omitted.

Apart from author being obviously inspired by the song I admit I don't see any connection or anything worth naming that "the pun." Maybe it's just me.

Making a phone call is an asynchronous event, and as the song suggests, sometimes you give someone your number but they never call back. With any distributed system in real world conditions, a similar situation arises where a request doesn't get handled or is lost along the way.

So he uses the phrase instead of "distributed systems?" (shrug)

Re: Call me maybe: Aerospike

#128

Post by annnnd »

bluef00t wrote: post #119
annnnd wrote: post #101

Earlier quoted context omitted.

Nooooooooooooooooooooo! Seriously, no! Use mongoDB, PostgreSQL, even flat files if you must, but HBase? We used it in production about 3-4 years ago and it was a nightmare from both usage and especially maintenance point. Fortunately we had a flat-files based backup system so we were able to rescue data every! Single! Time! the damn thing crashed and took (part of) data with it. Of course, this is anecdotal evidence,…

I see where you are coming from. HBase was unstable 3-4 years ago, but after a great amount of dev effort and battle hardening from Cloudera, Salesforce, etc., it is very stable now. We have ~ 400 nodes running in production for a very critical use case and have seen 0 data loss edge cases in the last 2 years, along with some of our servers running > 6 months without any reboots. We use is in a very real time use cas…

Thanks for the info on HBase stability. I probably won't use it again (once burnt...), but if they really managed to pull their act together - good for them!

Re: Call me maybe: Aerospike

#129

Post by tootie »

jhugg wrote: post #59
tootie wrote: post #25

Is there any reason he's never tried to analyze a "classic" RDBMS like Oracle or SQL Server? I have to imagine they'd clobber a lot of this hipster technology.

Well, those systems are largely not distributed, barring SQL Parallel Data Warehouse and, very arguably, RAC. Jepson tests network partitions... so less useful.

The whole draw of Oracle for the last 20+ years is how well it can replicate across a cluster over a network. Maybe it's not quite the same as how modern KV stores work, but it's still guaranteeing consistency across a network and is therefore a candidate for jepsen testing.

Re: Call me maybe: Aerospike

#130

Post by tootie »

samkone wrote: post #26
tootie wrote: post #25

Is there any reason he's never tried to analyze a "classic" RDBMS like Oracle or SQL Server? I have to imagine they'd clobber a lot of this hipster technology.

Lol He can't because Oracle licensing forbids to do this kind of work and make the results public.

Thank for a real answer. Do you have a source for that assertion. Is Microsoft the same?
Post reply on HN