Call me maybe: Aerospike

Post by obstinate » Wed, May 06, 2015, 6:15 AM UTC

perlgeek wrote: post #77

Ankhers wrote: post #16

I currently work for a company that uses Aerospike quite heavily. In the past couple weeks, we have begun to notice data inconsistencies in our counters. We are seeing fluctuations in the data, despite having no decrement operations. We have the enterprise edition of Aerospike, allowing us to be in constant contact with their support team and developers. A couple weeks later, and we still have no idea why this is hap…

When storing financial data, I'd certainly go with some kind of event sourcing: store deltas / financial transactions, not counters. The counters are just a sum over all deltas. If performance is an issue, you can make the counters available in a second database that's only for reading, and updated from the original deltas.

Post by manigandham » Wed, May 06, 2015, 6:33 AM UTC

obstinate wrote: post #82

manigandham wrote: post #73

We use Aerospike heavily. It works just fine. I'm constantly surprised by the general tone of comments on posts like these as if it's some crazy revelation that this software still obeys the fundamental laws of distributed systems. There is no perfect database out there, all of them will fail with network partitions. Aerospike was designed to work in clusters that are very close together, often the same rack. It has…

> We trust Aerospike because it's been hardened by lots of much much larger companies with very high production usage Why would you trust something after its untrustworthiness is demonstrated before your eyes? Just because some other companies use it and haven't yet publicized dissatisfaction? That is not how you make sound engineering decisions.

Post by linc01n » Wed, May 06, 2015, 6:59 AM UTC

Ankhers wrote: post #22

obituary_latte wrote: post #18

Earlier quoted context omitted.

Can you add any detail to this anecdote? It's interesting and important, but detail might help steer others appropriately. What kind of inconsistency? What kind of fluctuation?

I'm current working in AdTech. We are using counters to keep track of a lot of things that are important to us (e.g., reasons for not bidding on a given campaign, money spent within a given transaction, number of bid requests we get by exchange, etc). I personally have found two different, yet very similar, data fluctuations. The first I found when debugging an issue. I noticed the counters going up, dip down, then c…

Post by vtuulos » Wed, May 06, 2015, 7:28 AM UTC

phamilton wrote: post #79

jmalicki wrote: post #74

Earlier quoted context omitted.

Or you could just store that in a cookie.

Store what in the cookie? Every ad the user has seen across the entire web, along with how many times they've seen it in the last N minutes/hours/days/weeks? A cookie won't fit all that data and a more traditional database generally won't work. In memory k/v stores like Redis won't work due to data size (TBs of data). Hbase/Cassandra/etc sort of work with latency in the 5ms range. That's fairly expensive in a 90ms SL…

Post by masklinn » Wed, May 06, 2015, 7:52 AM UTC

jbergens wrote: post #86

sans-culottes wrote: post #65

Earlier quoted context omitted.

Not to mention "Call me maybe: Redis" [0]. Rinse and repeat for every NoSQL database out there. Point is, @aphyr skewers everybody. Aerospike is just the flavor of the month. [0] https://aphyr.com/posts/283-call-me-maybe-redis

I think FoundationDb got great scores from Aphyr, sadly it is now owned by Apple. For the others the big problem is when they promise some kind of ACID and cannot achieve it, if they were explicit with what the supported all customers could make informed decisions. Relations systems are generally bad ad horizontal scaling and can get very slow with full ACID over many servers.