71–80 of 132 posts
What is the 3 color chart that I've seen posted in several of these articles? I get that it breaks down the different CAP combinations with the things that it allows/implies. But is there a good breakdown of all the terms and what they mean?
I currently work for a company that uses Aerospike quite heavily. In the past couple weeks, we have begun to notice data inconsistencies in our counters. We are seeing fluctuations in the data, despite having no decrement operations. We have the enterprise edition of Aerospike, allowing us to be in constant contact with their support team and developers. A couple weeks later, and we still have no idea why this is hap…
We* tested the increment functionality heavily (300K-1M aggregate ops/sec) before we turned it on in revenue service. We use it for a couple of different things, event counting is absolutely the major use case.
In a single-cluster world, it works phenomenally well. In a XDR world, things get a little tricky, and we had to change the way our application logic worked to compensate for it.
Any more information you can share about your use case?
*a big ad tech company that uses Aerospike heavily
I'm constantly surprised by the general tone of comments on posts like these as if it's some crazy revelation that this software still obeys the fundamental laws of distributed systems.
There is no perfect database out there, all of them will fail with network partitions. Aerospike was designed to work in clusters that are very close together, often the same rack. It has much tighter timings and tolerances in exchange for providing much higher performance in certain situations and definitely has one of the best SSD focused storage systems I've come across.
If you don't have a high performance network interconnect between nodes, then there will be more issues with Aerospike since it relies on that more than some other system that use Paxos for all writes (like aphyr mentions). We run several TB's of data accessed at 100k+ TPS including very fine grained counters and everything works. And yes, we run on the cloud in AWS and SoftLayer and have yet to have major problems with the proper network setup.
Btw, there is a comment below from the current CTO of AppNexus, one of the companies that pioneered real-time bidding for digital ads and runs several million auctions per second on one of the biggest ad exchanges available. They were the first customer for Aerospike and from everything I've learned from their team, it works really well for them, and they definitely are not happy to just "lose" data however insignificant it might seem. Volume changes everything and even a fraction of a percent will add up. We trust Aerospike because it's been hardened by lots of much much larger companies with very high production usage, the key is being aware of all the technical requirements and the environment you're deploying in.
I think the real major issue here that people seem upset with are the general claims and marketing information. I can't speak to all that and there are definitely some things like 100% uptime which do seem overly confident, but this is true of every single technology vendor out there unfortunately. I'm not saying Aerospike is any better or worse as a company but marketing material only goes so far and it would surprise me if further research wasn't done for any mission critical system.
Earlier quoted context omitted.
I'm really not seeing what you gain out of running fine-grained control all-the-time here. Even if it were vital for a customer that you hit a budget target exactly you could dynamically change the granularity of control as you got closer. If anything predictive modeling would give you better budget use when you do have the flexibility than granular adjustment would (I don't know much about the area though and just g…
A better example is frequency capping. Ever watch something on Hulu and see the same ad 4 times in a twenty-minute commercial? Or even, worse, back to back? With a real-time data stack you can avoid the duplicated ad a good percent of the time. Better experience for buyers, for publishers, and for users.
Fast, functional or reliable... pick two.
We use Aerospike heavily. It works just fine. I'm constantly surprised by the general tone of comments on posts like these as if it's some crazy revelation that this software still obeys the fundamental laws of distributed systems. There is no perfect database out there, all of them will fail with network partitions. Aerospike was designed to work in clusters that are very close together, often the same rack. It has…
Some of them will fail in a way that keeps your data safe, others will fail in a way that preserves uptime but gives you temporarily inconsistent data. Aerospike apparently does neither. Why is it unreasonable to expect them not to falsely claim otherwise?
The "crazy revelation" for me was not that Aerospike's software is, like everything else, subject to the CAP theorem. It's that they apparently think it's awesome to claim that it isn't, and charge tens of thousands of dollars for their product on that justification.
I currently work for a company that uses Aerospike quite heavily. In the past couple weeks, we have begun to notice data inconsistencies in our counters. We are seeing fluctuations in the data, despite having no decrement operations. We have the enterprise edition of Aerospike, allowing us to be in constant contact with their support team and developers. A couple weeks later, and we still have no idea why this is hap…
If performance is an issue, you can make the counters available in a second database that's only for reading, and updated from the original deltas.
Fast, functional or reliable... pick two.
ITYM consistent, available or partition-tolerant...
Something Aerospike didn't realize.
(And nope, sorry, I'm completely uninterested in your anecdotes about how you haven't personally lost data when [1] there's a clear data loss scenario highlighted in the post, [2] Aerospike actively recommend services like EC2 and GCE that routinely partition, and [3] there are people in this thread who have experienced the same problems).
Earlier quoted context omitted.
A better example is frequency capping. Ever watch something on Hulu and see the same ad 4 times in a twenty-minute commercial? Or even, worse, back to back? With a real-time data stack you can avoid the duplicated ad a good percent of the time. Better experience for buyers, for publishers, and for users.
Or you could just store that in a cookie.
A cookie won't fit all that data and a more traditional database generally won't work. In memory k/v stores like Redis won't work due to data size (TBs of data). Hbase/Cassandra/etc sort of work with latency in the 5ms range. That's fairly expensive in a 90ms SLA, but you can make it work. It does limit the amount of work you are able to do.
Earlier quoted context omitted.
A better example is frequency capping. Ever watch something on Hulu and see the same ad 4 times in a twenty-minute commercial? Or even, worse, back to back? With a real-time data stack you can avoid the duplicated ad a good percent of the time. Better experience for buyers, for publishers, and for users.
> A better example is frequency capping. Ever watch something on Hulu and see the same ad 4 times in a twenty-minute commercial? Or even, worse, back to back? Yeah, but when that happens I usually don't think, oh hey they are lacking an optimal in memory distributed database solution. I think, well... their engineers suck. Or they don't care. Pick one. edit: His point is vague, so there is nothing technical to respon…
Like most problems, scale makes it hard.