Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Call me maybe: Aerospike

aphyr.com

Post reply on HN
Print view View original

91–100 of 132 posts

Re: Call me maybe: Aerospike

#91

Post by woozy »

manigandham wrote: post #73

We use Aerospike heavily. It works just fine. I'm constantly surprised by the general tone of comments on posts like these as if it's some crazy revelation that this software still obeys the fundamental laws of distributed systems. There is no perfect database out there, all of them will fail with network partitions. Aerospike was designed to work in clusters that are very close together, often the same rack. It has…

I adore its SSD read/write optimization. The rest were understood as marketing materials.

Re: Call me maybe: Aerospike

#92

Post by vruiz »

manigandham wrote: post #83
obstinate wrote: post #82

Earlier quoted context omitted.

> We trust Aerospike because it's been hardened by lots of much much larger companies with very high production usage Why would you trust something after its untrustworthiness is demonstrated before your eyes? Just because some other companies use it and haven't yet publicized dissatisfaction? That is not how you make sound engineering decisions.

Demonstrated? It's just a single post, we would have to replicate these results ourselves and our specific environment. We use this 24/7 in a production system and have not encountered any issues and it matches actual data and experience from real conversations and meetings with other companies. We don't make decisions from blog posts.

aphyr is a well respected expert in the matter. You can reproduce the test yourself https://github.com/aphyr/jepsen

Consider that if you have not experienced any issues yet it might say more about your network stability than aerospike. Of course you can chose to ignore it, but network partitions eventually do happen, and when it does I hope your data is not mission critical.

Re: Call me maybe: Aerospike

#93

Post by jitix »

CyrusL wrote: post #44
jboggan wrote: post #39

Earlier quoted context omitted.

What is the rationale for storing monetary values in this sort of system? Not being snarky, just legitimately curious what scale of service could possibly necessitate that and what solutions didn't work beforehand.

Transactions in AdTech are different than normal payments. For example, imagine an ad campaign spending $30k/month at a rate of $5 per 1,000 impressions. The customer may want their budget spread evenly throughout the month, so the software sets a daily budget of $1000. But this really represents 200,000 daily impressions, each of which is a transaction that subtracts from the available balance in real-time. The buye…

IN such scenario, 'near' realtime would work just fine. Just process the impressions through storm or spark and put the results in HBase (CP type store) or even PostgreSQL.

Re: Call me maybe: Aerospike

#94

Post by manigandham »

vruiz wrote: post #92
manigandham wrote: post #83

Earlier quoted context omitted.

Demonstrated? It's just a single post, we would have to replicate these results ourselves and our specific environment. We use this 24/7 in a production system and have not encountered any issues and it matches actual data and experience from real conversations and meetings with other companies. We don't make decisions from blog posts.

aphyr is a well respected expert in the matter. You can reproduce the test yourself https://github.com/aphyr/jepsen Consider that if you have not experienced any issues yet it might say more about your network stability than aerospike. Of course you can chose to ignore it, but network partitions eventually do happen, and when it does I hope your data is not mission critical.

Are all the other companies and engineers using it useless then? One guy is not enough to make any opinion either way. And yes, we put a lot of effort into making sure the hardware and network are good, because when that works right everything else works well too. Trying to solve hardware issues with software is bound to lead to misery.

We have network partitions all the time, that's how we upgrade. On average each node is replaced every 2 weeks and we just terminate it through the API (both softlayer and aws). No big deal and we haven't lost any data yet, confirmed by other records in other datastores that have to match up.

If aphyr's post is the ultimate rating, why would anyone use anything thing else he's written about?

Re: Call me maybe: Aerospike

#95

Post by manigandham »

teraflop wrote: post #76
manigandham wrote: post #73

We use Aerospike heavily. It works just fine. I'm constantly surprised by the general tone of comments on posts like these as if it's some crazy revelation that this software still obeys the fundamental laws of distributed systems. There is no perfect database out there, all of them will fail with network partitions. Aerospike was designed to work in clusters that are very close together, often the same rack. It has…

> There is no perfect database out there, all of them will fail with network partitions. Some of them will fail in a way that keeps your data safe, others will fail in a way that preserves uptime but gives you temporarily inconsistent data. Aerospike apparently does neither. Why is it unreasonable to expect them not to falsely claim otherwise? The "crazy revelation" for me was not that Aerospike's software is, like e…

1) Aerospike is open-source and has a free community edition if you need it.

2) Yes, marketing claims are BS. If this was a reason to not use something, we'd have to stop using pretty much every other commercial piece of software we have. That's why we test and run software in our environment, and there... aerospike works. Really well. Even with network partitions. So I can understand kyle's tests in this post and the reasoning and results but there's still a big gap between this testing and the reality our company has experienced.

Re: Call me maybe: Aerospike

#96

Post by minaguib »

cldellow wrote: post #62
hurin wrote: post #49

Earlier quoted context omitted.

I'm really not seeing what you gain out of running fine-grained control all-the-time here. Even if it were vital for a customer that you hit a budget target exactly you could dynamically change the granularity of control as you got closer. If anything predictive modeling would give you better budget use when you do have the flexibility than granular adjustment would (I don't know much about the area though and just g…

phamilton touched on another good example - the budget can be implicitly set per-user via a frequency cap. If you see the user and have a chance to bid on an impression for them, the odds are good you'll see them again in two seconds -- winning two auctions means you've overspent by 100%. Oops.

Perhaps also worth pointing out that in RTB, quite often you won't know if you've actually won the auction or not for a few seconds later (when the exchange calls you back with a win notification, or you get it directly via a pingback from the user's device)

During that delay you might have actually already processed new bid requests (auctions) for the same user.

Depending on the order's characteristics and how much you're willing to deviate from target - especially when observed within a small time window - the above poses additional challenges w.r.t. overspending.

Re: Call me maybe: Aerospike

#97

Post by draven »

nemothekid wrote: post #46
gmagnusson wrote: post #42

I've used Aerospike at scale (approx 1MM tx per second) in private network, and smaller loads in cloud. I have always found it to be fast, reliable and extremely easy to operate (upgrade, modify cluster members, etc) w/o any downtime or interruption. It is a critical tool in my toolbox. I also have found their support and engineering team to be excellent. I admire the work that Aphyr does - though at the end of the d…

While people tend to use Aphyr's posts as anti-nosql-cannon-fodder, I don't think he has ever advocated for anyone to straight up not use a database or used any of these results to show that no one should use NoSQL ever. These seem like highly detailed Github Issues (infact the recent elasticsearch was a GH issue-turned-blog post), and these issue are brought to attention so that they could be fixed - not to slander…

In addition the testing methodology (the blog post) and the code (on github) are available so the tests can be reproduced to validate any enhancement or bugfix.

Those blog posts are also a great at debunking marketing claims.

Re: Call me maybe: Aerospike

#98

Post by acqq »

I was wondering what is going on, with the titles of this form appearing on HN and now browsing the blog, it appears that the author is the fan of the "Call me maybe" titled posts (a lot of them there, and then also here!). From what I understand, this phrase, as used in the titles by him, seem to mean to him something like the "review of" something or the "comment on" something. For what it's worth.

Re: Call me maybe: Aerospike

#99

Post by beberlei »

acqq wrote: post #98

I was wondering what is going on, with the titles of this form appearing on HN and now browsing the blog, it appears that the author is the fan of the "Call me maybe" titled posts (a lot of them there, and then also here!). From what I understand, this phrase, as used in the titles by him, seem to mean to him something like the "review of" something or the "comment on" something. For what it's worth.

It is a pun on this song https://www.youtube.com/watch?v=fWNaR-rxAic

Re: Call me maybe: Aerospike

#100

Post by annnnd »

linc01n wrote: post #85
Ankhers wrote: post #22

Earlier quoted context omitted.

I'm current working in AdTech. We are using counters to keep track of a lot of things that are important to us (e.g., reasons for not bidding on a given campaign, money spent within a given transaction, number of bid requests we get by exchange, etc). I personally have found two different, yet very similar, data fluctuations. The first I found when debugging an issue. I noticed the counters going up, dip down, then c…

I work in AdTech too. I'm still looking for a perfect counter solution. The counter we are using always overrun (which is better than up/down/up/down). We still manage to hack it by patching the number periodically. p.s. I am working in an Ad Network but not plugging into exchange. Our system is not capable for that.

Genuinly curious: did you try implementing realtime stream processing solution, using Storm or similar?
Post reply on HN