Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Call me maybe: Aerospike

aphyr.com

Post reply on HN
Print view View original

61–70 of 132 posts

Re: Call me maybe: Aerospike

#61

Post by hurin »

phamilton wrote: post #58
hurin wrote: post #49

Earlier quoted context omitted.

I'm really not seeing what you gain out of running fine-grained control all-the-time here. Even if it were vital for a customer that you hit a budget target exactly you could dynamically change the granularity of control as you got closer. If anything predictive modeling would give you better budget use when you do have the flexibility than granular adjustment would (I don't know much about the area though and just g…

A better example is frequency capping. Ever watch something on Hulu and see the same ad 4 times in a twenty-minute commercial? Or even, worse, back to back? With a real-time data stack you can avoid the duplicated ad a good percent of the time. Better experience for buyers, for publishers, and for users.

> A better example is frequency capping. Ever watch something on Hulu and see the same ad 4 times in a twenty-minute commercial? Or even, worse, back to back?

Yeah, but when that happens I usually don't think, oh hey they are lacking an optimal in memory distributed database solution.

I think, well... their engineers suck. Or they don't care. Pick one.

edit: His point is vague, so there is nothing technical to respond to. I am very much interested in a good technical example - but the things mentioned so far are by all appearances relatively straight-forward and linear, hence lack of effort or bad engineering are the only reasonable assumptions left.

Re: Call me maybe: Aerospike

#62

Post by cldellow »

hurin wrote: post #49
cldellow wrote: post #48

Earlier quoted context omitted.

The impressions may be sparse, e.g. say you're retargeting CEOs (demographic information you're getting from a DSP) who have visited your website in the last month (via a pixel you drop) who are in New York City (via a geoIP DB). So, fine, a probabilistic model might work well. And you might decide to bid on 100% of impressions. And you might decide that you have to bid $200 CPM to win -- which you're OK doing, becau…

I'm really not seeing what you gain out of running fine-grained control all-the-time here. Even if it were vital for a customer that you hit a budget target exactly you could dynamically change the granularity of control as you got closer. If anything predictive modeling would give you better budget use when you do have the flexibility than granular adjustment would (I don't know much about the area though and just g…

phamilton touched on another good example - the budget can be implicitly set per-user via a frequency cap. If you see the user and have a chance to bid on an impression for them, the odds are good you'll see them again in two seconds -- winning two auctions means you've overspent by 100%. Oops.

Re: Call me maybe: Aerospike

#63

Post by cwyers »

hurin wrote: post #61
phamilton wrote: post #58

Earlier quoted context omitted.

A better example is frequency capping. Ever watch something on Hulu and see the same ad 4 times in a twenty-minute commercial? Or even, worse, back to back? With a real-time data stack you can avoid the duplicated ad a good percent of the time. Better experience for buyers, for publishers, and for users.

> A better example is frequency capping. Ever watch something on Hulu and see the same ad 4 times in a twenty-minute commercial? Or even, worse, back to back? Yeah, but when that happens I usually don't think, oh hey they are lacking an optimal in memory distributed database solution. I think, well... their engineers suck. Or they don't care. Pick one. edit: His point is vague, so there is nothing technical to respon…

I don't get your point here. He's explaining why it works the way it does. You're saying you don't think about it as a user. That doesn't invalidate or even respond to his point.

Re: Call me maybe: Aerospike

#64

Post by ploxiln »

gmagnusson wrote: post #42

I've used Aerospike at scale (approx 1MM tx per second) in private network, and smaller loads in cloud. I have always found it to be fast, reliable and extremely easy to operate (upgrade, modify cluster members, etc) w/o any downtime or interruption. It is a critical tool in my toolbox. I also have found their support and engineering team to be excellent. I admire the work that Aphyr does - though at the end of the d…

"though at the end of the day" "nothing is perfect" ... Aerospike makes blatantly ridiculous promises in their high-level descriptions of their database. That makes our jobs harder (before Aphyr makes them easier) because we don't know what Aerospike is actually good at or exactly what kinds of data loss potential we need to architect our systems around.

Isn't it kind of annoying that some technical projects bolster their popularity/ecosystem with very fancy websites and impressive/competitive claims, but to really do your job right you have to throw all that away? The best you can do is try to get a sense from the reports of others who have tried something (and may or may not have been rigorous in their evaluation) so you can pick good candidates to even put through trials. (so again, thanks Aphyr)

Re: Call me maybe: Aerospike

#65

Post by sans-culottes »

misframer wrote: post #60
Ankhers wrote: post #54

Earlier quoted context omitted.

We are first evaluating MongoDB. I believe the main reason behind this is we are already using Mongo in other parts of our application, so there is no additional setup when converting. Note that nothing is set in stone. The decision to begin migrations only happened today. It is possible that we will end up using some other technology altogether, or even we find out the issues we are having with Aerospike and continu…

You should take a look at "Call me maybe: MongoDB" [0] [0] https://aphyr.com/posts/284-call-me-maybe-mongodb

Not to mention "Call me maybe: Redis" [0]. Rinse and repeat for every NoSQL database out there.

Point is, @aphyr skewers everybody. Aerospike is just the flavor of the month.

[0] https://aphyr.com/posts/283-call-me-maybe-redis

Re: Call me maybe: Aerospike

#66

Post by bluef00t »

Ankhers wrote: post #54
digitalzombie wrote: post #52

Earlier quoted context omitted.

What are you migrating to ? As is away from Aerospike to... ? Thanks, very interesting anecdote/case.

We are first evaluating MongoDB. I believe the main reason behind this is we are already using Mongo in other parts of our application, so there is no additional setup when converting. Note that nothing is set in stone. The decision to begin migrations only happened today. It is possible that we will end up using some other technology altogether, or even we find out the issues we are having with Aerospike and continu…

Try HBase. At another AdTech company, we have been very happy with it. https://eng.yammer.com/call-me-maybe-hbase

Re: Call me maybe: Aerospike

#67

Post by cschneid »

What is the 3 color chart that I've seen posted in several of these articles? I get that it breaks down the different CAP combinations with the things that it allows/implies. But is there a good breakdown of all the terms and what they mean?

Re: Call me maybe: Aerospike

#68

Post by SystemOut »

Rapzid wrote: post #55
Ankhers wrote: post #22

Earlier quoted context omitted.

I'm current working in AdTech. We are using counters to keep track of a lot of things that are important to us (e.g., reasons for not bidding on a given campaign, money spent within a given transaction, number of bid requests we get by exchange, etc). I personally have found two different, yet very similar, data fluctuations. The first I found when debugging an issue. I noticed the counters going up, dip down, then c…

Some of this stuff looks to me, as somebody not familiar with the domain of course, as a really good use case for event sourcing and in particular something Kafka/Samza could tackle well. For future consideration and delayed evaluation of course. I guess if you absolutely must have the most up-to-date information so you can make decisions on it RIGHT NOW that wouldn't work very well :| Or would it? If your bids and s…

Bids generally have to be responded to within 100ms on these AdTech stacks and if you start to timeout they slow down bids being sent to you....which makes your bid manager less desirable by advertisers. You can probably use stream processing to build and update your models but I'd be surprised if you could handle the bid responses through the same mechanism.

Re: Call me maybe: Aerospike

#69

Post by ploxiln »

Ankhers wrote: post #54
digitalzombie wrote: post #52

Earlier quoted context omitted.

What are you migrating to ? As is away from Aerospike to... ? Thanks, very interesting anecdote/case.

We are first evaluating MongoDB. I believe the main reason behind this is we are already using Mongo in other parts of our application, so there is no additional setup when converting. Note that nothing is set in stone. The decision to begin migrations only happened today. It is possible that we will end up using some other technology altogether, or even we find out the issues we are having with Aerospike and continu…

oh... dude... exact same mistake twice...

OK, let me try to be more constructive. Since accounts are independent, shard based on account (in the application, not in some magic shard-distributing layer). Treat each shard as its own cluster.

If you want super fast requests but can accept being down for an hour or two a couple times a year, a shard can be a single beefy host with a replicating slave. I'd consider either Redis or Mysql/Postgresql. Really, these old-style sql databases can be the fastest things that have the kind of consistency you need.

I've maintained a mongodb cluster configured a couple of different ways. Performance at high load and reasonable consistency is not as great as some older alternatives.

Re: Call me maybe: Aerospike

#70

Post by threeseed »

electrum wrote: post #27
tootie wrote: post #25

Is there any reason he's never tried to analyze a "classic" RDBMS like Oracle or SQL Server? I have to imagine they'd clobber a lot of this hipster technology.

He tested PostgreSQL: https://aphyr.com/posts/282-call-me-maybe-postgres

I don't understand how this is comparable though.

All of other databases were tested in clustered mode. Why not PostgreSQL as well ?

Post reply on HN