The Best Minds of My Generation Are Thinking About How To Make People Click Ads (or serve them efficently)
Take a look at finance / trading. Things are even worse over there, but they don’t blog about it.
5M Bid request/s, 2ms max response time – The Road to Damascus
61–70 of 71 posts
Re: 5M Bid request/s, 2ms max response time – The Road to Damascus
#62Earlier quoted context omitted.
A paid youtube? Not sure they would be worth it without the endless amounts of free content by free users. Facebook? There are plenty of sites like facebook without users. Charge for an account.. reduce the userbase, the platform loses value without the free users.
A paid Facebook means it would be designed to actually serve you (instead of wasting your time) and people would use it to connect with friends instead of liking endless shitposts.
If it is just friends why waste money on facebook just to talk. Most would rather spend that money on a more focused community/hobby like gaming and connect ingame.
Re: 5M Bid request/s, 2ms max response time – The Road to Damascus
#63Is this complete note? You didn't mention what you end up using. Is it golang or pony or F$? CoreFX mention in the end confused me more.
I work with Wael. Development is still ongoing. One implementation uses Golang, the other uses F# with a library that wraps libuv for faster network performance. Pony was used to write the stress-testing client for both implementations.
1 - See httpbeast in the latest round of the Techempower benchmarks: https://www.techempower.com/benchmarks/#section=data-r17&hw=...
Re: 5M Bid request/s, 2ms max response time – The Road to Damascus
#64Re: 5M Bid request/s, 2ms max response time – The Road to Damascus
#65I spent a few cycles in media buying and later in sell-side ad tech. Please do say what you will about advertising and its effects on the web, but I will say this: it is a world of fascinating tech. As a buyer I experienced janky pacing all of the time across various platforms, because this is a HARD problem. We had to manually adjust campaigns on a daily basis to ensure pacing worked properly. It was common to stop…
The bidder/pacer state is not necessarily massive, and certainly it does not consist of all the gazillions of past events. Depending on the strategy/bidding model, it can range from a few MB to several GBs, something that can fit in a beefy node.
> Google Fi/Spanner and BigTable have certainly been developed to support these issues.
I doubt any external store can be used with so low latency constraints (2-10ms) and high throughput (millions RPS). Perhaps Aerospike but even that is a stretch to put it in the hot-path. At this scale you're pretty much limited to fetch the state in memory and update it asynchronously every couple of minutes/hours.
Source: I also work in ad tech.
Re: 5M Bid request/s, 2ms max response time – The Road to Damascus
#66Glad to see this kind of stuff written with a strongly typed language instead of Python.
Why, they wasted months evaluating with an obsession for statically typed languages that have so far not produced anything more quickly or markedly better than that which others are producing with less pedantic languages.
Re: 5M Bid request/s, 2ms max response time – The Road to Damascus
#67Earlier quoted context omitted.
I can totally see where you're coming from. But major engineering achievements require efforts of many skilled people, who often like to be paid really well for their work. And the way the world works today is that a lot of big money is in the fields that are of questionable value to the society: advertising, finance, military, etc. And even in the fields that seem at first glance to be socially valuable, like health…
This is why federal research and development agencies should pay higher salaries to attract said engineers. Breakthroughs achieved would be funded by taxpayers and customers of these agencies and the patents for technology produced would be more likely enter the public domain.
Re: 5M Bid request/s, 2ms max response time – The Road to Damascus
#68Earlier quoted context omitted.
I work with Wael. Development is still ongoing. One implementation uses Golang, the other uses F# with a library that wraps libuv for faster network performance. Pony was used to write the stress-testing client for both implementations.
Have you considered Nim? You can achieve some really high performance with it[1]. Since you've considered Rust, Go, C and even Pony, Nim should really be on your list. 1 - See httpbeast in the latest round of the Techempower benchmarks: https://www.techempower.com/benchmarks/#section=data-r17&hw=...
Also why is evey immature native language considered, but the two speed demon languages without garbage collection - ISPC and C++ - are nowhere to be found?
Re: 5M Bid request/s, 2ms max response time – The Road to Damascus
#69Earlier quoted context omitted.
I’d much rather go back to randomly targeted “punch the monkey” ads. Far better than the current crop of ads which are usually one of fraudulent, malware, or borderline pornographic. Was I supposed to respond “oh no, current ads are so much better”?
> current crop of ads which are usually one of fraudulent, malware, or borderline pornographic. That doesn't match my experience at all, but maybe you can chalk it up to targeting?
Re: 5M Bid request/s, 2ms max response time – The Road to Damascus
#70Interested to know what Rust was missing. I built an ad exchange last year and it has been great. I have been using nightly builds, mostly for access to async/await, and it has been very fast and stable. I have had to submit a few pull requests to various projects along the way, but didn't find the ecosystem prohibitively lacking.
Would you mind sharing what libraries you are using?
That isn't all of them but those are the main ones.