Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Real World Recommendation System

blog.fennel.ai

Post reply on HN
Print view View original

111–120 of 156 posts

Re: Real World Recommendation System

#111

Post by jedberg »

samhw wrote: post #93
jedberg wrote: post #56

Earlier quoted context omitted.

FAANG was created by the TV personality Jim Cramer to talk about high growth tech stocks. At the time Netflix was doubling every year. It was based purely on finance. It's now been taken over by the tech industry to be shorthand for places that are highly selective in their hiring and tend to work on cutting edge tech at scale. That being said, the impact of Netflix on tech is pretty big. They pioneered using the clo…

> FAANG was created by the TV personality Jim Cramer to talk about high growth tech stocks. At the time Netflix was doubling every year. It was based purely on finance. That, and FAAG had less of a ring to it. Edit: Dammit, the GP made the same observation. Oh well, I'm keeping it.

If Netflix hadn't been such high growth and not included, Cramer probably would have gone with GAAF. :)

Re: Real World Recommendation System

#112

Post by whimsicalism »

vincentmarle wrote: post #53
oofbey wrote: post #51

Off-topic, but how did Netflix manage to get itself inserted into the FAANG acronym anyway? Their impact on the tech industry is trivial compared to all the others. Sure, if you just take out the N it's offensive, but we could have said "GAFA" or "FAAMG" would be more accurate to include Microsoft in their place.

There was a point in time when FAANG offered the best compensation packages for engineers (Netflix was one of them) - so that's where the term originated from but while it's outdated in many respects (Microsoft is not included, Facebook is now Meta, Google is now Alphabet etc etc) it's still sticky for some reason.

Microsoft in 2022 does not compensate as well as any of those. Microsoft in 2021 only out-compensated Amazon.

Re: Real World Recommendation System

#113

Post by samhw »

lmc wrote: post #94
whimsicalism wrote: post #78

Earlier quoted context omitted.

By "the field", you surely mean the academic field. In the industry, we run controlled experiments to validate all the time. Recommender systems is one of the few areas in ML where almost all of the knowledge is contained in industry, not academia.

That was my thinking - anything of value is product-specific and behind closed doors. It's not my field, but something I see come up from time to time that seems weirdly over-represented in ML articles.

I work on these systems, and if anything my only complaint about the field is the propensity to solve every optimisation problem with ML. I have seen people solve textbook-grade linear, and even differentiable, optimisation problems.

And the reason it happens despite the 'invisible hand' etc is because it still works, it just happens to be horrendously inefficient. I think that's the main area of inefficiency in the industry: not in getting the job done, nor even arguably in accuracy - at least not severely - but in overcomplicating the solution[0] because we've formed a cargo cult around one particular method of optimisation, beyond all nuance.

[0] I mean 'overcomplicating' in absolute terms. Of course the very crux of my point is that, from the data scientist's perspective, it's not overcomplicated - it's less complicated than using e.g. ILP precisely because we have made libraries like TensorFlow so incredibly easy and tempting to use.

Re: Real World Recommendation System

#114

Post by emptysea »

judge2020 wrote: post #101
KaiserPro wrote: post #17

> As a result, primary databases (e.g. MySQL, Mongo etc.) almost never work I mean it does. As far as I'm aware Facebook's ad platform is mostly backed by hundreds of thousands of Mysql instances. But more importantly this post really doesn't describe issues of scale. Sure it has the stages of recommendation, that might or might not be correct, but it doesn't describe how all of those processes are scheduled, coordin…

> . As far as I'm aware Facebook's ad platform is mostly backed by hundreds of thousands of Mysql instances. Same for YouTube itself https://www.mysql.com/customers/view/?id=750 and they use Vitess for horizontal scaling: https://vitess.io/

YouTube has since migrated to Spanner, there’s a podcast episode with one of the Vitess creators that covers the politics of the switch

Re: Real World Recommendation System

#115

Post by splonk »

greatpostman wrote: post #36

I’ve built one of these at FAANG. Generally the different parts of the system are completely separate teams that interact through apis and ingest systems. Usually there’s a mix of online and offline calculations, where features are stored in a nosqldb and some simple model runs in a tomcat server at inference time, or the offline result is just retrieved. Almost everything is precomputed. We had an api layer where an…

I have as well, and your comment matches my experience more than the article does. Different teams own different systems, and there's basically no intersection between "things that require a ton of data/computation" and "things that must be computed online".

Re: Real World Recommendation System

#117

Post by pedrosorio »

keewee7 wrote: post #96
colesantiago wrote: post #45

Earlier quoted context omitted.

Why TikTok in particular? What is the engineering story behind TikTok's recommendation system? How did they get it right?

TikTok seem to be learning from what the user is actually watching and for how long and not just the user's "Like"/"Not Interested In" actions. However it still seem to learn from the "Not Interested In" action more than any other platform.

This is a pretty misinformed take when it’s publicly known that YouTube was already doing this (learn from what the user is watching and for how long) the year Bytedance was founded (2012):

https://blog.youtube/news-and-events/youtube-now-why-we-focu...

Re: Real World Recommendation System

#118

Post by vikingcaffiene »

jeffbee wrote: post #9
vikingcaffiene wrote: post #3

Gentle reminder to anyone reading this that your problems are probably not FAANG problems. If you architect your system trying to solve problems you don't have, you are gonna have a bad time.

"And note that you don’t even have to be at FAANG scale to run into this problem - even if you have a small inventory (say few thousand items) and a few dozen features, you’d still run into this problem. " -TFA

Fair enough. I still think people should read stuff like this with a healthy measure of skepticism.

Re: Real World Recommendation System

#119

Post by oofbey »

splonk wrote: post #115
greatpostman wrote: post #36

I’ve built one of these at FAANG. Generally the different parts of the system are completely separate teams that interact through apis and ingest systems. Usually there’s a mix of online and offline calculations, where features are stored in a nosqldb and some simple model runs in a tomcat server at inference time, or the offline result is just retrieved. Almost everything is precomputed. We had an api layer where an…

I have as well, and your comment matches my experience more than the article does. Different teams own different systems, and there's basically no intersection between "things that require a ton of data/computation" and "things that must be computed online".

Yep. The author, as a peddler of recommendations solutions, has an incentive to convince people that this problem is very complicated, and they should hire a consultant.

In practice, good old Matrix Factorization works really well. Can you beat it with a huge team and tons of GPU hours to train fancy neural nets? Probably. Can you set up a nightly MF job on a single big machine and serve results quickly? Sure can.

Re: Real World Recommendation System

#120

Post by nerdponx »

oofbey wrote: post #119
splonk wrote: post #115

Earlier quoted context omitted.

I have as well, and your comment matches my experience more than the article does. Different teams own different systems, and there's basically no intersection between "things that require a ton of data/computation" and "things that must be computed online".

Yep. The author, as a peddler of recommendations solutions, has an incentive to convince people that this problem is very complicated, and they should hire a consultant. In practice, good old Matrix Factorization works really well. Can you beat it with a huge team and tons of GPU hours to train fancy neural nets? Probably. Can you set up a nightly MF job on a single big machine and serve results quickly? Sure can.

I think the question of "fancy technique versus simple technique" is beside the point. Assume for the sake of argument that you have a research organization and that it's worth your while to commit their time to a recommendation system.

The point here is that you don't typically need a huge amount of computation power to serve recommendations, even if the underlying model is sophisticated and required a lot of computation to train.

Likewise for data access, the online recommendation system typically does not need full access to the databases that the researchers need access to.

Post reply on HN