Would love to see that load balancer implementation, as I'm a scrub and this project fascinates me.
nginx makes it really simple to setup a load balancer, it defaults to distributing requests equally between all upstream hosts but you can always assign weights to each of your server. https://docs.nginx.com/nginx/admin-guide/load-balancer/http-...
Building an Internet Scale Meme Search Engine
11–20 of 154 posts
Re: Building an Internet Scale Meme Search Engine
#12My question is about the image distribution costs. All the memes on the site seem to be coming straight off an object storage, all that bandwidth consumption has got to add up(?). Some sort of a CDN might help depending on the search patterns.
Re: Building an Internet Scale Meme Search Engine
#13Re: Building an Internet Scale Meme Search Engine
#14This is remarkable. I'd love to see that combined with some kind of sentiment analysis like Microsoft offers, just to see if something useful comes out of it. Sometimes, I don’t know the exact words when looking for a meme, but once I see it, I know that’s the one.
Re: Building an Internet Scale Meme Search Engine
#15Re: Building an Internet Scale Meme Search Engine
#16My #1 recommendation for anyone thinking about the convoluted OCR solution: use a cheap OCR API and save yourself months of time / hassle / upkeep. Google's OCR API is a good place to start, but AWS has one too and dozens of others out there.
Re: Building an Internet Scale Meme Search Engine
#17Earlier quoted context omitted.
(Not the author) Maybe they leveraged https://knowyourmeme.com/ ? But that can't possibly have all the random memes, could it?
Author here: KnowYourMeme is one of many sites that memes are continually ingested from (any site that has memes I try to ingest regularly) :)
Re: Building an Internet Scale Meme Search Engine
#18What was your average price per iPhone, if you don’t mind disclosing?
Re: Building an Internet Scale Meme Search Engine
#19Fences worldwide will be overjoyed to hear of this novel application.