Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Production Twitter on one machine? 100Gbps NICs and NVMe are fast

thume.ca

Post reply on HN
Print view View original

291–300 of 500 posts

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#291

Post by afrodc_ »

fzingle wrote: post #288
threeseed wrote: post #283

Earlier quoted context omitted.

It's an easy thought experiment. Do you think that the most successful web companies in the world with arguably the best people i.e. Amazon, Facebook, Instagram, TikTok, LinkedIn, Pinterest, Youtube, Netflix, Snapchat etc. have no idea what they are doing. That the highly complex, expensive and latency impacting recommendation systems could be replaced by trivial sorting. Or maybe they do work, do translate to increa…

Maybe they are considering that we shouldn't build and optimize our society solely for the purposes of maximizing revenue. Would it be better to live in a world where Twitter (for example) existed because it is a useful thing and not because it might make lots of money?

Doesn't it lose a lot of money and its usefulness is directly correlated with its current massive usage.

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#292

Post by bitbckt »

Regarding tweet distribution, I was one of the folks who built the first scalable solution to this problem at Twitter (called Haplocheirus). We used the Yahoo “Feeding Frenzy” design, pushing tweets through a Redis-backed caching layer.

Feel free to continue using that (historically-correct) answer in interviews. :P

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#293

Post by jiggawatts »

no_way wrote: post #159
varunkmohan wrote: post #119

Earlier quoted context omitted.

Genuinely asking, why do you think Twitter needs 24 million vcpus to run? This is not apples to apples but Whatsapp is a product that entirely ran on 16 servers at the time of acquisition (1.5 billion users). It really begs the question why Twitter uses so much compute if there are companies that have operated significantly more efficiently. Twitter was unprofitable during acquisition and spent around half their reve…

Chat apps are mostly one on one interaction, it is much harder run an open platform where every user can potentially interact with every other user, not even talking about search and how complex it gets. If Twitter is bloated or not is a valid discussion, but comparison it to WhatsApp is not.

Ironically the one-to-many broadcasts are much easier to implement on a single box than as a scalable service spread across thousands of small container instances.

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#295

Post by [deleted] »

jacobsenscott wrote: post #290

Nice. While there may be some impracticalities to actually doing this for twitter, 99% of the software out there could run on a fraction of a single commodity server. People complain about the carbon burn of crypto, and they are right, but I bet it is dwarfed by the carbon burn of all the shitty over-provisioned and over-architected CRUD apps running interpreted languages. Unfortunately with universities teaching pyt…

[deleted]

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#296

Post by seritools »

threeseed wrote: post #184
seritools wrote: post #176

Earlier quoted context omitted.

> if it really was [which it isn't]

But many people including the OP think it is. It’s like me running a web crawler on my phone and saying I can replace Google.

What? The OP covers much more than "just shuffling tweet sized buffers to network offload cards"

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#297

Post by etaioinshrdlu »

jacobsenscott wrote: post #290

Nice. While there may be some impracticalities to actually doing this for twitter, 99% of the software out there could run on a fraction of a single commodity server. People complain about the carbon burn of crypto, and they are right, but I bet it is dwarfed by the carbon burn of all the shitty over-provisioned and over-architected CRUD apps running interpreted languages. Unfortunately with universities teaching pyt…

I've been experimenting with GPT3 as a "compiler" and it's always amazing when it works. Extrapolating from here, I think this is right on the nose -- we feed in any high level language and get reasonably efficient assembly out the other end, furthermore, the assembly is more human-like than a compiler would give you.

There's some big problems with this approach today, namely, it's not always right, and it may sometimes be half right (miss edge cases).

But think of where this AI technology is headed -- it stands to reason it will eventually work pretty much perfect.

And then I think we'll see another very strong trend -- large AI models replacing other forms of software. Why write a compiler when GPT3 can compile C to asm? Why write an interpreter when GPT3 can "compile" python to C?

The AI model is hilariously less-efficient than traditional software, but it will be far far cheaper and faster to create than the traditional equivalent.

What other types of software will be replaced by AI models?

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#298

Post by thorncorona »

quickthrower2 wrote: post #196
JosephRedfern wrote: post #149

Earlier quoted context omitted.

I suppose that in practice you'd need to consider burst bandwidth and not just 95/99 percentiles.

… HN thread that reinvents CDN …

Twitter is big on real time though. Each user gets served their own feed in real time.

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#299

Post by jbotdev »

jacobsenscott wrote: post #290

Nice. While there may be some impracticalities to actually doing this for twitter, 99% of the software out there could run on a fraction of a single commodity server. People complain about the carbon burn of crypto, and they are right, but I bet it is dwarfed by the carbon burn of all the shitty over-provisioned and over-architected CRUD apps running interpreted languages. Unfortunately with universities teaching pyt…

I think people overestimate how much CPU time a typical CRUD app spends on actual business logic, even with an interpreted language like Ruby or Python. What I’ve seen is the bottleneck is largely memory, such that you can pack a ton of these apps on a machine with a few cores and a lot of RAM.

The stuff that actually is CPU-bound often ends up being written in an appropriate language, or uses C extensions (e.g. ML and data science libraries for Python).

Re: Production Twitter on one machine? 100Gbps NICs and NVMe are fast

#300

Post by poslathian »

wtallis wrote: post #255
vidarh wrote: post #237

Earlier quoted context omitted.

Biggest problem with this is the lack of considering analytics.twitter.com and ads.twitter.com. Twitter stores event data about everything that happens to a tweet, and lets you target ads with a lot of precision. While some of those writes may well be acceptable to lose, letting you write to caches, effectively you need to assume there are more analytics events triggering writes to something than there are tweet view…

A Twitter-like service that fits on a single server could probably get by with the reduced revenue that comes with not offering obsessively fine-grained analytics and ad targeting.

Best comment. And one you don’t hear nearly enough from typical product and eng managers.
Post reply on HN