Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Who operates at scale without containers?

news.ycombinator.com

Post reply on HN
Print view View original

301–310 of 446 posts

Re: Ask HN: Who operates at scale without containers?

#301

Post by nhoughto »

KerrAvon wrote: post #292
nhoughto wrote: post #270

Earlier quoted context omitted.

Recent convo with Cloudflare they said they are mostly container-less, debian packages installed and running on the the host, which is impressive at their scale and complexity. They did seem to think that this approach was hitting its limits and what the future might look like when I spoke to ppl there recently (end 2021), assumption being the future could be containers.

Scaling limits or technical ones? If scaling limits, should still be pretty solid for homelab usage.

seemed more to be technical and organisational. More and more teams all the time, more and more products, more and more technical requirements / tuning etc, needed a strong abstraction to reason about it all.

Homelab would be fine =)

Re: Ask HN: Who operates at scale without containers?

#302

Post by selcuka »

goodpoint wrote: post #257
spicybright wrote: post #167

Earlier quoted context omitted.

And at the very least, being old lets you be able to evaluate these new things more accurately.

It obviously does, but experience is not the only way: you can always read what people wrote and did in the past. It only takes some humility.

You can, and it will certainly help, but as the old saying goes "unfortunately no one can be told what the Matrix is, you have to see it for yourself". In some contexts no amount of reading will give you the insight that experience does.

Re: Ask HN: Who operates at scale without containers?

#303

Post by maccolgan »

senorsmile wrote: post #296
JohnBooty wrote: post #282

Earlier quoted context omitted.

Devops is not my area of expertise to put it mildly, but "a single big-ass server" scenarios are often pretty well covered by "a second big-ass server configured as a mirror/hot-spare", right? Depends on how many nines of uptime you need, of course, and other such things. (Obviously, there are reams of cases where this isn't sufficient)

In 2009 I had a pair of (what I thought at the time) very cleverly architected VM servers with complete redundancy between them. Either one could be pulled and there'd maybe be a few seconds of data loss at the worst. One day lightning struck the building and even managed to "jump" the grounded pair of UPS's and fried all hard disks in both nodes. Thankfully I had both onsite and offsite backups; but it took a couple…

I think it's always better to have the mirror be located in a separate datacenter and then connect both via a SDN.

Re: Ask HN: Who operates at scale without containers?

#304

Post by hkt »

rattlesnakedave wrote: post #226
usaphp wrote: post #126

Earlier quoted context omitted.

Or it could be that you just became old and don't want to learn new things anymore ))

This certainly seems more common. Every time I’ve had to refactor 20 year old cruft it’s been because of engineers with the launch-and-iterate mentality 20 years ago, that stopped caring about the “iterate” part once they gained enough job security.

Blaming the engineers rather than the businesses seems.. odd. It isn't like the people writing that code had control over what the company set as priorities. If anything, what you experienced seems to be a reflection of corporate short termism.

Re: Ask HN: Who operates at scale without containers?

#306

Post by locusofself »

I work at Microsoft and we have a lot of big services that run on Windows Servers. There is orchestration with a system called "Service Fabric" that schedules the applications and handles upgrades sortof like kubernetes does, but for the most part there are no containers involed.

Re: Ask HN: Who operates at scale without containers?

#307

Post by jugg1es »

locusofself wrote: post #306

I work at Microsoft and we have a lot of big services that run on Windows Servers. There is orchestration with a system called "Service Fabric" that schedules the applications and handles upgrades sortof like kubernetes does, but for the most part there are no containers involed.

I did a POC using service fabric years ago but I have not really read anything about it since. My company at the time was unable to give me three servers per tenant, so I had to drop it - but it's good to hear it's still being used.

Re: Ask HN: Who operates at scale without containers?

#308

Post by gibspaulding »

turkeywelder wrote: post #42
wanderr wrote: post #28

Grooveshark didn't use any of that. We were very careful about avoiding dependencies where possible and keeping our backend code clean and performant. We supported about 45M MAU at our biggest, with only a handful of physical servers. I'm not aware of any blog posts we made detailing any of this, though. And if you're not familiar with the saga, Grooveshark went under for legal, not technical reasons. The backend API…

I miss Grooveshark so much - Licensing issues aside it was one of the best UIs for music ever. I'd love to hear more stories about the backend

My feelings as well! The way it put your queue front and center and gave you so much control over how things were added worked really well for me. Spotify leaves a lot to be desired in its UI.

I still remember opening the site one day and reading the weird appology letter. I think that was the first time I saw something I really cared about just dissappear off of the internet.

Re: Ask HN: Who operates at scale without containers?

#309

Post by dknecht »

cyberge99 wrote: post #11

I believe tools like nomad and consul shine here. Using nomad as a job scheduler and deployer allows you to use various modules for jobs: java, shell, ec2, apps (and containers). I use it in my homelab and it’s great. That said, I don’t use it professionally. I think Cloudflare is running this stack alongside firecracker for some amazing edge stuff.

That is correct. We run baremetal and all process run on all boxes. The one exception is we put our remote browsing product in gvisor which does use containers.

Re: Ask HN: Who operates at scale without containers?

#310

Post by tschellenbach »

We do, power activity feeds and chat for a billion end users using cloudformation, cloud-init and puppet on the infra side. Code is based on Go, rocksdb and raft.

Don't see why we would want the overhead of K8 and docker. Infrastructure is already defined in code, everything is redundant and automated, why have 1 extra layer that can break and cause performance issues?

Use docker for local dev though its good for that.

Post reply on HN