Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I am building a cloud

crawshaw.io

Post reply on HN
Print view View original

471–480 of 589 posts

Re: I am building a cloud

#471

Post by ownagefool »

electroly wrote: post #405
ownagefool wrote: post #232

Earlier quoted context omitted.

It depends what you're doing it. My app is fairly simple node process with some side car worker processes. k8s enables me to deploy it 30 times for 30 PRs, trivially, in a standard way, with standard cleanup. Can I do that without k8s? Yes. To the same standard with the same amount of effort? Probably not. Here, I'd argue the k8s APIs and interfaces are better than trying to do this on AWS ( or your preferred cloud p…

> I'd argue the k8s APIs and interfaces are better than trying to do this on AWS I think Amazon ECS is within striking distance, at least. It does less than K8S, but if it fits your needs, I find it an easier deployment target than K8S. There's just a lot less going on.

I ran renderapp in ECS before I ran it in k8s.

The deployment files / structure were mostly equivalent with the main differences being I can't shell into ECS and I lose kubectl in favour of looking at the AWS GUI ( which for me is a loss, for others maybe not ).

The main difference is k8s has a lot of optionality, and folks get analysis paralysis with all the potential there. You quickly hit this in k8s when you have to actually need the addon to get cloudwatch logs.

This is also where k8s has sharp edges. Since amazon takes care of the rest of the infrastructure for you in ECS, you don't really need to worry about contention and starving node resources resulting in killing your logging daemon, which you could technically do in k8s.

However, you'll note that this is a vendor choice. EKS Auto Mode does away with most of the addons you need to run yourself, simplifying k8s, moving it significantly closer to a vendor supported solution.

Re: I am building a cloud

#473

Post by foobarian »

valzam wrote: post #313
wutwutwat wrote: post #217

Earlier quoted context omitted.

It's not just low level, in most cases, it's also overkill. Most companies aren't "web scale" ™ and don't need an orchestrator built for google level elasticity, they need a vm autoscaling group if anything. Most apps don't need such granular control over fs access, network policies, root access, etc, they need `ufw allow 80 && ufw enable` Most apps don't need a 15 stage, docker layer caching optimized, archive promo…

If this works `git clone me@github.com:me/mine.git release_01 && ln -s release_01 /var/www/me/mine/current` then your Docker builds should also be extremely quick. Where I have seen extremely slow docker builds is with Python services using ML libraries. But those I reallly don't want to be building on the production servers. "ECS would have worked for 99% of these apps, if they even needed that." I used to agree wit…

Wait a minute, if this is on AWS then what are we talking about? On-prem k8s sounds fine to me but you don't have the ECS option.

Re: I am building a cloud

#474

Post by pxc »

This is the problem for me with the cloud:

> Finally, clouds have painful APIs. This is where projects like K8S come in, papering over the pain so engineers suffer a bit less from using the cloud. But VMs are hard with Kubernetes because the cloud makes you do it all yourself with lumpy nested virtualization. Disk is hard because back when they were designing K8S Google didn’t really even do usable remote block devices, and even if you can find a common pattern among clouds today to paper over, it will be slow. Networking is hard because if it were easy you would private link in a few systems from a neighboring open DC and drop a zero from your cloud spend. It is tempting to dismiss Kubernetes as a scam, artificial make work designed to avoid doing real product work, but the truth is worse: it is a product attempting to solve an impossible problem: make clouds portable and usable. It cannot be done.

Please learn from Unix's mistakes. Learn from Nix. Support create-before-destroy patterns everywhere. Forego all global namespaces you can. Support rollbacks everywhere.

If any cloud provider can do that, cloud IaC will finally stop feeling so fake/empty compared to a sane system like NixOS.

Re: I am building a cloud

#475

Post by ownagefool »

shimman wrote: post #320
ownagefool wrote: post #232

Earlier quoted context omitted.

It depends what you're doing it. My app is fairly simple node process with some side car worker processes. k8s enables me to deploy it 30 times for 30 PRs, trivially, in a standard way, with standard cleanup. Can I do that without k8s? Yes. To the same standard with the same amount of effort? Probably not. Here, I'd argue the k8s APIs and interfaces are better than trying to do this on AWS ( or your preferred cloud p…

Call me old fashion but I prefer tools like Dokploy that make deployment across different VPS extremely easy. Dokploy allows me to utilize my home media server, using local instances of forgejo to deploy code, to great effect. k8s appears to be a corporate welfare jobs program where trillion dollar multinational monopolistic companies are the only ones who can collectively spend 100s of millions sustaining. Since mos…

I took over tech for a POS company some years ago. They were a .net shop with about 80 developers, less than 200 concurrent connections, 6 figures spend cloud, and 0 nines uptime with a super traditional setup.

Point being, it's not the tools the causes the probem.

Re: I am building a cloud

#476

Post by ownagefool »

wredcoll wrote: post #410
shimman wrote: post #320

Earlier quoted context omitted.

Call me old fashion but I prefer tools like Dokploy that make deployment across different VPS extremely easy. Dokploy allows me to utilize my home media server, using local instances of forgejo to deploy code, to great effect. k8s appears to be a corporate welfare jobs program where trillion dollar multinational monopolistic companies are the only ones who can collectively spend 100s of millions sustaining. Since mos…

K8s is just a standardized api for running "programs" on hardware, which is a really difficult problem it solves fairly well. Is it complex? Yes, but so is the problem it's trying to solve. Is its complexity still nicer and easier to use than the previous generation of multimachine deployment systems? Also yes.

I wrote a scheduler for VMs a long time ago. k8s is basically just the same thing but for containers.

It really confuses me how someone can argue for cloud providers over a decent open solution without realising their argument is simply they don't want to be managing the thing.

And that's fine, most teams shouldn't be neck deep in managing a platform. But that doesn't make the solution bad.

Re: I am building a cloud

#477

Post by ianburrell »

ipsento606 wrote: post #419
Hasz wrote: post #412

Earlier quoted context omitted.

Took a while to find this. K8s is great, IMO most of the people with alternative setups are just rebuilding (usually worse) or compressing (specific to their use case) k8s features that have been GA for a long time. Spend some time learning it, using it to deploy simple apps, and you won't go back to deploying in a VM again imo. This only gets better with ai-assisted development, any model is going to produce much be…

I deploy prod by running a shell script I wrote that rsyncs the latest version of the codebase to my server, then sshs into the server and restarts the relevant services how could k8s improve my deployment process?

What happens when your new version is broken? Kubernetes would rollback to old version. You have to rerun the deployment script and hope you have the old version available. Kubernetes will even deploy new version to some copies, test it, and then roll out the whole thing when it works.

Also, Kubernetes uses immutable images and containers so you don't have to worry about dependencies or partial deploys.

Re: I am building a cloud

#478

Post by limaho »

evanphx wrote: post #376
limaho wrote: post #299

Earlier quoted context omitted.

This is where I'm at. Using Podman daily to run Python scripts and apps and it's been going great! However trying to build things like monitoring, secure secret injection, centralized inventory, remote logging, etc. has fallen on us. Has lead to some shadow IT (running our own container image registry, hashicorp vault instance, etc.) which makes me hesitant to share with others in the company how we're operating. I l…

Are you doing that across a fleet of machines or just one?

We have 4 servers we run containers on. Calling that a fleet feels too generous. Not much rhyme or reason as to what containers run on which server

Re: I am building a cloud

#479

Post by dodu_ »

a34729t wrote: post #390
dajonker wrote: post #74

> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…

As the strongest engineer I ever worked with commented: "Across multiple FAANG-adjacent companies, I've never seen a k8s migration go well and not require a complete reimplementation of k8s behind the APIs."

Is that because kubernetes was the right fit from the beginning, or because the initial implementation was designed around kubernetes, which caused the migration to eventually end up taking that same shape?

Re: I am building a cloud

#480

Post by stavros »

messh wrote: post #384
sahil-shubham wrote: post #131

The point about VMs being the wrong shape because they’re tied to CPU/memory resonates hard. The abstraction forces you to pay for time, not work. I ended up buying a cheap auctioned Hetzner server and using my self-hostable Firecracker orchestrator on top of it ( https://github.com/sahil-shubham/bhatti , https://bhatti.sh ) specifically because I wanted the thing he’s describing — buy some hardware, carve it into as…

Vms hosted in Hetzner auction instances is exactly how shellbox works. I published more details in here: https://shellbox.dev/blog/race-to-the-bottom.html

This is pretty cool, I turned a NUC at home into this, and would probably rather use you guys instead. However, is there a way for me to keep a session open without being connected? Sometimes I want the session to be there so I can connect/disconnect to check up on it, so I want "just disconnecting for a bit" to be different from "I don't care about this any more, destroy it".

At home, I've done that with a Zellij session (everything is tied to the session, and quitting Zellij completely means "I'm done with this". Merely disconnecting keeps it running).

Post reply on HN