Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I am building a cloud

crawshaw.io

Post reply on HN
Print view View original

231–240 of 589 posts

Re: I am building a cloud

#231

Post by BirAdam »

dajonker wrote: post #74

> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…

So... if you're at the point where you're using a single VM, I have to ask why bother with docker at all? You're paying a context switch overhead, memory overhead, and disk overhead that you do not need to. Just make an image of the VM in case you need to drop it behind an LB.

Re: I am building a cloud

#232

Post by ownagefool »

adamtulinius wrote: post #89
dajonker wrote: post #74

> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…

If you spin up Kubernetes for "a couple of containers to run your web app", I think you're doing something wrong in the first place, also coupled with your comment about adding SDN to Kubernetes. People use Kubernetes for way too small things, and it sounds like you don't have the scale for actually running Kubernetes.

It depends what you're doing it.

My app is fairly simple node process with some side car worker processes. k8s enables me to deploy it 30 times for 30 PRs, trivially, in a standard way, with standard cleanup.

Can I do that without k8s? Yes. To the same standard with the same amount of effort? Probably not. Here, I'd argue the k8s APIs and interfaces are better than trying to do this on AWS ( or your preferred cloud provider ).

Where things get complicated is k8s itself is borderline cloud provider software. So teams who were previously good using a managed service are now owning more of the stack, and these random devops heros aren't necessarily making good decisions everywhere.

So you really have three obvious use cases:

a) You're doing something interesting with the k8s APIs, that aren't easy to do on a cloud provider. Essentially, you're a power user. b) You want a cloud abstraction layer because you're multi-cloud or you want a lock-in bargaining chip. c) You want cloud semantics without being on a cloud provider.

However, if you're a single developer with a single machine, or a very small team and you're happy working through contended static environments, you can pretty much just put a process on a box and call it done. k8s is overkill here, though not as much as people claim until the devops heros start their work.

Re: I am building a cloud

#233

Post by trvz »

Medowar wrote: post #151
zuhsetaqi wrote: post #135

Earlier quoted context omitted.

Just for comparison I use the cheapest netcup root server: RS 1000 G12 AMD EPYC™ 9645 8 GB DDR5 RAM (ECC) 4 dedicated cores 256 GB NVMe Costs 12,79 € Results with the follwing command: fio --name=randreadwrite \ --filename=testfile \ --size=5G \ --bs=4k \ --rw=randrw \ --rwmixread=70 \ --iodepth=32 \ --ioengine=libaio \ --direct=1 \ --numjobs=4 \ --runtime=60 \ --time_based \ --group_reporting IOPS Read: 70.1k IOPS W…

That is a bit of a unfair comparison. The Hetzner and DO instances are shared hosting, you are using dedicated ressources. Using a Netcup VPS 1000 G12 is more comparable. read: IOPS=18.7k, BW=73.1MiB/s write: IOPS=8053, BW=31.5MiB/s Latency Read avg: 5.39 ms, P99.9: 85.4 ms, max 482.6 ms Write avg: 3.36 ms, P99.9: 86.5 ms, max 488.7 ms

Storage performance is practically always a shared resource, and that's what y'all are talking about here...

Re: I am building a cloud

#234

Post by ownagefool »

goombaskoop wrote: post #133
dajonker wrote: post #93

Earlier quoted context omitted.

I totally agree, but that's not what happens in reality: the average devops knows k8s and will slap it onto anything they see (if only so they can put in on their resume). The average manager hears about k8s, gets convinced they need and hires beforementioned devops to build it.

> the average devops knows k8s and will slap it onto anything they see This is certainly the case from all the third person accounts I hear. Online. I never actually met a single one that is like that, if anything, those same people are the ones that are first to tell me about their Hetzner setups.

To be fair, I have k8s on my hetzner :p

Re: I am building a cloud

#235

Post by philipallstar »

chrisweekly wrote: post #207
darkwater wrote: post #145

Earlier quoted context omitted.

And the average developer doesn't even know where to start to deploy things in prod. When the feature product asks passes QA... to the next sprint! we are done!

Whose responsibility is it to establish the prerequisite CICD pipelines, HITL workflows, and Observability infr in order for devs to shepherd changes to prod (and track their impact)? Hint: it's not the developer's.

This was the point of "devops" (the concept, not the job title): the team should be responsible for development and operations, so one isn't prioritised hugely over the other.

Re: I am building a cloud

#236

Post by tlb »

phrotoma wrote: post #210
tlb wrote: post #165

I think clouds pay a huge abstraction penalty to allow tiny VMs. I guess it helps with onboarding and $10 personal VPNs. But I have never needed a fraction of a computer. I want to rent some number of full computers of various sizes, consisting of CPU, memory, and flash disk. Hetzner is closer than AWS, and I think/hope that’s what Crawshaw is aiming for.

Allow? I understood tiny VM's to be something (at least AWS) added to try to squeeze more utilization out of idle hardware.

I understand the appeal from AWS's perspective. Customer A pays for a 32 vCPU VM, which they run on 32-core hardware. Then they can also squeeze in customer B's 1 vCPU instance running a blog, and no one notices. Free money!

But I don't want to be either of those customers. It means the whole system has an extra layer of abstraction, so they can juggle VMs around. It's why you need slow EBS instead of just getting a flash drive in the same case as the CPU, with 0.01x the latency.

Re: I am building a cloud

#237

Post by BirAdam »

satnhak wrote: post #229

AWS. Months of complex dev work to build using their CDK. Terrible disk speed. Frustrating permissions systems. Tiny deployments that take 30 minutes. Rollbacks that get stuck for hours. What you end up with is about 4 CPUs and 16Gb of RAM for $1000+ per month. No wonder Bezos could send his wife and Katie Perry on a jolly into space. The world's richest man 1 IOP at a time. For that money I can get 5 big bare metal…

If you're using cloudflare tunnels, you don't even need to be on OVH. You could seriously host anywhere, like your own basement.

Re: I am building a cloud

#239

Post by mkj »

BirAdam wrote: post #231
dajonker wrote: post #74

> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…

So... if you're at the point where you're using a single VM, I have to ask why bother with docker at all? You're paying a context switch overhead, memory overhead, and disk overhead that you do not need to. Just make an image of the VM in case you need to drop it behind an LB.

How is docker a context switch overhead? It's the same processes running on the same kernel.

Re: I am building a cloud

#240

Post by bitexploder »

rantanplan wrote: post #201
littlestymaar wrote: post #180

Earlier quoted context omitted.

I have nom doubt that there are legit use cases for something like k8s at Google or other multi-billion companies. But if its use was confined to this use case, pretty much nobody would be using it (unless as a customer of the organization's infra) and barely would be talking about it (like how there isn't too much talk about Borg). The reason k8s is a thing in the first place is because it's being used by way too ma…

The legit use case for companies like Google/Amazon etc is only to sell it to customers. None of these companies use K8s internally for real critical workloads.

Ehm, that is simply not true. Google built it for themselves first. It is essentially the open source version of the internal architecture. It gets used.
Post reply on HN