Cloud services like AWS or Google Cloud Platform may be the wrong choice • Hacker News Board

Re: Cloud services like AWS or Google Cloud Platform may be the wrong choice

#201

Post by oceanplexian » Sat, Oct 15, 2022, 4:04 AM UTC

sofixa wrote: post #167

oceanplexian wrote: post #149

Earlier quoted context omitted.

If you open a ticket with a real remote hands you should get a response back in minutes; typically someone will be on-site in your cage in under an hour. You also don’t “deploy a server” to reduce load you plan ahead every few months and deploy thousands at a time. Even then- if you have a good relationship with a really good systems integrator, they can ship and rack machines in a matter of days, not weeks. I’m incr…

> If you open a ticket with a real remote hands you should get a response back in minutes; typically someone will be on-site in your cage in under an hour Remote hands won't order your servers, configure your networking, install OSes/configure your PXE, and all the other tedious things running your own DC entails. Yes, most DIY DCs are done terribly, that's to whole point - if so many people struggle with that, doesn…

They will do whatever is in the contract. Yes, they will hook up a crash cart and PXE the server (If you need that), done it hundreds of times in the old days. However in modern datacenters no one "installs OSes and configures the network". You plug it in, turn it on, everything self-provisions and starts serving traffic.

Re: Cloud services like AWS or Google Cloud Platform may be the wrong choice

#202

Post by MacsHeadroom » Sat, Oct 15, 2022, 6:57 AM UTC

mwcampbell wrote: post #89

fareesh wrote: post #81

I am currently in the middle of setting up AWS and that decision graph made me chuckle because it resonates quite deeply. I need to do GPU inference but I don't want to run the machine 24x7. I may use it for about 4 hours per day at best. Lambda doesn't offer GPUs and neither does ECS+Fargate. It seems like I could setup an endpoint using Sagemaker and then destroy it when no longer needed, and automate all of this b…

If your GPU inference can run on an Intel integrated GPU, you could rent a dedicated server from OVH for ~$130 a month and use the integrated GPU on that. I don't know about affordable dedicated servers from mature providers with Nvidia GPUs.

I pay a tiny fraction of that for a dedicated served from Hetzner with a modern iGPU transcoding and streaming video 24/7.

Re: Cloud services like AWS or Google Cloud Platform may be the wrong choice

#203

Post by Nextgrid » Sat, Oct 15, 2022, 1:03 PM UTC

nine_k wrote: post #186

Nextgrid wrote: post #173

Earlier quoted context omitted.

> You don't need k8s for a small operation, just spin a couple of VMs and set them up via a few lines of Ansible. You can replace "couple VMs" with a dedicated Hetzner/OVH/Kimsufi server, it'll be the same except you won't get ripped off on egress bandwidth and performance.

I agree that AWS egress bandwidth is a rip-off, but cloud != AWS only. Many cloud providers, from DO to Vultr, offer sane egress prices. The cheapest dedicated server at Hetzner (an excellent provier indeed) is €44.39 / mo, while their cheapest VM option is €4.51 / mo, literally an order of magnitude less. If you don't need the power of a dedicated server for your small project (and even 10 small projects), you don't…

OVH has their Kimsufi range which gives you low-power dedicated with unmetered bandwidth and the basic ones go under 10 bucks a month. It's 100Mbps, but unlimited traffic, and a real low-power CPU is most likely still better than cloud "vCPUs".

Re: Cloud services like AWS or Google Cloud Platform may be the wrong choice

#204

Post by Nextgrid » Sat, Oct 15, 2022, 1:04 PM UTC

tatersolid wrote: post #188

Nextgrid wrote: post #168

Earlier quoted context omitted.

I don't see the point of public cloud. In practice, it still requires a sysadmin (now called "DevOps engineers") so it's not any better than rented bare-metal in terms of maintenance overhead, while still being extremely expensive. Use a managed PaaS to begin with (you pay more but it does genuinely save you time as there is no management overhead), then when you're ready to do things yourself go straight to hosted b…

> so it's not any better than rented bare-metal in terms of maintenance overhead In my experience the maintenance overhead of the cloud is much lower. My dayjob (B2B SaaS) spent about 75% of the infrastructure team’s time on things like patching switch firmware, balancing UPS loads, diagnosing flaky switch ports or transceivers, managing logging growth, etc. None of that made our products better from a customer persp…

> patching switch firmware, balancing UPS loads, diagnosing flaky switch ports or transceivers, managing logging growth

I've never had to worry about messing with switches/cabling/UPSes with my Hetzner or OVH servers.

I'm not sure why people always believe that bare-metal == self-managed colo. That's not the case and colo only really makes sense for very large companies who can actually save by buying & managing their own hardware or have specific requirements that dedicated server providers don't offer.

Log growth can be addressed by using a managed service (including AWS Cloudwatch if you really wanted to, but you'd have to be a masochist). Frankly, you'd have the same issue if you were on EC2.

Re: Cloud services like AWS or Google Cloud Platform may be the wrong choice

#205

Post by fareesh » Sat, Oct 15, 2022, 1:24 PM UTC

Moissanite wrote: post #153

fareesh wrote: post #81

I am currently in the middle of setting up AWS and that decision graph made me chuckle because it resonates quite deeply. I need to do GPU inference but I don't want to run the machine 24x7. I may use it for about 4 hours per day at best. Lambda doesn't offer GPUs and neither does ECS+Fargate. It seems like I could setup an endpoint using Sagemaker and then destroy it when no longer needed, and automate all of this b…

Depends on what "4 hours per day" really means. If you want an interactive endpoint, putting in the work to set up an ECS task you can start and stop feels like the best approach. If you have longer-running inference tasks and just want to pick up results asynchronously, Batch (which is a layer on top of ECS) seems like the way to go. SageMaker might have an abstraction which is a closer fit for your particular use-c…

The task takes about 1-4 minutes. In an ideal world, it can start immediately and then shutdown, but more realistically it will have to spin up or batch things up at a convenient point in time. Sagemaker seems more expensive that doing it via ECS.

Re: Cloud services like AWS or Google Cloud Platform may be the wrong choice

#206

Post by sofixa » Sat, Oct 15, 2022, 1:29 PM UTC

oceanplexian wrote: post #201

sofixa wrote: post #167

Earlier quoted context omitted.

> If you open a ticket with a real remote hands you should get a response back in minutes; typically someone will be on-site in your cage in under an hour Remote hands won't order your servers, configure your networking, install OSes/configure your PXE, and all the other tedious things running your own DC entails. Yes, most DIY DCs are done terribly, that's to whole point - if so many people struggle with that, doesn…

They will do whatever is in the contract. Yes, they will hook up a crash cart and PXE the server (If you need that), done it hundreds of times in the old days. However in modern datacenters no one "installs OSes and configures the network". You plug it in, turn it on, everything self-provisions and starts serving traffic.

> However in modern datacenters no one "installs OSes and configures the network". You plug it in, turn it on, everything self-provisions and starts serving traffic.

Absolutely agreed. That's what i used to in part do, and it's a massive effort to do everything automatically and efficiently, and it needs multiple people's time to create and maintain all the infrastructure, glue between different systems, scripts, tools. Even components as basic as DHCP suck absolutely (your options are either something from the 1990s, isc-dhcp-server, which lacks an API in any real sense, or Kea, made from the same people, which really shows), and before Tinkerbell there was literally nothing that could be used to automate such a thing at scale.

And more to my point, how many datacenters do you think are "modern"? I've only encountered one that was starting to get there (where i used to work before an acquisition by a company with arcane practices in their DCs), and having worked with hundreds of customers with "on prem" stuff, for the vast majority it's a legacy horror show.

Re: Cloud services like AWS or Google Cloud Platform may be the wrong choice

#207

Post by locutous » Sat, Oct 15, 2022, 2:48 PM UTC

tatersolid wrote: post #188

Nextgrid wrote: post #168

Earlier quoted context omitted.

I don't see the point of public cloud. In practice, it still requires a sysadmin (now called "DevOps engineers") so it's not any better than rented bare-metal in terms of maintenance overhead, while still being extremely expensive. Use a managed PaaS to begin with (you pay more but it does genuinely save you time as there is no management overhead), then when you're ready to do things yourself go straight to hosted b…

> so it's not any better than rented bare-metal in terms of maintenance overhead In my experience the maintenance overhead of the cloud is much lower. My dayjob (B2B SaaS) spent about 75% of the infrastructure team’s time on things like patching switch firmware, balancing UPS loads, diagnosing flaky switch ports or transceivers, managing logging growth, etc. None of that made our products better from a customer persp…

Most of these stated problems just go away if you choose a co-location vs your previous on prem.

Whenever I price out cloud rentals, I consistently find that the fees will equal the cost of the machine in less than a year.

Re: Cloud services like AWS or Google Cloud Platform may be the wrong choice

#208

Post by marcus_holmes » Sun, Oct 16, 2022, 1:21 AM UTC

waz0wski wrote: post #113

marcus_holmes wrote: post #59

Earlier quoted context omitted.

I got handed a Terraform project for a GCP-based service. Simple dev, staging, prod environment. Secrets managed by secret manager, SQL Run without a public IP address for prod (but accessible via SSH for admins). I more or less gave up after a month of beating my head on the brick wall. We hired an expert. Took him another month to get it all more or less sorted. There were still aspects that we wanted that we could…

That's a generic and well documented stack that utilizes GCP defaults and works out of the box. An "expert" should not take a month to fail to set it up. I've deployed similar, additionally including GKE, via terraform in a day - Checking TF code for an example 3-env GCP/GKE/CloudSQL stack it's less than 300 LoC That said, it's not all good - my ongoing complaint with terraforming GCP is that the provider lags behind…

Maybe we should have hired you instead of a "terraform expert" ;)

and yes, which features worked in GCP but not in terraform GCP were not clear, and there was always a "this works in beta" thing going on.

Re: Cloud services like AWS or Google Cloud Platform may be the wrong choice

#209

Post by lelanthran » Sun, Oct 16, 2022, 4:49 AM UTC

buzzdenver wrote: post #84

f0e4c2f7 wrote: post #19

This article is written in kind of a controversial way but it seems like the throughline of the argument is something like "use heroku until you have 100k users". This seems very reasonable to me. I thought it was going to be a pitch for on prem, which is also fine for certain scales. I think generally the scaling steps from startup to megacorp go: Heroku/Dokku > Public Cloud >Dedicated servers in someone else's DC >…

> "use heroku until you have 100k users" Thing is that for most/many startups 100k users is not a lot. Rejiggling your basic infra just as your growth is starting to accelerate is a non-trivial task, a risk, and something that doesn't fundamentally move the needle.

> Thing is that for most/many startups 100k users is not a lot.

Depends; if you're a startup offering free or ad-supported services and the exit plan is "be bought out by existing entrenched competitor", then, yes, 100k users is not enough to hit your goals.

If you're a startup offering B2B services, even 10k users is enough to be madly profitable.

Re: Cloud services like AWS or Google Cloud Platform may be the wrong choice

#210

Post by makestuff » Tue, Oct 18, 2022, 2:10 PM UTC

0x445442 wrote: post #143

makestuff wrote: post #112

Earlier quoted context omitted.

We have been using CDK on AWS and it is really nice because you can do complex things through Typescript.

Been a couple of years since I used AWS and I remember when CDK was just coming out. My big question at the time was whether or not the CDK would alert you to errors at compile time and save a bunch of whaling and gnashing of teeth that comes with Terraform.

Yeah it is much more mature now. Basically the CDK will generate YAML cloudformation templates at compile time so any errors are generally caught then. If you utilize typescript it is even safer since you know there are no missing parameters or anything like that.

I have not used terraform that much, but they did launch a CDK for terraform that does a similar thing https://www.terraform.io/cdktf. Basically you write in code and at compile time it converts it to the terraform templates.