Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alternative clouds are booming as companies seek cheaper access to GPUs

techcrunch.com

Post reply on HN
Print view View original

91–100 of 336 posts

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#91

Post by matsz »

baobabKoodaa wrote: post #30
matsz wrote: post #29

Earlier quoted context omitted.

This is precisely why I stick to dedicated servers for my own personal projects. $40/month for a machine, doesn't get any more predictable than this.

If you rent a dedicated server from AWS, you will be hit for various additional fees which will likely dwarf that $40/month. So the issue here is not really shared vs dedicated instances. The issue here is that a particular cloud provider (namely AWS) has set up an opaque fee structure.

Oh no, I would never go AWS for dedicated. Should've specified that in my comment so people don't get disappointed if they try to get that from AWS.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#92

Post by ikiris »

thorncorona wrote: post #65
dinvlad wrote: post #50

Earlier quoted context omitted.

Yeah, Hetzner has 20 vCPU 64GB RAM i5-13500 servers for $40/mo: https://www.hetzner.com/dedicated-rootserver/matrix-ex/ This is ~10x cheaper than the closest AWS option, and without the extra fees. Also of note that Hetzner is profitable, which means AWS has been operating at an insane markup.

> Also of note that Hetzner is profitable, which means AWS has been operating at an insane markup. You can also evidence of this on their 10-Q :^)

If you think offering a cloud service is at all the same as offering someone a box in a rack, I invite you to compete in the space.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#93

Post by bauruine »

apitman wrote: post #77
bauruine wrote: post #71

Earlier quoted context omitted.

Yeah it's more than absurd. I have a rented server that does 230TB egress a month. That's 4710$ in AWS egress fees (0.02$ per GB). I pay 40 Euros or about 0.0002 per GB and that includes an AMD Ryzen 5 3600 with 64GB RAM.

May I ask who your provider is?

That's https://mevspace.com/ but hetzner, scaleway, ovh, psychz.net, reliablesite.net etc. have similar pricing. If you increase your budget to 100 there are many, many more in Europe and the US.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#95

Post by indigodaddy »

zackangelo wrote: post #46

Would like to take a moment to recommend fly.io for GPU workloads. I've been building a prototype using them for the last couple of weeks and it's been great to use. I didn't have to jump through any hoops or apply for any quota adjustments to get started. And I especially appreciate how easy they make it to automatically scale your GPU instances to zero based on traffic.

Wow that is quite decent pricing actually. This makes me want to try to deploy something like llamafile/ollama or similar to fly.io+gpu for my personal on-demand llama/llm whims. (@simonw I’m looking at you— I feel like if you haven’t already done this on fly.io, that you’re probably thinking about it :) ) Seems private enough.. could throw some basic auth on top of it for me and trusted friends/family so I don’t get crazy bills. But privacy-wise with fly.io I think it’s good enough.

Only problem might be everytime it spins up to download the large model might be wasteful as far as getting charged for network/bw usage— wonder if it would be more cost-efficient to have persistent storage or just see how much time and bw it is to download on every cold start…

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#96

Post by hn_throwaway_99 »

rco8786 wrote: post #72
hn_throwaway_99 wrote: post #49

Earlier quoted context omitted.

> My go to line here is that Cloud was a ZIRP. Like the whole entire thing. Took us ~10 years to wind up the cloud, and it will take years to unwind it, but the mass migration away is already happening. This is Hacker News echo chamber stuff. There is certainly no mass migration away from the cloud. Yes, I personally saw companies in the 2010s say "we're moving everything to the cloud!" without adequate planning or c…

> for many, many companies the cost of hiring people to manage all this infrastructure and services is usually way more than a well-managed cloud project. Yea, this is the line that everyone uses. I've worked at these companies, and the reality just doesn't line up with that. You end up still needing your whole Ops team, they're just building cloud tooling instead of on-prem tooling. As to whether or not the migratio…

> I just mean that as a whole we're going to be moving back to simpler deployments, simpler architectures, and most importantly, fixed/predictable costs.

That I can definitely agree with, I just believe that's fully possible to do with cloud (for the most part, though there are certainly some head turners like the recent news that AWS was charging for forbidden attempts on private S3 buckets, which is bonkers) and cloud cost management tools.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#97

Post by dinvlad »

notarealllama wrote: post #80
dinvlad wrote: post #50

Earlier quoted context omitted.

Yeah, Hetzner has 20 vCPU 64GB RAM i5-13500 servers for $40/mo: https://www.hetzner.com/dedicated-rootserver/matrix-ex/ This is ~10x cheaper than the closest AWS option, and without the extra fees. Also of note that Hetzner is profitable, which means AWS has been operating at an insane markup.

Dang, that is a fantastic deal. The €100 / month is even better - DDR5 RAM, 2TB NVME raid 1, and it's all customizable too. Just have to wait for Ubuntu 24.04 to be available and might have to make this switch.

Yeah, they're all pretty fantastic. Can even upgrade existing servers with more RAM/disk, or store ~250 TB of data for $400/mo etc.

Not to mention unlimited free 1 Gbit egress/server

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#98

Post by oooyay »

baobabKoodaa wrote: post #30
matsz wrote: post #29

Earlier quoted context omitted.

This is precisely why I stick to dedicated servers for my own personal projects. $40/month for a machine, doesn't get any more predictable than this.

If you rent a dedicated server from AWS, you will be hit for various additional fees which will likely dwarf that $40/month. So the issue here is not really shared vs dedicated instances. The issue here is that a particular cloud provider (namely AWS) has set up an opaque fee structure.

I suspect they're not getting a dedicated server from Amazon. Anyone doing that is either foolish or is so locked into AWS infrastructure that it cost-wise makes more sense than to expand a network to another provider. Fun times, indeed.

What I started doing is just running my own "cloud" out of my house for personal projects. I have all of the things I need. There's some overhead in terms of maintenance and up-front setup cost in terms of time and equipment, but after that it's pretty smooth sailing.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#99

Post by yashg »

notarealllama wrote: post #80
dinvlad wrote: post #50

Earlier quoted context omitted.

Yeah, Hetzner has 20 vCPU 64GB RAM i5-13500 servers for $40/mo: https://www.hetzner.com/dedicated-rootserver/matrix-ex/ This is ~10x cheaper than the closest AWS option, and without the extra fees. Also of note that Hetzner is profitable, which means AWS has been operating at an insane markup.

Dang, that is a fantastic deal. The €100 / month is even better - DDR5 RAM, 2TB NVME raid 1, and it's all customizable too. Just have to wait for Ubuntu 24.04 to be available and might have to make this switch.

Just setup two Ubuntu 24.04 servers in last couple of days. One at Hetzner and another on AWS.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#100

Post by toast0 »

zeroxfe wrote: post #44
matsz wrote: post #29

Earlier quoted context omitted.

This is precisely why I stick to dedicated servers for my own personal projects. $40/month for a machine, doesn't get any more predictable than this.

There's no way you're getting a machine for $40/month. :-) Did you mean a dedicated VM or VPS? (I have a bunch of actual dedicated machines with different providers, and this would save me a lot of money.) (Edit: holy moly those prices are fantastic!)

I've got a dedicated machine for $30/month. It's ancient, a xeon L5640 with 16 GB ram, and 1 TB spinning disk, but it's dedicated and it works great. Well actually, the first one stopped working well, and I got a replacement with double those specs for the same price; and the second one is working great. I also run with full disk encryption, because I don't trust their opsec on wiping drives, so that's a bit of a hassle to reboots, I have to get a console with IPMI and put in the disk password, although I saw something [1] last night that inspires me to consider the possibility of automation.

I recommend shopping at https://lowendbox.com/ and https://lowendtalk.com/categories/offers

[1] https://github.com/emtiu/freebsd-outerbase

Post reply on HN