121–130 of 336 posts
Then, to migrate away you need all sorts of devops folks and the ability to deal with incompatibilities.
Uncertainty about pricing and the hardware bottleneck is a real problem for our users.
I just raised this point in our blog today.
Earlier quoted context omitted.
It is really the second one. The minute you want a second site or even HA at a single site the complexity and costs start to explode.
k3s is very simple to setup and and nodes to. if the machines are on the same LAN or the internet then it's not such a complex job albeit you need to know the basics of kubernetes.
Earlier quoted context omitted.
People complain about this relentlessly, but never change. Additionally, every single business owner I know complains that whatever it is they're selling, (1) it isn't worth the brain damage to sell to customers looking for the lowest price, (2) as long as people comparison shop in a harebrained way, hook pricing (aka up front pricing that looks low and turns out high) is only rational. It's not like they're providin…
>hook pricing (aka up front pricing that looks low and turns out high) is only rational. Yes, many deceptive business practices are rational acts on the part of the businesses. That doesn't mean they should be tolerated.
And it's a fact that you can get even much better on-demand (not to mention reserved) pricing from the big clouds if you're a decent startup with connections.
If one of these clouds offered fair pricing to SMBs, it could be a great bottoms up growth strategy.
(*) Not LambdaLabs afaik, but they rarely have on demand capacity anyway, and you can only get reasonable price with 3 year reservation (which is, surprise surprise, more than the hardware cost).
It has been amazing to watch this industry explode, and we believe it is great for consumers. The same instances on Amazon versus these alternative providers are 3x more expensive.
NVIDIA and many hardware providers are leaning into this trend. As clouds become more and more vertically integrated, AMD, NVIDIA, and others will benefit from spreading their hardware to more clouds.
Knowing that these models will not be running in 3 easily controlled clouds may also benefit us in the long run as each provider will have different levels of comfort with models of varying capabilities.
How are these companies even getting these GPUs? I would imagine NVIDIA would give them all to Microsoft and Google if they are in short supply.
I head up Product at Lambda. We are an NVIDIA preferred partner (and keep winning the preferred partner award). I don't know what the allocation numbers are for other companies, but we get a lot. It's in NVIDIAs best interest to spread the love around and make sure all the GPUs don't go to the hyperscalers.
Earlier quoted context omitted.
> AWS' entire business model is making the pricing so confusing My go to line here is that Cloud was a ZIRP. Like the whole entire thing. Took us ~10 years to wind up the cloud, and it will take years to unwind it, but the mass migration away is already happening. To be clear, I don't mean like AWS is going out of business or anything. Just that companies are a) realizing how insanely expensive it is, b) realizing ho…
> My go to line here is that Cloud was a ZIRP. Like the whole entire thing. Took us ~10 years to wind up the cloud, and it will take years to unwind it, but the mass migration away is already happening. This is Hacker News echo chamber stuff. There is certainly no mass migration away from the cloud. Yes, I personally saw companies in the 2010s say "we're moving everything to the cloud!" without adequate planning or c…
https://x.com/michaeldell/status/1780672823167742135?s=46&t=...
Would like to take a moment to recommend fly.io for GPU workloads. I've been building a prototype using them for the last couple of weeks and it's been great to use. I didn't have to jump through any hoops or apply for any quota adjustments to get started. And I especially appreciate how easy they make it to automatically scale your GPU instances to zero based on traffic.
Wow that is quite decent pricing actually. This makes me want to try to deploy something like llamafile/ollama or similar to fly.io+gpu for my personal on-demand llama/llm whims. (@simonw I’m looking at you— I feel like if you haven’t already done this on fly.io, that you’re probably thinking about it :) ) Seems private enough.. could throw some basic auth on top of it for me and trusted friends/family so I don’t get…