I noticed quite a few spelling and grammar mistakes - could do with a bit of an edit pass?
We were wrong about GPUs
11–20 of 604 posts
Re: We were wrong about GPUs
#12We wrote all sorts of stuff this week and this is what gets to the front page. :P
Sounds like you might have been wrong in spending time on the other stuff. ;)
Who even knows what the customer is ever going to want? Pivot. Pivot. Pivot.
PS: And pouring one out for the engineering hours that went into shipping GPUs. Sometimes it's a fine product, but just doesn't fit.
Re: We were wrong about GPUs
#13I feel like these guys are missing a pretty important point in their own analysis. I tried setting up a ollama LLM on a fly.io GPU machine and it was near impossible because of fly.io limitations such as: 1. Their infrastructure doesnt support streaming responses well at all (which is important part of the LLM experience in my view) 2. The LLM itself is massive, and cant be part of the docker image I was building and…
I mean, yes? Managing giant model weight files is a big problem with getting people on-demand access to Docker-based micro-VMs. I don't think we missed that point so much as that we acknowledged it, and found some clarity in the idea that we weren't going to break up our existing DX just to fix it. If there were lots and lots and lots of people trying to self-host LLMs running into this problem, it would have been a harder call.
Re: We were wrong about GPUs
#14I noticed quite a few spelling and grammar mistakes - could do with a bit of an edit pass?
In the current days of AI I think spelling and grammar mistakes is perhaps a great way to tell it is still written by human...... ( Until AI copy this )
Re: We were wrong about GPUs
#15Side note: "we were wrong" - are there any more noble and beautiful words in the English language?
“I was wrong.”
Closely followed by, “I was right.” :)
Re: We were wrong about GPUs
#16I noticed quite a few spelling and grammar mistakes - could do with a bit of an edit pass?
Nah, fly.io has a company culture that is all about having lots of bugs and issues, and that includes blog posts.
The idea that a cloud compute provider can’t make GPU compute into an profitable business is pretty laughable.
Re: We were wrong about GPUs
#17Side note: "we were wrong" - are there any more noble and beautiful words in the English language?
It's great when people admit they were wrong but I can't help to find those headlines clickbaity. A bit like "stop doing this..." and we think: omg, am I doing the same deadly mistake?
I love the idea that Kurt needed to better couch a post saying he was wrong about something.
Re: We were wrong about GPUs
#18I feel like these guys are missing a pretty important point in their own analysis. I tried setting up a ollama LLM on a fly.io GPU machine and it was near impossible because of fly.io limitations such as: 1. Their infrastructure doesnt support streaming responses well at all (which is important part of the LLM experience in my view) 2. The LLM itself is massive, and cant be part of the docker image I was building and…
I mean, yes? Managing giant model weight files is a big problem with getting people on-demand access to Docker-based micro-VMs. I don't think we missed that point so much as that we acknowledged it, and found some clarity in the idea that we weren't going to break up our existing DX just to fix it. If there were lots and lots and lots of people trying to self-host LLMs running into this problem, it would have been a…
Did you consider other use cases in which people need custom models and inference other than just open source LLMs ?
Re: We were wrong about GPUs
#19Side note: "we were wrong" - are there any more noble and beautiful words in the English language?
[deleted]
Re: We were wrong about GPUs
#20> GPUs terrified our security team.
Ha ha, it didn't terrify Modal. It ships with all those security problems, and pretends it doesn't have them. Sorry Eric.