For image generation, this has already happened. To what degree, I can't tell, as I don't do image generation much so I don't have numbers on Midjourney subscriptions or any other image-AI-as-a-service sites. But civitai.com has become a place where people share their models, based off of Stable Diffusion or other similar bases, with various fine-tunings to achieve desired results. You name it, you can find a model f…
Civit ai is like 99% porn though. Most production usage of image gen is google or open ai as they are by far the best
Local AI needs to be the norm
781–790 of 804 posts
Re: Local AI needs to be the norm
#782Re: Local AI needs to be the norm
#783Earlier quoted context omitted.
Curious, why did Zed with ACP not work for you?
Because I wanted the full ide on my iPhone so I can code while away from my laptop doing fun stuff with my kids. And I don’t like the Claude codex fire and forget approach. The ide I built has a full terminal, file system, git integration and AI agent. It uses a private cloud Linux container that is persistent so I can install packages and do anything I want from any phone, computer or browser. It’s amazing that we l…
Re: Local AI needs to be the norm
#784Earlier quoted context omitted.
The only way that this even vaguely works, best I can tell, would be on that decade-or-two timeline, but therein lies the problem: all this money getting pumped into data centers right now is going to produce data centers that are running old, inefficient, slow GPUs by 5-years-from-now standards. And GPUs are by far the most expensive part of these data centers… having the buildings is barely an asset. We’re investin…
These datacenters are already running old, inefficient, slow GPUs from five years ago in addition to newly released cards, because anything newer than that is extremely bottlenecked and they need all the compute they can get. Why should it be any different in five years' time? Even nVidia is rumored to be about to bring back the RTX 3060 which is an Ampere architecture card that got released around 2021. It's just fi…
Re: Local AI needs to be the norm
#785Earlier quoted context omitted.
I don't know that any AI bubble will pop. AI can be used to accelerate therapies, cures, make scientific advancements. Add to that, quantum science technology which if successful, should accelerate things, depending on who's the one at the wheel. Problem is the gap between now and then (e.g. age abundance). It's going to be a difficult road for good number of the population until that day comes. I'm scouting potentia…
Both the bubble popping and it's legitimate use cases can exist at the same time. For example, the www bubble popped, but the Internet didn't go away
Re: Local AI needs to be the norm
#786Earlier quoted context omitted.
That DRAM would get even more use if it was removed from these machines and placed into a shared pool :) I joke, but thanks to the brutal DRAM market there has been some movement in this direction lately...
I think the question of who controls the model is far more pressing than the question of who owns the DRAM. It's easy to rattle off a half-dozen different vectors of likely enshittification over the next few years -- ranging from increasing censorship, to lower rate limits, to removal of existing features and forced addition of unwelcome new ones, to extortionate price increases, to unexplained and irreversible accou…
Re: Local AI needs to be the norm
#787Earlier quoted context omitted.
Civit ai is like 99% porn though. Most production usage of image gen is google or open ai as they are by far the best
It's not quite that high anymore, and they hard-split the NSFW stuff out to civitai.red So the main site can be respectable enough for payment processors. There's plenty of SFW stuff to go around now.
Re: Local AI needs to be the norm
#788Earlier quoted context omitted.
These datacenters are already running old, inefficient, slow GPUs from five years ago in addition to newly released cards, because anything newer than that is extremely bottlenecked and they need all the compute they can get. Why should it be any different in five years' time? Even nVidia is rumored to be about to bring back the RTX 3060 which is an Ampere architecture card that got released around 2021. It's just fi…
If those data centers were good enough, they’d save themselves a few billion dollars and just do more of the same, wouldn’t they? Many current video games struggle on the 3060— it’s like 10 times slower for interference than a 4090 even. They’re reintroducing it because their upstream business of selling brand new insanely expensive GPUs required for every new data center is making it impossible for people to buy GPU…
Re: Local AI needs to be the norm
#789Earlier quoted context omitted.
Because I wanted the full ide on my iPhone so I can code while away from my laptop doing fun stuff with my kids. And I don’t like the Claude codex fire and forget approach. The ide I built has a full terminal, file system, git integration and AI agent. It uses a private cloud Linux container that is persistent so I can install packages and do anything I want from any phone, computer or browser. It’s amazing that we l…
Literally will break overnight when some key dependency changes. Your LLM might not be able to fix it. Then i guess you regenerate it all from scratch? Sounds exhausting tbh.
Also, because I wrote and own the code I don’t have to update if I don’t want to. I could choose instead to build around the dependency. That’s much more control over than when Microsoft bought GitHub and destroyed the Atom ide which I loved in favor of vscode which I still hate
Re: Local AI needs to be the norm
#790Earlier quoted context omitted.
> And stop acting like some dog shit 8B model the average Joe can run on a laptop is even close to being comparable to what Claude or even Codex can currently do. I'm not, you've actually illustrated my point. LLMs in 2022 were very impressive. By 2024 the general public was finding them an acceptable replacement for many research driven tasks and massive shortcuts for other tasks (coding, image work, document preper…
> LLMs in 2022 were very impressive. No they weren't. They were a gimmick - it is only in the past 6 or so months that frontier models have started to do stuff beyond mere gimmicks when it comes to coding, and you could make the argument that Mythos has been the first 'Holy shit' moment that we've had that has stepped us beyond 'Yeah that's really neat but...' > Those models are absolutely runnable on consumer hardwa…
This is simply untrue. Using agentic orchestration I was writing production code daily 3 years ago. Hallucinations happened sometimes and context window was smaller (so you had to do some funky workarounds to deal with larger codebases), but it was workable. There have been a lot of marked improvements from a code perspective then - a lot model related yes, but also a lot in the ease of use, interfaces, etc.
> Another thing I'd say is that you clearly have no clue what 'consumer hardware' means, or what consumers that can even get this stuff running locally would have to do to get it to even rival the frontier models in terms of their usability (most consumers are't going to just boot into Ubuntu and run this thing from a command line) flow, to say nothing of the hardware requirements.
You've moved the goalposts. My point was that the "danger" of no new open models being released isn't that high as the existing ones are already impressive. Their ease of use or daily driving isn't relevant to that. If there were a need, someone could wrap a clean interface and support around it, or run it as their own cloud solution.
You seem to be arguing something adjacent to my point, which is fine I guess but I have little to say. Also multiple of your comments have come across quite aggressive and rude. Just food for thought if you want to work on that or not.