61–70 of 156 posts
What models are you using on LM Studio for what task and with how much memory? I have a 48GB macbook pro and Gemma3 (one of the abliterated ones) fits my non-code use case perfectly (generating crime stories which the reader tries to guess the killer). For code, I still call Google to use Gemini.
Earlier quoted context omitted.
There's a "headless" checkbox in settings->developer
Still, you need to install and run the AppImage at least once to enable the "lms" cli which can later be used. Would be nice with a completely GUI-less installation/use method too.
Just ordered a $12k mac studio w/ 512GB of integrated RAM. Can't wait for it to arrive and crank up LM Studio. It's literally the first install. I'm going to download it with safari. LM Studio is newish, and it's not a perfect interface yet, but it's fantastic at what it does which is bring local LLMs to the masses w/o them having to know much. There is another project that people should be aware of: https://github.c…
I love LM studio but I’d never waste 12k like that. The memory bandwidth is too low trust me. Get the RTX Pro 6000 for 8.5k with double the bandwidth. It will be way better
If the primary use case is input heavy, which is true of agentic tools, there’s a world where partial GPU offload with many channels of DDR5 system RAM leads to an overall better experience. A good GPU will process input many times faster, and with good RAM you might end up with decent output speed still. Seems like that would come in close to $12k?
And there would be no competition for models that do fit entirely inside that VRAM, for example Qwen3 32B.
Just ordered a $12k mac studio w/ 512GB of integrated RAM. Can't wait for it to arrive and crank up LM Studio. It's literally the first install. I'm going to download it with safari. LM Studio is newish, and it's not a perfect interface yet, but it's fantastic at what it does which is bring local LLMs to the masses w/o them having to know much. There is another project that people should be aware of: https://github.c…
I'd love to host my own LLMs but I keep getting held back from the quality and affordability of Cloud LLMs. Why go local unless there's private data involved?
Earlier quoted context omitted.
Yup, I'm spoiled by Claude 3.7 Sonnet right now. I had to stop using opus for plan mode in my Agent because it is just so expensive. I'm using Gemini 2.5 pro for that now. I'm considering ordering one of these today: https://www.newegg.com/p/N82E16816139451?Item=N82E1681613945... It looks like it will hold 5 GPUs with a single slot open for infiniband Then local models might be lower quality, but it won't be slow! :)
I was using Claude 3.7 exclusively for coding, but it sure seems like it got worse suddenly about 2–3 weeks back. It went from writing pretty solid code I had to make only minor changes to, to being completely off its rails, altering files unrelated to my prompt, undoing fixes from the same conversation, reinventing db access and ignoring existing coding 'standards' established in the existing codebase. Became so unt…
Prompts + tools matter.
Is there an app that uses OpenRouter / Claude or something locally but has MCP support?
LM Studio has quickly become the best way to run local LLMs on an Apple Silicon Mac: no offense to vllm/ollama and other terminal-based approaches, but LLMs have many levers for tweaking output and sometimes you need a UI to manage it. Now that LM Studio supports MLX models, it's one of the most efficient too. I'm not bullish on MCP, but at the least this approach gives a good way to experiment with it for free.
LM Studio is quite good on Windows with Nvidia RTX also.
Earlier quoted context omitted.
I'd love to host my own LLMs but I keep getting held back from the quality and affordability of Cloud LLMs. Why go local unless there's private data involved?
Offline is another use case.
Earlier quoted context omitted.
I love LM studio but I’d never waste 12k like that. The memory bandwidth is too low trust me. Get the RTX Pro 6000 for 8.5k with double the bandwidth. It will be way better
Why would they pay 2/3 of the price for something with 1/5 of ram? The whole point of spending that much money for them is to run massive models, like the full R1, which the Pro 6000 cant
And you are never going to sit around waiting for anything larger than the 96+gb of ram that the RTX pro has.
If you’re using it for background tasks and not coding it’s a different story