71–80 of 156 posts
Earlier quoted context omitted.
8 GB of RAM with local LLMs in general is iffy: a 8-bit quantized Qwen3-4B is 4.2GB on disk and likely more in memory. 16 GB is usually the minimum to be able to run decent models without compromising on heavy quantization.
But 8GB of Apple RAM is 16GB of normal RAM. https://www.pcgamer.com/apple-vp-says-8gb-ram-on-a-macbook-p...
Earlier quoted context omitted.
8 GB of RAM with local LLMs in general is iffy: a 8-bit quantized Qwen3-4B is 4.2GB on disk and likely more in memory. 16 GB is usually the minimum to be able to run decent models without compromising on heavy quantization.
But 8GB of Apple RAM is 16GB of normal RAM. https://www.pcgamer.com/apple-vp-says-8gb-ram-on-a-macbook-p...
Any suggestions?
I’ve been wanting to try LM Studio but I can’t figure out how to use it over local network. My desktop in the living room has the beefy GPU, but I want to use LM Studio from my laptop in bed. Any suggestions?
Earlier quoted context omitted.
Offline is another use case.
Nothing like playing around with LLMs on an airplane without an internet connection.
I’ve been wanting to try LM Studio but I can’t figure out how to use it over local network. My desktop in the living room has the beefy GPU, but I want to use LM Studio from my laptop in bed. Any suggestions?
[>_] -> [.* Settings] -> Serve on local network ( o)
The initial experience with LMStudio and MCP doesn't seem to be great, I think their docs could do with a happy path demo for newcomers. Upon installing the first model offered is google/gemma-3-12b - which in fairness is pretty decent compared to others. It's not obvious how to show the right sidebar they're talking about, it's the flask icon which turns into a collapse icon when you click it. I set the MCP up with…
Is there an app that uses OpenRouter / Claude or something locally but has MCP support?
Earlier quoted context omitted.
It's "Tools as a service", so it's really trying to make tool calling easier to use.
Near as I can tell it's supposed to make calling other people's tools easier. But I don't want to spin up an entire server to invoke a calculator. So far it seems to make building my own local tools harder, unless there's some guidebook I'm missing.
[flagged]
That's clearly your own product (it links to Koroworld in the footer and you've posted about that on Hacker News in the past). Are you sharing any of your revenue from that $79 license fee with the https://ollama.com/ project that your app builds on top of?