91–100 of 156 posts
Is there an app that uses OpenRouter / Claude or something locally but has MCP support?
Just ordered a $12k mac studio w/ 512GB of integrated RAM. Can't wait for it to arrive and crank up LM Studio. It's literally the first install. I'm going to download it with safari. LM Studio is newish, and it's not a perfect interface yet, but it's fantastic at what it does which is bring local LLMs to the masses w/o them having to know much. There is another project that people should be aware of: https://github.c…
Earlier quoted context omitted.
Near as I can tell it's supposed to make calling other people's tools easier. But I don't want to spin up an entire server to invoke a calculator. So far it seems to make building my own local tools harder, unless there's some guidebook I'm missing.
Your not spinning up a whole server lol, most MCP's can be run locally, and talked to over stdio, like their just apps that the LLM can call, what they talk to or do is up to the MCP writer, its easier to have a MCP that communicates what it can do and handles the back and forth, than writing a non-standard middleware to handle say calls to an API or handle using applescript, or vmware or something else...
Earlier quoted context omitted.
Near as I can tell it's supposed to make calling other people's tools easier. But I don't want to spin up an entire server to invoke a calculator. So far it seems to make building my own local tools harder, unless there's some guidebook I'm missing.
It's a protocol that doesn't dictate how you are calling the tool. You can use in-memory transport without needing to spin up a server. Your tool can just be a function, but with the flexibility of serving to other clients.
Just ordered a $12k mac studio w/ 512GB of integrated RAM. Can't wait for it to arrive and crank up LM Studio. It's literally the first install. I'm going to download it with safari. LM Studio is newish, and it's not a perfect interface yet, but it's fantastic at what it does which is bring local LLMs to the masses w/o them having to know much. There is another project that people should be aware of: https://github.c…
I'd love to host my own LLMs but I keep getting held back from the quality and affordability of Cloud LLMs. Why go local unless there's private data involved?
Earlier quoted context omitted.
I'm using it on MacBook Air M1 / 8 GB RAM with Qwen3-4B to generate summaries and tags for my vibe-coded Bloomberg Terminal-style RSS reader :-) It works fine (the laptop gets hot and slow, but fine). Probably should just use llama.cpp server/ollama and not waste a gig of memory on Electron, but I like GUIs.
8 GB of RAM with local LLMs in general is iffy: a 8-bit quantized Qwen3-4B is 4.2GB on disk and likely more in memory. 16 GB is usually the minimum to be able to run decent models without compromising on heavy quantization.
macOS virtual memory works well on swapping in and out stuff to SSD.
Earlier quoted context omitted.
Why would they pay 2/3 of the price for something with 1/5 of ram? The whole point of spending that much money for them is to run massive models, like the full R1, which the Pro 6000 cant
Because waiting forever for initial prompt processing with realistic number of MCP tools enabled on a prompt is going to suck without the most bandwidth possible And you are never going to sit around waiting for anything larger than the 96+gb of ram that the RTX pro has. If you’re using it for background tasks and not coding it’s a different story
Earlier quoted context omitted.
I was using Claude 3.7 exclusively for coding, but it sure seems like it got worse suddenly about 2–3 weeks back. It went from writing pretty solid code I had to make only minor changes to, to being completely off its rails, altering files unrelated to my prompt, undoing fixes from the same conversation, reinventing db access and ignoring existing coding 'standards' established in the existing codebase. Became so unt…
Could be the prompt and/or tool descriptions in whatever tool you are using Claude in that degraded. Have definitely noticed variance across Cursor, Claude Code, etc even with the exact same models. Prompts + tools matter.
I have one running locally with this config:
{
"mcpServers": {
"coderunner": {
"url": "http://coderunner.local:8222/sse"
}
}
}
Earlier quoted context omitted.
Ollama doesn’t even have a way to customize the context size per model and persist it. LM studio does :)
This isn't true. You can `ollama run {model}`, `/set parameter num_ctx {ctx}` and then `/save`. Recommended to `/save {model}:{ctx}` to persist on model update