Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MCP in LM Studio

lmstudio.ai

Post reply on HN
Print view View original

121–130 of 156 posts

Re: MCP in LM Studio

#121

Post by nix0n »

boredemployee wrote: post #68
nix0n wrote: post #8

Earlier quoted context omitted.

LM Studio is quite good on Windows with Nvidia RTX also.

care to elaborate? i have rtx 4070 12gb vram + 64gb ram, i wonder what models I can run with it. Anything useful?

LM Studio's model search is pretty good at showing what models will fit in your VRAM.

For my 16gb of VRAM, those models do not include anything that's good at coding, even when I provide the API documents via PDF upload (another thing that LM Studio makes easy).

So, not really, but LM Studio at least makes it easier to find that out.

Re: MCP in LM Studio

#122

Post by MangoToupe »

zackify wrote: post #70
tymscar wrote: post #56

Earlier quoted context omitted.

Why would they pay 2/3 of the price for something with 1/5 of ram? The whole point of spending that much money for them is to run massive models, like the full R1, which the Pro 6000 cant

Because waiting forever for initial prompt processing with realistic number of MCP tools enabled on a prompt is going to suck without the most bandwidth possible And you are never going to sit around waiting for anything larger than the 96+gb of ram that the RTX pro has. If you’re using it for background tasks and not coding it’s a different story

> And you are never going to sit around waiting for anything larger than the 96+gb of ram that the RTX pro has.

Am I the only person that gives aider instructions and leaves it alone for a few hours? This doesn't seem that difficult to integrate into my workflow.

Re: MCP in LM Studio

#123

Post by MangoToupe »

asteroidburger wrote: post #75
seanmcdirmid wrote: post #69

Earlier quoted context omitted.

Nothing like playing around with LLMs on an airplane without an internet connection.

If I can afford a seat above economy with room to actually, comfortably work on a laptop, I can afford the couple bucks for wifi for the flight.

Woah there Mr Money, slow down with these assumptions. A computer is worth the investment. But paying a cent extra to airlines? Unacceptable.

Re: MCP in LM Studio

#125

Post by diggan »

MangoToupe wrote: post #122
zackify wrote: post #70

Earlier quoted context omitted.

Because waiting forever for initial prompt processing with realistic number of MCP tools enabled on a prompt is going to suck without the most bandwidth possible And you are never going to sit around waiting for anything larger than the 96+gb of ram that the RTX pro has. If you’re using it for background tasks and not coding it’s a different story

> And you are never going to sit around waiting for anything larger than the 96+gb of ram that the RTX pro has. Am I the only person that gives aider instructions and leaves it alone for a few hours? This doesn't seem that difficult to integrate into my workflow.

> Am I the only person that gives aider instructions and leaves it alone for a few hours?

Probably not, but in my experience, if it takes longer than 10-15 minutes it's either stuck in a loop or down the wrong rabbit hole. But I don't use it for vibe coding or anything "big scope" like that, but more focused changes/refactors so YMMV

Re: MCP in LM Studio

#126

Post by diggan »

smcleod wrote: post #117
zackify wrote: post #36

Earlier quoted context omitted.

I love LM studio but I’d never waste 12k like that. The memory bandwidth is too low trust me. Get the RTX Pro 6000 for 8.5k with double the bandwidth. It will be way better

RTX is nice, but it's memory limited and requires to have a full desktop machine to run it in. I'd take slower inference (as long as it's not less than 15tk/s) for more memory any day!

I'd love to see more Very-Large-Memory Mac Studio benchmarks for prompt processing and inference. The few benchmarks I've seem either missed to take prompt processing into account, didn't share exact weights+setup that were used or showed really abysmal performance.

Re: MCP in LM Studio

#127

Post by diggan »

imranq wrote: post #31
chisleu wrote: post #2

Just ordered a $12k mac studio w/ 512GB of integrated RAM. Can't wait for it to arrive and crank up LM Studio. It's literally the first install. I'm going to download it with safari. LM Studio is newish, and it's not a perfect interface yet, but it's fantastic at what it does which is bring local LLMs to the masses w/o them having to know much. There is another project that people should be aware of: https://github.c…

I'd love to host my own LLMs but I keep getting held back from the quality and affordability of Cloud LLMs. Why go local unless there's private data involved?

There are some use cases I use LLMs for where I don't care a lot about the data being private (although that's a plus) but I don't want to pay XXX€ for classifying some data and I particularly don't want to worry about having to pay that again if I want to redo it with some changes.

Using local LLMs for this I don't worry about the price at all, I can leave it doing three tries per "task" without tripling the cost if I wanted to.

It's true that there is an upfront cost but way easier to get over that hump than on-demand/per-token costs, at least for me.

Re: MCP in LM Studio

#129

Post by diggan »

seanmcdirmid wrote: post #69
mycall wrote: post #64

Earlier quoted context omitted.

Offline is another use case.

Nothing like playing around with LLMs on an airplane without an internet connection.

Some of us don't have the most reliable ISPs or even network infrastructure, and I say that as someone who lives in Spain :) I live outside a huge metropolitan area and Vodafone fiber went down twice this year, not even counting the time the country's electricity grid was down for like 24 hours.

Re: MCP in LM Studio

#130

Post by diggan »

prophesi wrote: post #120
phren0logy wrote: post #115

Earlier quoted context omitted.

I have not heard this rumor. Source?

I believe they're talking about the rumors by an Apple supply chain analyst, Ming-Chi Kuo. https://www.techspot.com/news/106159-apple-m5-silicon-rumore...

Seems Apple is waking up to the fact that if it's too easy to run weights locally, there really isn't much sense to having their own remote inference endpoints, so time to stop the party :)
Post reply on HN