Live data from Hacker News

Ask HN: What default model do you use and why?

news.ycombinator.com

101–110 of 115 posts

Re: Ask HN: What default model do you use and why?

#101
post #75
post #69

Earlier quoted context omitted.

> on a GB10 cluster I'd love to hear more about your setup. I have a single GB10 and am thinking about adding an additional one.

I'm coming up on a month with my 2x GB10 cluster. I was at 1x for a week before I pulled the trigger for the second. Initial setup was a tad annoying because I had to update their firmwares and then power cycle them to get the 200GbE link working at full speed. After setting that up, it has been pretty smooth. I don't directly deal with the cluster, usually I just have the LLM itself handle updates/stopping to load d…

How much does it cost per month, which provider and what do you get out of it?

Re: Ask HN: What default model do you use and why?

#102
I subscribed to ChatGPT for a few years, but over the holidays noticed that it was making frequent errors and hallucinations when dealing with more obscure Linux and Firefox issues. Trying Gemini as a backup produced much better answers, to the point I switched my subscription. I do everything through the app/website rather than the API; I understand this is pricier but I like having convenient access, a searchable history, memory, etc.

I try the ChatGPT free tier as a backup second opinion sometimes, but find their newest model to be painfully rambling.

I want to like Claude and might consider subscribing to it instead, but am put off by reports of its tight usage limits. With Gemini, though, I never hit a limit unless I conduct a few Deep Research queries at the highest level.

Re: Ask HN: What default model do you use and why?

#106
I use a lot of different harnesses, so in CC i use sonnet as main w/ opus advisor, in codex i use astra all the time, in obvious I usually use the GLM models. In general I've found that having a fast main model with a big smart advisor is a nice UX if I'm going to be on keyboard interacting with it, for a long-running agent that I'm not attending to I care much less about the main model speed and more about cost/quality.

Re: Ask HN: What default model do you use and why?

#108
post #75

Earlier quoted context omitted.

I'm coming up on a month with my 2x GB10 cluster. I was at 1x for a week before I pulled the trigger for the second. Initial setup was a tad annoying because I had to update their firmwares and then power cycle them to get the 200GbE link working at full speed. After setting that up, it has been pretty smooth. I don't directly deal with the cluster, usually I just have the LLM itself handle updates/stopping to load d…

How much does it cost per month, which provider and what do you get out of it?

Hmm? I'm running the models locally... 2x Sparks consume ~150W at peak, and they usually spend more time waiting on results from whatever task they're working on, so I imagine that the contribution to my electricity bill is maybe a dollar/mo or less. Though, of course, each Spark was $4000, so the total I've spent is equivalent to several years of the maximum tier for most cloud model susbcriptions.

What I get out of it is the ability to hand login credentials to my other computers to manage their updates, bug fixes etc. Eg. After updating my proxmox server, the nvme drive kept dying. Was able to let my local AI in to figure out and fix what was wrong (known issue). A cloud-based AI could've done it too, but I don't want to be sending internal passwords out of my network like that.

Plus, the ability to freely delegate tasks or exploration of things cloud models generally avoid. For example, I draw as a hobby, and when I'm struggling with a pose but can't quite figure out what I'm missing, I pass it into a VLM for advice, but Claude etc get unnecessarily cautious because they interpret an anatomical sketch as a naked person.

Post reply on HN