Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using an open model feels surprisingly good

matthewsaltz.com

Post reply on HN
Print view View original

141–150 of 157 posts

Re: Using an open model feels surprisingly good

#141

Post by pimeys »

mayank wrote: post #22
pimeys wrote: post #18

Earlier quoted context omitted.

I wrote my own OpenClaw one weekend and I am running it as my assistant through Matrix with DeepSeek v4 Flash (and Qwen). It probably costs me about 2 dollars a month and is even more useful than ChatGPT would be due to me having full control on what tools it has access to. I can do things like take a photo of a doctor's note among add the appointment to my calendar, send a PDF to my archive tagged, OCR'd etc, search…

Oddly enough, I did the exact same thing this past weekend, for a couple hundred usd in Fable overage. It was truly remarkably easy to build and package it exactly to my whims, in my case as a single docker container with a process reaper that runs llama, my Go code, tts, chat harness, browser in xvfb, and even a mailer daemon. With Gemma, it even runs on a RPi 5. What’s absolutely wild to me is that over a couple ho…

Funny. I use Fable a lot at work, but this one I paid from my own pocket and coded it with GLM 5.2.

Paid 20 euros in total.

Re: Using an open model feels surprisingly good

#142

Post by arjie »

pimeys wrote: post #140
arjie wrote: post #31

Earlier quoted context omitted.

Haha! Love to hear it. That's exactly what I have too: a claw-like[0] system which I originally used Sonnet for and now use my DeepSeek v4 Flash with. In my case, I was foolish enough to run it all on my hardware which is pretty damned fast but has a duty cycle of 5% and runs idle most of the time. It's definitely better done via API. Tell me more about the Alexa-like system! I have mine at home on a custom OpenWakeW…

I have these around the house: https://www.home-assistant.io/voice-pe/ Then you can set the background AI to be any OpenAI compatible API. So I just created one to my local Rust Agent, and connected it to home assistant. Now I can yell from the couch to create me a new Proxmox container with the next free static IP address etc. :D

This is very cool. I think I’ll get it instead of using my home built device (a respeaker attached to a low power machine). Looks like it supports custom wake words too. Thank you!

It’s funny how we converged on a similar system. My agent is also written in Rust.

Re: Using an open model feels surprisingly good

#143

Post by wuschel »

drnick1 wrote: post #30
wps wrote: post #19

The reason Claude code is so popular is because it’s really good at taking super vague human prose “Claude build me a million dollar SaaS”-type prompts and spitting out thousands of lines of code which cover tons of surface-level edge cases, build in tons of functionality, etc The smaller/open models are less good at that. But that’s not how software development is done. You don’t prompt a whole app and be done with…

Frontier models like Opus 5 are also extremely good a "research" in an academic sense, including combining elements from different fields or literatures and creating something quite novel sometimes, publishable even, from vague/speculative prompts. This is on top of implementing known methods in about 1/100th of the time it would take by hand, allowing for fast exploration of ideas. They can also roll things out from…

Interesting and very curious what you manage to verify and produce with it.

What kind of fields are we talking about here? Compsci? EEng? BioChem?

Re: Using an open model feels surprisingly good

#144

Post by Ancapistani »

Gigachad wrote: post #21
nujabe wrote: post #10

This is such a poor quality post, reads more like a diary entry than a substantive technical post.

I will take 5000 "diary entry" blog posts over LLM generated walls of text that say nothing. People need to learn to just post the prompt rather than the LLM output which just fluffs the prompt.

I've not blogged since the rise of AI started in earnest, but I intend to do just that: the prompt I used in a collapsed callout at the top, and the actual post body below. Yes, I'll do a ton of editing, but the prompt itself is close enough to a summary to be useful, and if someone is in a hurry it'll be the fastest way to get the gist.

Re: Using an open model feels surprisingly good

#145

Post by slopinthebag »

techpression wrote: post #71
hangonhn wrote: post #61

Earlier quoted context omitted.

Yeah I feel like China and their OSS model companies really did the world a great service by ensuring closed source US companies don't have an monopoly on LLMs and thus capture all the value of AI development and its impact on society. It's not going to be the foundational labs that will capture all the value -- I'm sure they will do just fine. A lot of it now will flow to the companies providing the inferences and s…

It’s not a service, I hope nobody thinks this is being done ”for the good of humanity” or whatever. Doesn’t mean there’s not a benefit to people, but the financial politics behind it could turn out extremely good for China, and they are very well aware of just that.

You gotta remember there are people working at these companies with their own motivations, beliefs, and values. And to the extent that their government allows them to, they will act on those.

I think the people in the Chinese labs have a different set of motivations, beliefs, and values that make them more amenable to sharing AI as a common good rather than some economical advantage. I've heard that workers in the Chinese labs share far more knowledge between themselves, and the narrative around AI in China is different from what we see in the USA. So it is possible, I believe, that the labs are to some extent releasing open models "for the good of humanity", or at least they believe it is the right thing to do.

Re: Using an open model feels surprisingly good

#146

Post by drnick1 »

wuschel wrote: post #143
drnick1 wrote: post #30

Earlier quoted context omitted.

Frontier models like Opus 5 are also extremely good a "research" in an academic sense, including combining elements from different fields or literatures and creating something quite novel sometimes, publishable even, from vague/speculative prompts. This is on top of implementing known methods in about 1/100th of the time it would take by hand, allowing for fast exploration of ideas. They can also roll things out from…

Interesting and very curious what you manage to verify and produce with it. What kind of fields are we talking about here? Compsci? EEng? BioChem?

Scientific computing/statistics.

Re: Using an open model feels surprisingly good

#147

Post by 4d4m »

Gigachad wrote: post #29
wps wrote: post #27

Earlier quoted context omitted.

I’m the exact same way. $15 on openrouter lasted me so long it would’ve got me fired at FAANG. Despite this, my number of commits is dramatically higher. Small, beautifully scoped changes is just good software development, and good for the wallet as well. I think the issue is that no one is content with incremental progress. We all know one shots are mostly possible, so the age of the personal project is kind of over…

I can see why Anthropic is freaking out right now. China has undermined their whole business. AI models will be basic commodities where hosting providers earn a tiny margin over the raw costs rather than the predicted fortunes from being the gatekeepers to the technology.

+1, and commodification was inevitable from the outset. This was the case before China. China hate is xenophobic and unproductive: produced both model improvements and desirable products people want at this moment in time. You use the best/cheapest model for the job/user input.

Parallels to the US auto industry not paying attention to its buyers and the right strategies when they lost technical dominance to other countries a few decades back.

Re: Using an open model feels surprisingly good

#149

Post by pimeys »

arjie wrote: post #142
pimeys wrote: post #140

Earlier quoted context omitted.

I have these around the house: https://www.home-assistant.io/voice-pe/ Then you can set the background AI to be any OpenAI compatible API. So I just created one to my local Rust Agent, and connected it to home assistant. Now I can yell from the couch to create me a new Proxmox container with the next free static IP address etc. :D

This is very cool. I think I’ll get it instead of using my home built device (a respeaker attached to a low power machine). Looks like it supports custom wake words too. Thank you! It’s funny how we converged on a similar system. My agent is also written in Rust.

Yes. I tested OpenClaw that burned 15€ just by starting it and I hated its configuration. Spent 20€ in tokens and built my own in a day, that does everything I want and sips tokens.

What a time to be alive.

Re: Using an open model feels surprisingly good

#150

Post by suprjami »

lordnacho wrote: post #98
wps wrote: post #19

The reason Claude code is so popular is because it’s really good at taking super vague human prose “Claude build me a million dollar SaaS”-type prompts and spitting out thousands of lines of code which cover tons of surface-level edge cases, build in tons of functionality, etc The smaller/open models are less good at that. But that’s not how software development is done. You don’t prompt a whole app and be done with…

Are local models more snappy? I'm at a stage where I can work with the output of LLMs, and the next win is really just getting things written out quickly.

Depends on your setup. If you drop $10k on an RTX Pro 6000 then yeah Qwen 35B MoE will absolutely fly.

If you have a pair of 3090s and run Qwen 27B, or an old Threadripper with heaps of system RAM and Deepseek or MiniMax or Kimi, no it won't be as fast as Claude.

Most local LLM nerds are not running locally for superior speed, we're doing it for sovereignty and/or privacy, or maybe just because it's fun which accidentally became useful this year.

Post reply on HN