Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using an open model feels surprisingly good

matthewsaltz.com

Post reply on HN
Print view View original

31–40 of 157 posts

Re: Using an open model feels surprisingly good

#31

Post by arjie »

pimeys wrote: post #18
arjie wrote: post #8

To be honest, I am surprised by how good DeepSeek V4 Flash is. I use Claude Code and Codex on Claude 5 Opus and GPT-5.6 Sol most of the time, but when I use DS V4 Flash I don't feel like it's really that bad. And with oh-my-pi and just plain pi it's pretty good. To be honest the frontier models are much better at tool calling so in an assistant flow they're better but I did the dumb thing and optimized the harness fo…

I wrote my own OpenClaw one weekend and I am running it as my assistant through Matrix with DeepSeek v4 Flash (and Qwen). It probably costs me about 2 dollars a month and is even more useful than ChatGPT would be due to me having full control on what tools it has access to. I can do things like take a photo of a doctor's note among add the appointment to my calendar, send a PDF to my archive tagged, OCR'd etc, search…

Haha! Love to hear it. That's exactly what I have too: a claw-like[0] system which I originally used Sonnet for and now use my DeepSeek v4 Flash with.

In my case, I was foolish enough to run it all on my hardware which is pretty damned fast but has a duty cycle of 5% and runs idle most of the time. It's definitely better done via API.

Tell me more about the Alexa-like system! I have mine at home on a custom OpenWakeWord model trained on the word 'Aurora'. And I, too, have it hooked up to Home Assistant. I have a bunch of Eufy E21 baby cameras mounted on the walls[1] so it has vision through the house. The vision model uses GPT-5.5 on the subscription because I haven't yet set up a Qwen or Gemma multimodal that can see.

With Frigate on my home server I can even watch for events like my daughter waking up! And at night the agent sends out the vacuum if we've cleared the floor of baby toys. I feel this close to the dream of sci-fi AI. Because I auto-forward a bunch of my email to it etc. it knows about what's going on with me.

My wife will sometimes ask the agent information about me etc. and it's way easier to get a fast response rather than waiting for me to see the message etc.[2]

0: https://news.ycombinator.com/item?id=47538158

1: https://wiki.roshangeorge.dev/w/Blog/2025-12-01/Grounding_Yo...

2: https://imgur.com/a/tP12lfj

Re: Using an open model feels surprisingly good

#32

Post by basch »

Gigachad wrote: post #29
wps wrote: post #27

Earlier quoted context omitted.

I’m the exact same way. $15 on openrouter lasted me so long it would’ve got me fired at FAANG. Despite this, my number of commits is dramatically higher. Small, beautifully scoped changes is just good software development, and good for the wallet as well. I think the issue is that no one is content with incremental progress. We all know one shots are mostly possible, so the age of the personal project is kind of over…

I can see why Anthropic is freaking out right now. China has undermined their whole business. AI models will be basic commodities where hosting providers earn a tiny margin over the raw costs rather than the predicted fortunes from being the gatekeepers to the technology.

the writing should have been on the wall when meta did it with llama. it was very apparent that one or two entities could release a "good enough" product into the wild commoditizing the frontier of yesterday.

Re: Using an open model feels surprisingly good

#33

Post by irishcoffee »

loire280 wrote: post #7
irishcoffee wrote: post #2

It surprises me that this concept took as long as it did to gain traction in… hacker news. 15 years ago folks here were compiling kernels and gentoo distros. Lately it’s been “you should just pay the man, it’s cheaper than running these things yourself”

Rolling your own Linux is nearly free and could be done on any computer you had lying around. Dropping >$5k on a computer to run a local model (badly) doesn't really scratch my "hacker" itch. The author of this article works for an AI infrastructure company and ran Kimi K3 on their infrastructure - this post reads like marketing.

You can run a small model on almost any hardware, for free, and just play around. It doesn’t need to be some “frontier” (whatever the hell that means) model by any means. Grab some new tech and slap it on whatever hardware you have laying around, see what you think.

Making it about the hardware costs is not the play, they’re not an actual barrier.

Re: Using an open model feels surprisingly good

#34

Post by irishcoffee »

brcmthrowaway wrote: post #3
irishcoffee wrote: post #2

It surprises me that this concept took as long as it did to gain traction in… hacker news. 15 years ago folks here were compiling kernels and gentoo distros. Lately it’s been “you should just pay the man, it’s cheaper than running these things yourself”

Rsync the dropbox, dude

Still working on the fields medal, I’ll give that a shot next.

Re: Using an open model feels surprisingly good

#35

Post by aprilnya »

loufe wrote: post #9

This is only a thinly veiled ad. It's fine, I was curious about this exact setup, anyways. What would be useful is a cost metric. I'm curious how much I'd be willing to spend as a premium to not have those companies piping my conversations directly to the NSA. Maybe only some conversations? Claude and OpenAI are heavily subsidized, by all accounts, so Kimi K3 on a private endpoint might end up costing more or less -…

With OpenRouter this is pretty easy to see - each model page has various providers listed along with an icon marking providers with zero data retention, and the price. You can also set up an account-wide "guardrail" to prevent routing to non-ZDR providers.

Re: Using an open model feels surprisingly good

#37

Post by wxw »

> The best way I can describe it is like opening vim after spinning a bunch of time in a big fancy editor.

Harness-aside, I get this feeling sometimes when I swap from a big frontier model to something more nimble like Composer.

I can get into a better thinking and q&a loop with fast models, similar to how I can flow through a file more easily with vim.

Re: Using an open model feels surprisingly good

#38

Post by adithyassekhar »

spicyusername wrote: post #25

Something I've been thinking about a lot is that this new technology's primary UI is natural language. Human's are REALLY primed for natural language. If it sounds good it must be good. The code this new model writes is incredible! It told me so!

It did say it was an unusually clean codebase.

Re: Using an open model feels surprisingly good

#39

Post by anuramat »

irishcoffee wrote: post #33
loire280 wrote: post #7

Earlier quoted context omitted.

Rolling your own Linux is nearly free and could be done on any computer you had lying around. Dropping >$5k on a computer to run a local model (badly) doesn't really scratch my "hacker" itch. The author of this article works for an AI infrastructure company and ran Kimi K3 on their infrastructure - this post reads like marketing.

You can run a small model on almost any hardware, for free, and just play around. It doesn’t need to be some “frontier” (whatever the hell that means) model by any means. Grab some new tech and slap it on whatever hardware you have laying around, see what you think. Making it about the hardware costs is not the play, they’re not an actual barrier.

> you can run a small model

but why?

Re: Using an open model feels surprisingly good

#40

Post by cootsnuck »

Gigachad wrote: post #29
wps wrote: post #27

Earlier quoted context omitted.

I’m the exact same way. $15 on openrouter lasted me so long it would’ve got me fired at FAANG. Despite this, my number of commits is dramatically higher. Small, beautifully scoped changes is just good software development, and good for the wallet as well. I think the issue is that no one is content with incremental progress. We all know one shots are mostly possible, so the age of the personal project is kind of over…

I can see why Anthropic is freaking out right now. China has undermined their whole business. AI models will be basic commodities where hosting providers earn a tiny margin over the raw costs rather than the predicted fortunes from being the gatekeepers to the technology.

[dead]
Post reply on HN