Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Two Qwen3 models on one DGX Spark: the residency math

devashish.me

Post reply on HN
Print view View original

51–57 of 57 posts

Re: Two Qwen3 models on one DGX Spark: the residency math

#51

Post by shireboy »

peddling-brink wrote: post #5
shireboy wrote: post #3

I’ve been considering a move to local llm setup, having been underwhelmed coat vs value of various online offerings. But at the same time worried anything I get will be obsolete in a couple months. And I don’t want to have to babysit it. I really want some agents managing and creating side hustles for me and have some other things. I’m technical-have written my own harness and use gh copilot and grok daily and have a…

Well, if you are making side-hustle money now using online models that, critically, you could also run at home, then it sounds like it’s just a matter of numbers. Oh and, unless you spend a lot more than 5k, your local model will still be slower than the online model. What’s your estimated ROI? Assuming that’s not true based on your phrasing, you’d be shooting yourself in the foot. Start using online models with the…

My thought process is that I don’t mind a slower model if it can work in background for me 24/7 fleshing out side gig ideas I have floating around but no time to focus on myself. I take your point though, and it’s why I haven’t bit the bullet yet. I could buy a lot of tokens for 5k. If I could make that effective then the roi of offline should be something I can calculate fairly easily.

Re: Two Qwen3 models on one DGX Spark: the residency math

#52

Post by edg5000 »

dofm wrote: post #40
edg5000 wrote: post #26

Earlier quoted context omitted.

From the Codex system prompt (verbatim): ``` (...) - Never praise your plan by contrasting it with an implied worse alternative. For example, never use platitudes like \"I will do rather than \", \"I will do , not \". - Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user's query. (...) ``` It seems the OpenA…

FWIW the mere fact that the goblin stuff is necessary and is in the system prompt suggests to me that OpenAI's approach of training ridiculously large models that can do everything for everyone is hopelessly cooked. Though I concede it is not that much different than straightening the tie of your most valuable employee before you unwisely put them in front of a client and saying "please don't tell them about the regr…

Indeed. Maybe in the future labs will be more distinct in what they care about most (and what their model is best at), rather than trying to max out all benchmarks.

This may happen once we see finetuned GLM/Kimi/DeepSeek companies enter the market. I think it's not happening yet because of the hardware supply chain issues.

Re: Two Qwen3 models on one DGX Spark: the residency math

#53

Post by CamperBob2 »

zozbot234 wrote: post #44
CamperBob2 wrote: post #38

Earlier quoted context omitted.

My impression is that with the latest round of high-profile releases, the open-weight "market" is coalescing around two players, DS4 Flash for speed and GLM 5.2 for smarts. Qwen is being left behind to pick up the scraps for the terminally GPU-poor. We know they have what it takes to fight back, and they know it... so I agree, there's no reason not be optimistic about future Qwen releases. But then I've never really…

DeepSeek V4 Pro seems to have significantly lower overhead than GLM 5.2 for the same context size. If the two are about equally smart, that's not a very good look for GLM. E.g. the KV-cache storage for GLM at full context is significantly larger, which directly impacts the effectiveness of batching on memory-constrained hardware. Keep in mind that the existing DeepSeek Pro is a preview model, we might be about to see…

If the two are about equally smart, that's not a very good look for GLM.

They aren't, though. GLM 5.2 is very far out in front of everybody else in the open-weight business when it comes to coding. They seem to have put a disproportionate effort into improving coding, and while it paid off for that, it does seems to have cost some efficiency.

You could say that GLM 5.2 is to DS4 as Fable is to Opus. Fable is is no better at a lot of tasks than Opus, but it codes like nothing else ever built.

Re: Two Qwen3 models on one DGX Spark: the residency math

#54

Post by woadwarrior01 »

verdverm wrote: post #41
woadwarrior01 wrote: post #35

Earlier quoted context omitted.

> unsloth has toxic employees in their discord Would you mind elaborating on this?

Sure, I shared a project in their #research channel where I used their qwen36moe quant to refresh my PhD research. The channel had a topic that ended with something like "and all things research..." One of their people accused me of self-promotion, and I reiterated that I shared it in that channel because it was their quant doing something (I thought) interesting as a research model. The number of people interested i…

Thanks for sharing.

Re: Two Qwen3 models on one DGX Spark: the residency math

#55

Post by devashish86 »

Myrmornis wrote: post #36

The article was clearly written by an LLM. Please say so at the top.

fair point, i'll publish the methodology in the beginning next time onward.

fwiw, this is my high level process.

1. i keep comprehensive notes _while_ i'm experimenting(like checkpoints). this is a mix of commits and a append-only changelog file

2. if some part of what i've done seems like i can share with public, i create an outline of it. small paragraph with a few points

3. then i ask the model to merge the two to generate a post. i have my own style guide but ofcourse, the model idiosyncrasies will always creep in.

Re: Two Qwen3 models on one DGX Spark: the residency math

#56

Post by devashish86 »

reasonableklout wrote: post #50
barrkel wrote: post #10

Earlier quoted context omitted.

Can you try and tune your Claude or whatever LLM you're using for your text to phrase things in plain English. Way less use of antithesis, at least. You can probably find a skill for it, if not get an LLM to write your own.

Yes, there are lots of obvious LLM tells that don't add value, like "the math has to be empirical, not aspirational", use of colorful technical language like "knobs" and "wiring", etc. It distracts from the content.

fair points. will take this into account in the next post.
Post reply on HN