191–200 of 682 posts
Still needs 32-64GB memory to run it locally. 64GB Macbook pro with an M5 chip costs more than 4k Euros in Germany. A more practical model would be a language specific (e.g Python or JVM language) and excellent at tool calling and reasoning. Maybe that way they can shrink it even more.
I lament the comments saying this in any way redeems Meta (the company). The researchers releasing this stuff have almost nothing to do with Meta other than being bankrolled by the slaughterhouse. You aren't the customer, you are the pawn in big tech's game of thrones. Your good will is a commodity to be traded, almost literally. It will be used against you the moment it's convenient. This is open weights because Met…
It’s rather amusing to me to read comments like this, and then simultaneously whenever a Chinese company or team releases open-weight models or whatever there is a giant round of applause, America is so behind, and there’s nothing but positive things to say about the intelligent, creative, and well-intentioned Chinese engineers (which is true, America certainly doesn’t have a monopoly on great people). Don’t you know…
Holding both those positions would be hypocritical all right, but are you sure it's the same people commenting/voting in both cases? I don't think there's a strong consensus on Hacker News. Even something like the time of day an article is posted might get different engagement depending on who is active in which time zones.
Earlier quoted context omitted.
You can partially tell by the tokeniser; which gives you some hint into the training corpus mix. is four Gemma4 tokens, but one Qwen3.6 token.
Where do you find this information for each model?
Earlier quoted context omitted.
Even if you had a 64GB machine: Are you willing to reserve 90% of your memory to run a LLM? With dirt cheap models like deepseek-v4-flash that will run "forever" on $10, the answer for me is clearly: no.
I'm waiting for the speed/quality per dollar metric to go down a little bit further and then I will def run it at home. Its not just that you send a sentence to an API endpoint, you always send EVERYTHING to that agent as a context. You want to analyse your spending history? You now send everything to someone. Either no one cares but understands this implication on how easy it is to really capture you or no one reall…
Some interesting findings from the chat template designs: 1. The template name is Onyx ATEM as found in the tool call exception message 2. It appears to be following a harmony-style chat template. But the tool use seems to be a xml like : / / 3. atem: a internal joke of meta in reverse? https://huggingface.co/meta-models/Muse-Glimmer-30B/blob/mai...
Earlier quoted context omitted.
> FB at one point had a very badass engineering culture Perpetually kneecapped by one of the worst management cultures I've ever seen
Would you say those badass engineers were/are managed by Careless People?
Will be interesting to see how Qwen3.8 27B compares against this once it releases this week. Seems like dense 30B is back in fashion? EDIT: An open weight version of Muse Spark 1.2 is going to be released as well: https://x.com/alexandr_wang/status/2086756152034066792 https://xcancel.com/alexandr_wang/status/2086756152034066792
It’s really interesting timing, Qwen over thinking is what kills it for me. I’m just glad we have more options in this size class now.
Earlier quoted context omitted.
This model I think will be too slow for that on Spark, even at 4 bit quant. It's a dense model, not MoE like e.g. Qwen 35b or Gemma 4 26B A4B. On a Spark it will be memory bandwidth limited I haven't tried yet (working on it) but back of the napkin estimate puts it at around 15tok/s even after converting to NVFP4. Prefill would be much higher though. That 15tok/sec is pretty typical for dense models of this size: NVF…
Native dflash support on day 1 helps a lot! High quality speculative decoding speeds up a lot of agentic work.
That's pretty respectable.
Still working on optimizing and cleaning up before I push it.
Earlier quoted context omitted.
"With dirt cheap models like deepseek-v4-flash that will run "forever" on $10, the answer for me is clearly: no." When it's free, you are the product.
Ah yes, I'm sure Trovalds and Stallman are harvesting my data through free software, aren't they? This argument is used by boomers who were fed cold war era propoganda that surely everybody is selfish, and you're always at fault.
With the business model for API based LLMs looking iffy at best it seems like we’re heading back to the “server under your desk” era of IT again.
Considering how all the big players are playing fast [1] and loose [2] with limits, billing [3] and adding undisclosed changes that burn your tokens on autopilot [4], it can't happen soon enough. [1]: Limits may change without notice, including due to capacity constraints. - https://support.google.com/gemini/answer/16275805?sjid=14713... . [2]: "standard limits" are never defined - https://support.google.com/gemini/a…