481–490 of 682 posts
For the same cost, you could run on a frontier model on a pro plan for two years. The economics dont make a lot of sense for this to me, so I would love some input on why people want to do this instead (privacy, for fun, etc).
The post suggests that you need an rtx 5090 use it, which is currently selling for around $5,000 USD. I wouldn't exactly call that "my device", since my device costs about 25% of that for the entire computer. For the same cost, you could run on a frontier model on a pro plan for two years. The economics dont make a lot of sense for this to me, so I would love some input on why people want to do this instead (privacy,…
And in terms of interesting use cases: recently pointed an agent at Blender and gave it vision. That setup can essentially iterate on a scene forever.
The post suggests that you need an rtx 5090 use it, which is currently selling for around $5,000 USD. I wouldn't exactly call that "my device", since my device costs about 25% of that for the entire computer. For the same cost, you could run on a frontier model on a pro plan for two years. The economics dont make a lot of sense for this to me, so I would love some input on why people want to do this instead (privacy,…
Earlier quoted context omitted.
Based on the benchmarks, it seems that Muse Glimmer barely edges out against Qwen3.6 27B, except for tool-calling skills (MCP, etc.). I wouldn't be surprised if they released it now because they are afraid they wouldn't beat Qwen3.8 27B.
Do AI companies make release plans based on upcoming other models like this? I would think all the processes that go into the repository and weight infrastructure pre-training, checkpointing, knowledge distillation, model compression, post training pipeline, ecosystem integrations, inference API, benchmarking, human eval/safety/alignment, docs, etc... all that dictates the release schedule.
AI companies release models when they are ready - not on a set schedule. The steps are required to produce a release candidate, so a company can choose to bless the RC with the best metrics at any time - or alternatively continue (post/)training newer RCs if they feel they can do better, and they have time. When a model is ready for release is subjective, and can take into consideration what the competition is doing.
An obvious counter-example to schedules driving releases is the still unreleased Gemini 3.6 Pro.
Remember when we needed 200 servers for an enterprise website because Apache used one process or thread per connection - and Nginx collapsed that into a single box overnight? That moment for LLMs is near. It’s going to move us from the big iron era of AI to small portable brains. Nature has already proved it’s possible with 20 watts and very little heat generation. And I think the data center buildout will end in car…
Everyone keeps repeating this who doesn’t understand the underlying technology. Small llms are still way more efficiently server on big GPUs. Sharing server capacity takes advantage of the massive parallel throughput and sharing of memory bandwidth. You are sharing the GPUs with thousands of concurrent users.
Many tasks suited for AI assistants are background asynchronous tasks. They can run in the downtime where immediate demand is low, keeping overall utilization high enough.
Your argument is similar to those who argue that owning a GPU for gaming doesn't make sense when you can stream from something like GeForce Now. However like with gaming locally (improved latency) there are also benefits to local AI (data privacy).
Earlier quoted context omitted.
the “Actually… But wait!” style responses are so annoying, even Claude opus struggles with this so I’d be interested if meta has done something to cut down on that while still giving good responses
It's not perfect but it is very terse! Better than BottleCap managed to do with post-training Qwen in ThinkingCap. I suspect it will help a lot with enabling preserve-reasoning, because the biggest apparent limitation of this model is the 128K context window. Though the practical issue I am seeing on my M1 Max MBP is that performance suddenly drops off a cliff if I have DFlash enabled.
Earlier quoted context omitted.
Everyone keeps repeating this who doesn’t understand the underlying technology. Small llms are still way more efficiently server on big GPUs. Sharing server capacity takes advantage of the massive parallel throughput and sharing of memory bandwidth. You are sharing the GPUs with thousands of concurrent users.
I’m sure you could find this exact same comment regarding technology in 1999.
Anybody here succeed to run this on AMD?
Will be interesting to see how Qwen3.8 27B compares against this once it releases this week. Seems like dense 30B is back in fashion? EDIT: An open weight version of Muse Spark 1.2 is going to be released as well: https://x.com/alexandr_wang/status/2086756152034066792 https://xcancel.com/alexandr_wang/status/2086756152034066792
Considering that Meta distills Qwen[1] (and should!), it'd be hilarious if Muse loses the head-to-head; the "distillation attack!!1!" people claimed distillation on release n-1 is enough to match the intelligence of the latest version.
1. They wrote a paper about it
Earlier quoted context omitted.
Everyone keeps repeating this who doesn’t understand the underlying technology. Small llms are still way more efficiently server on big GPUs. Sharing server capacity takes advantage of the massive parallel throughput and sharing of memory bandwidth. You are sharing the GPUs with thousands of concurrent users.
It may not make financial sense for someone retired, not into tech, and/or data privacy to host their own LLMs. However if usage of AI in day to day lives continues to increase, I think it will eventually make sense for the majority. Many tasks suited for AI assistants are background asynchronous tasks. They can run in the downtime where immediate demand is low, keeping overall utilization high enough. Your argument…
We will appreciate it even more when popular agents become more costly or start including more ads.