211–220 of 226 posts
Earlier quoted context omitted.
Try 27b, it's significantly smarter than 35b-a3b (although it is slower, it's not so bad with MTP).
It is, but it's way too slow on a Strix Halo due to its limited bandwidth. (I'm still sad that they didn't make a 122B-A10B version of it, as it's the kind of model that fits best on a Strix Halo, and for 3.5 it was comparable in performance to the dense 27B version).
Earlier quoted context omitted.
Given this and the quality of open models, it makes no sense to me that there’s a future for Anthropic et all?
Packaging a capability into a consumable form will still be business. It's like web hosting; all the open source tools are there and free, and yet website tools, hosts, etc flourish.
And I don't see a lot of web hosts with twelve-digit valuations.
What is the idiots guude to run this one local now?
Earlier quoted context omitted.
Education had this sad 15 year period where it thought “competences” are all you need. Turns out that without the world knowledge to have a base of facts, it is not.
Basically: you can't teach people to think without giving them some facts and ideas to think with. It's like trying to teach woodworking without giving the students any wood.
Does python coding depend on political facts of the world? It might appear not, but actually, the process of reasoning is not an isolated act. The right and wrong way of doing things is codified in social evolution that absorbed all facets of life. Why should you optimize a piece of code for performance? Why performance is needed? What is a bug? What features and UI themes would be more intuitive for humans? There is…
This thing is just bonkers.
Its meant for a Windows machine using ollama but I'm sure anyone who wants to mess with it can point claude code at it to convert it for your own operating system and requirements. After install you can ask it to do something with "vibe 'create me a poem about cheese in cheese.txt'" its workspace is by default the directory the cli was located in when you called it.
Earlier quoted context omitted.
Yeah, if they can fit an 8B model that's really good at improving the output by thinking, running at 16K tok/s on Taalas would be mind-blowing.
Given this and the quality of open models, it makes no sense to me that there’s a future for Anthropic et all?
Earlier quoted context omitted.
Can't you just force it to do structured output via constrained generation?
Yes, I did end up figuring out a clean way to allow normal reasoning inside and then force JSON _after_ the closing . Example here: https://gist.github.com/noperator/6c711ab19027ea8056442df839...
I think the only way to prove that these models are truly as good as they claim is to wait and see if they are getting adopted in practice.