Live data from Hacker News

Neutrino-1 8B

fermionresearch.com

21–30 of 51 posts

Re: Neutrino-1 8B

#21

There’s a new announcement every other day wrt models. How do y’all keep track of them all same know what’s decent? Good grief! And if it’s decent today, it’s shit in eight months! I tool hop as much as the next dev but this is a bit much.

There's no need to, the 50 foot view is simply that many alternatives exist and they mostly fall into 3 meaningful weight classes with comparable performance among each class's members: too expensive to use indiscriminately, too big to run at home, and too small for complex work. As for names and faces in between, the overarching conclusion is that we're rapidly approaching commodity status and those don't really matter much

Re: Neutrino-1 8B

#22
post #14
post #5

Earlier quoted context omitted.

Independent testing of prismml suggest quite a capability drop off outside of their cherry picked benchmarks. I'll be curious to see what this model achieves though.

I really had high hopes for the larger Ternary Bonsai and it feels like there is scope to improve, but I get the sense (albeit a naïve, probably not fully informed sense) that improvement can perhaps only come by training directly into ternary.

I’ve actually been really impressed with the 27b model they recently released - amazing performance approaching 40 tok/s on m4 max and I didn’t run into any quality issues in the small set of tasks I tried. Haven’t gone full coding with it yet but suspect it’s better than say a 9b or 12b model.

Re: Neutrino-1 8B

#23
post #14
post #5

Earlier quoted context omitted.

Independent testing of prismml suggest quite a capability drop off outside of their cherry picked benchmarks. I'll be curious to see what this model achieves though.

I really had high hopes for the larger Ternary Bonsai and it feels like there is scope to improve, but I get the sense (albeit a naïve, probably not fully informed sense) that improvement can perhaps only come by training directly into ternary.

All you need is Ternary Aware Training and for AI researchers to come up with a backronym for TIT.

Re: Neutrino-1 8B

#24

AI slop site with AI slop research... Blog populated with incoherent PR material generated by Yet Another AI. Sigh...

Can Dang implement a slop rating on submitted pages? Not a block but at least a % likelihood of AI slop content and that could also be tied with a BS rating as well.

Could use AI for both which seems hilariously appropriate.

Re: Neutrino-1 8B

#25
post #15

Can’t say I’m a fan of containers for this. A big chunk of local LLM gains come (imo) from the open modular nature of llama.cpp and friends. Easy to modify. Easy to experiment. Containers are the proprietary binary blob in hardware world equivalent

What? How are those even related with each other? You can just as easy modify and experiment with llama.cpp in a container as outside of it, they really shouldn't impact one another. Containers don't suddenly make llama.cpp less "open modular" somehow, and I'm not sure how you'd arrive as such conclusion.

Re: Neutrino-1 8B

#26
Unfortunately it crashed out 'no space left on device' while installing the python demo/quickstart.

Only problem was there is plenty of space on the device. PLENTY (not quite 750gb).

Re: Neutrino-1 8B

#28

Earlier quoted context omitted.

Unfortunately Fermion Research appears to entirely AI generate all of their content here, even for the research section: https://www.fermionresearch.com/research/neutrino-8b/ "Neutrino-1 8B was trained natively in its shipping format. There is no full-precision product model that was rounded afterward: the ternary representation is the medium the weights learned in, and the training methods that hold this quality at…

I guess it's saying how many of the weights are -1, 0 or +1.

It is, but why? And what's with the bizarre way of phrasing that? Why the bizarre observation that, indeed, nobody asked for it?

Re: Neutrino-1 8B

#30

AI slop site with AI slop research... Blog populated with incoherent PR material generated by Yet Another AI. Sigh...

Can Dang implement a slop rating on submitted pages? Not a block but at least a % likelihood of AI slop content and that could also be tied with a BS rating as well. Could use AI for both which seems hilariously appropriate.

Or perhaps a slop flag on submissions that gets added once enough readers have flagged it as such. I'd love that feature.
Post reply on HN