So I'd like to see a Nemotron3 Ultra converted to a 1.58bit format then have them retrain on the open dataset.
Neutrino-1 8B
31–40 of 51 posts
Re: Neutrino-1 8B
#32Earlier quoted context omitted.
I guess it's saying how many of the weights are -1, 0 or +1.
It is, but why? And what's with the bizarre way of phrasing that? Why the bizarre observation that, indeed, nobody asked for it?
Also, wouldn't normalisation tend to produce a mean of zero in the weights, anyhow?
Re: Neutrino-1 8B
#33Earlier quoted context omitted.
I really had high hopes for the larger Ternary Bonsai and it feels like there is scope to improve, but I get the sense (albeit a naïve, probably not fully informed sense) that improvement can perhaps only come by training directly into ternary.
I’ve actually been really impressed with the 27b model they recently released - amazing performance approaching 40 tok/s on m4 max and I didn’t run into any quality issues in the small set of tasks I tried. Haven’t gone full coding with it yet but suspect it’s better than say a 9b or 12b model.
> suspect it’s better than say a 9b or 12b model
Whaaat, a 27b model might be better than 9b or 12b model? What would make you do such an outrageous claim?
Re: Neutrino-1 8B
#34The content on that page is too AI-generated to make sense to me; I don't understand what the model is for.
AI doesn't want anything, so it doesn't care whether it conveys meaning in its writing. And, apparently the developers of this project also don't care whether it conveys meaning. They just assume we'll wade through the slop? I dunno.
It has no ability to ground output in causality. It does not understand what causes humans to learn. It’s a correlation and randomness engine.
Re: Neutrino-1 8B
#35Earlier quoted context omitted.
I’ve actually been really impressed with the 27b model they recently released - amazing performance approaching 40 tok/s on m4 max and I didn’t run into any quality issues in the small set of tasks I tried. Haven’t gone full coding with it yet but suspect it’s better than say a 9b or 12b model.
> suspect it’s better than say a 9b or 12b model Whaaat, a 27b model might be better than 9b or 12b model? What would make you do such an outrageous claim?
Sorry I should have clarified - I meant that a ternary 27b model would outperform a non-quantized or 8 bit quantized 9 or 12b model - which it is generally close to (or much smaller than) in size. So yeah the comparison I was trying to make was between models of equivalent size or models that could run on similarly sized hardware.
Re: Neutrino-1 8B
#36Earlier quoted context omitted.
I guess it's saying how many of the weights are -1, 0 or +1.
It is, but why? And what's with the bizarre way of phrasing that? Why the bizarre observation that, indeed, nobody asked for it?
> sign-balanced to a hundredth of a point with no constraint asking for it."
This isn't their model, this is (probably?) ChatGPT doing a brag / promo deck authorial voice. It routinely uses half a dozen sentence constructions that are relatively uncommon in normal or technical speech. Eccentric. Persuasive. Trying too hard. Restating its point in a promotional way that doesn't sound natural, leading into a sentence where it hyperbolically sells you on having done the impossible.
Normal persuasive speech uses these constructions, especially public speaking doing a VC pitch or an Ancient Aliens, but it would be fucking strange if a person started chaining them in normal conversation and using little else, trying to insinuate competence.
You start to recognize it pretty quickly on Youtube.
Re: Neutrino-1 8B
#37Earlier quoted context omitted.
> suspect it’s better than say a 9b or 12b model Whaaat, a 27b model might be better than 9b or 12b model? What would make you do such an outrageous claim?
Sorry I should have clarified - I meant that a ternary 27b model would outperform a non-quantized or 8 bit quantized 9 or 12b model - which it is generally close to (or much smaller than) in size. So yeah the comparison I was trying to make was between models of equivalent size or models that could run on similarly sized hardware.
Ah yeah, that makes a ton more sense :) I mean, what you said earlier also makes sense but was too obvious, now it makes sufficient sense, thanks for explaining :)
Re: Neutrino-1 8B
#38hey guys, founder here. I am working completely solo right now running on grant funding, so things slipped through on the website quality, it is largely written by AI, I'll deslopify the blogs and be much clearer about methodology. Neutrino-1 is a starting point for Fermion, It's a density experiment on how much capability survives per byte. rn working on really fast and efficient TTS and realtime models to which this architecture is generalizing pretty well.
Sorry for all the confusion from reading through the blogs, should be fixed in a couple of days!
Re: Neutrino-1 8B
#39I was really hoping this would have something to do with neutrinos
Re: Neutrino-1 8B
#40Unfortunately it crashed out 'no space left on device' while installing the python demo/quickstart. Only problem was there is plenty of space on the device. PLENTY (not quite 750gb).
are you pn linux? reproduced the same thing just now its pip dumping torch into /tmp while it installs, and on most distros /tmp is ram bound and capped at half your memory, so it fills up while your actual disk jsut sits there. export TMPDIR=/some/big/path before installing should fix it. adding a proper check next release so it actually tells you which folder ran out.