101–110 of 255 posts
Heh, what a coincidence, just today one of my students presented research results which also confirmed this. He removed MLP from Qwen and the model still could do transformation tasks on input but lost knowledge.
Earlier quoted context omitted.
It is for building agentic capabilities into very small devices like phones, glasses, watches and more. Does that make sense?
[flagged]
That aside- a very small model that takes text and outputs structured json according to a spec is nice. It let's you turn natural language into a user action. For example, command palettes could benefit from this.
If you can do a tiny bit of planning (todo) and chain actions, it seems reasonable that you could traverse a rich state space to achieve some goal on behalf of a user.
Games could use something like it for free form dialog while stool enforcing predefined narrative graphs etc.
I'm sure you could come up with more. It's a fuzzy function.
Earlier quoted context omitted.
Pardon me, do I know you? Why are you attacking me?
I don't think they're attacking you, but suggesting you read more carefully. The information provided is correct and clear, but you need to let go of your own biases when consuming it. I personally prefer the M to the B. I guess as an engineer, noticing the units comes pretty naturally.
Announcing something that's 1/1000th is significant and remarkable! Hiding it in a single letter is burying the lede.
Earlier quoted context omitted.
[flagged]
Throwing a few things out - HN has changed over the years, but people make stuff to make stuff. There don't need to be product use cases. The tone of the comment goes against the spirit of HN - likely the reason for downvotes. That aside- a very small model that takes text and outputs structured json according to a spec is nice. It let's you turn natural language into a user action. For example, command palettes coul…
OK. Great! So it doesn't need to be a commercial product. But does it do something (anything?) interesting? I'm interested in your games example, I'd love to see it done in real life. IIUC, game AIs are actually much more constrained and predictable for play-ability reasons. If you let it go all free form a plurality of players have a "WTF??!?" experience which is super Not Good.
No FFN is blowing my mind. This is pretty much "Attention Is ACTUALLY All You Need". Reminds me of BERT Q&A which would return indices into the input context, but even that had a FFN. Really exciting work.
That M versus B is way too subtle. 0.026B is my suggestion
[flagged]
That M versus B is way too subtle. 0.026B is my suggestion
Earlier quoted context omitted.
Throwing a few things out - HN has changed over the years, but people make stuff to make stuff. There don't need to be product use cases. The tone of the comment goes against the spirit of HN - likely the reason for downvotes. That aside- a very small model that takes text and outputs structured json according to a spec is nice. It let's you turn natural language into a user action. For example, command palettes coul…
> people make stuff to make stuff. There don't need to be product use cases. OK. Great! So it doesn't need to be a commercial product. But does it do something (anything?) interesting ? I'm interested in your games example, I'd love to see it done in real life. IIUC, game AIs are actually much more constrained and predictable for play-ability reasons. If you let it go all free form a plurality of players have a "WTF?…
That being said, small models like these have plenty of use cases. They allow for extra "slack" to be introduced into a programmatic workflow in a compute constrained environment. Something like this could help enable the "ever present" phone assistant, without scraping all your personal data and sending it off to Google/OpenAI/etc. Imagine if keywords in a chat would then trigger searches on your local data to bring up relevant notes/emails/documents into a cache, and then this cache directly powers your autocomplete (or just a sidebar that pops up with the most relevant information). Having flexible function calling in that loop is key for fault tolerance and adaptability to new content and contexts.
Its cool. Enjoy it.