BitNet b1.58 2B4T Technical Report
arxiv.org
BitNet b1.58 2B4T Technical Report
1–10 of 33 posts
Re: BitNet b1.58 2B4T Technical Report
#2Does someone have a good understanding how 2B models can be useful in production? What tasks are you using them for? I wonder what tasks you can fine-tune them on to produce 95-99% results (if anything).
Re: BitNet b1.58 2B4T Technical Report
#3Does someone have a good understanding how 2B models can be useful in production? What tasks are you using them for? I wonder what tasks you can fine-tune them on to produce 95-99% results (if anything).
Summarization on mobile/embedded might be a good usecase?
Re: BitNet b1.58 2B4T Technical Report
#4Does someone have a good understanding how 2B models can be useful in production? What tasks are you using them for? I wonder what tasks you can fine-tune them on to produce 95-99% results (if anything).
I'm just playing / experimenting around with local LLM's. Just to see what I can do with them. One thing that comes to mind is gaming: E.g. text/dialog generation in procedural worlds / adventures.
Re: BitNet b1.58 2B4T Technical Report
#5Does someone have a good understanding how 2B models can be useful in production? What tasks are you using them for? I wonder what tasks you can fine-tune them on to produce 95-99% results (if anything).
2B models by themselves aren't so useful, but it's very interesting as a proof of concept, because the same technique used to train a 200B model could produce one that's much more efficient (cheaper and more environmentally friendly) than existing 200B models, especially with specialised hardware support.
Re: BitNet b1.58 2B4T Technical Report
#6They give some description of how their weights are stored: they pack 4 weights into an int8, indicating that their storage format isn't optimal (2 bits per weight instead of the optimal ~1.58 bits). But I don't know enough about LLM internals to know how material this is.
Could anyone break down the steps further?
Re: BitNet b1.58 2B4T Technical Report
#7Does someone have a good understanding how 2B models can be useful in production? What tasks are you using them for? I wonder what tasks you can fine-tune them on to produce 95-99% results (if anything).
The use case for small models include sentiment and intent analysis, spam and abuse detection, and classifications of various sorts. Generally LLM are thought of as chat models but the output need not be a conversation per se.
Re: BitNet b1.58 2B4T Technical Report
#8Does someone have a good understanding how 2B models can be useful in production? What tasks are you using them for? I wonder what tasks you can fine-tune them on to produce 95-99% results (if anything).
The on-device models used for Apple Intelligence (writing tools, notification and email/message summaries, etc.) are around ~3B parameters.
I mean, they could be better (to put it nicely), but there is a legitimate use-case for them and I'd love to see more work in this space.
https://machinelearning.apple.com/research/introducing-apple...
Re: BitNet b1.58 2B4T Technical Report
#9Does someone have a good understanding how 2B models can be useful in production? What tasks are you using them for? I wonder what tasks you can fine-tune them on to produce 95-99% results (if anything).
Anything you'd normally train a smaller custom model for, but with an LLM you can use a prompt instead of training.
Re: BitNet b1.58 2B4T Technical Report
#10You can try out the model in a demo they have setup: https://bitnet-demo.azurewebsites.net/