31–40 of 62 posts
This might be a very basic question, but as a dev whose only interaction with models is using the main commercial ones (sonnet, ChatGPT and the like), what are some usecases for these smaller local models? What usages can be reasonable to expect from them? Are there uses out of the box or does one have to go through some custom post-training to get useful behavior? I feel like there is a huge gap between understandin…
Hey all, I created this model with a top notch team. I answered many questions last week when this hit the front page, and happy to answer more here as well. https://news.ycombinator.com/item?id=44902148 Personally I'm excited that you all have access to this model now and hope you all get value out of using them.
Does it have function calls? Can we use it with MCP?
Hey all, I created this model with a top notch team. I answered many questions last week when this hit the front page, and happy to answer more here as well. https://news.ycombinator.com/item?id=44902148 Personally I'm excited that you all have access to this model now and hope you all get value out of using them.
Can someone (or OP) point me to a recipe to fine tune a model like this for natural language tasks like complicated NER or similar workflows? I tried finetuning Gemma3 270M when it came out last week without any success. A lot of tutorials are geared towards chat applications and role playing but I feel this model could be great for usecases like mine where I am trying to extract clean up and extract data from PDFs w…
https://huggingface.co/dslim/bert-base-NER
Just wondering if it’s worth testing and what it would be most useful for.
If you wanted to train it from scratch, how long would it take on a reasonable GPU setup?
That’s wild that with a KV cache and compilation on the Mac CPU you are faster than on an A100 GPU.
Do you think these very small models have some utility in the real world? Apart from learning and academic purposes of course.
- reply in a specific way, like a specific JSON schema, or in the voice of a character - be very good at classifying text (e.g. emails, or spam) - be a great summarizer for large amounts of text, e.g. turn emails into short titles or url slugs - adding tags/categories per your pre-defined rules (e.g. for communities, tagging content, marketing) - for detecting spam, or duplicates, or flagging things
You won't be able to write code or prose with these, but they're great for a huge array of very narrow set of use cases
What's neat about "stupid" models like this is that they're less likely to go off and dream up a bunch of irrelevant content, because they don't know much about the world / won't have too much context to pull from
This might be a very basic question, but as a dev whose only interaction with models is using the main commercial ones (sonnet, ChatGPT and the like), what are some usecases for these smaller local models? What usages can be reasonable to expect from them? Are there uses out of the box or does one have to go through some custom post-training to get useful behavior? I feel like there is a huge gap between understandin…
- private, on-device models (possibly with lower latency than models via web API); also edge devices
- algorithm research (faster and cheaper to prototype new ideas)
- cheap tasks, like classification/categorization; sure, you don't need a decoder-style LLM for that, but it has the advantage of being more free-form, which is useful in many scenarios; or maybe a sanity checker for grammar; or even a router to other model (GPT-5 style)
That’s wild that with a KV cache and compilation on the Mac CPU you are faster than on an A100 GPU.