11–20 of 49 posts
I can say that enough. Small Language Models are the future. https://www.lycee.ai/blog/why-small-language-models-are-the-...
Maybe I am missing something here, but why would you run "AI" on that task when you go from formal language to formal language? I don't get the usage of "regex/heuristics" either. Why can that task not be completely handled by a classical algorithm? Is it about the removal of non-content parts?
There’s html and then there’s… html. A nicely formatted subset of html is very different from a dom tag soup that is more or less the default nowadays.
Instead of applying an obscure set of heuristic by hand, let the LM figure out the best way starting from a lot of data.
The model is bound to be less debuggable and much more difficult to update, for experts.
But in the general case it will work well enough.
Basically, it's utility which completes commandline for you
While playing with it, we thought about creating a custom small model for this.
But it was really limiting! If we use small model trained on MAN pages, bash scripts, stack overflow and forums etc...
We miss the key component, using a larger model like flash is more effective as this model knows lot more about other things.
For example, I can ask this model to simply generate a command that lets me download audio from a youtube url.
It is impressively fast, but testing it on an arxiv.org page (specifically https://arxiv.org/abs/2306.03872) only gives me a short markdown file containing the abstract, the "View PDF" link and the submission history. It completely leaves out the title (!), authors and other links, which are definitely present in the HTML in multiple places!
I'd argue that Arxiv.org is a reasonable example in the age of webapps, so what gives?
1. The quality of HTML → Markdown conversion results is easier to evaluate.
2. The HTML → Markdown process is essentially a more sophisticated form of copy-and-paste, where AI generates specific symbols (such as ##, *) rather than content.
3. Rule-based systems are significantly more cost-effective and faster than running an LLM, making them applicable to a wider range of scenarios.
These are just my assumptions and judgments. If you have practical experience, I'd welcome your insights.
So regex version still beats the LLM solution. There's also the risk of hallucinations. I wonder if they tried to make SML which would rewrite or update the existing regex solution instead of generating the whole content again? This would mean less output tokens, faster inference and output wouldn't contain hallucinations. Although, not sure if small language models are capabable to write regex