Why Claude 3.5 Sonnet is missing from the benchmark? Even if the real reason is different and completely legitimate, or perhaps purely random, it comes across as "claude does better than our new model so we omitted it because we wanted the tallest bars on the chart to be ours". And as soon as the reader thinks that, they may start to question everything else in your work, which is genuinely awesome!
Reader-LM: Small Language Models for Cleaning and Converting HTML to Markdown
21–30 of 49 posts
Re: Reader-LM: Small Language Models for Cleaning and Converting HTML to Markdown
#22As per reddit their API that converts html to markdown can be used by appending url to https://r.jina.ai like https://r.jina.ai/https://news.ycombinator.com/item?id=41515...
I don't know if its using their new model or their engine
Re: Reader-LM: Small Language Models for Cleaning and Converting HTML to Markdown
#23Feels surprising that there isn't a modern best-in-class non-LLM alternative for this task. Even in the post, they described that they used a hodgepodge of headless Chrome, readability, lots of regex to create content-only HTML. Best I can tell, everyone is doing something similar, only differing in the amount of custom situation regex being used.
How could it possibly be (a better solution) when there are X different ways to do any single thing in html(/css/js)? If you have a website that uses a canvas to showcase the content (think presentation or something like that), where would you even start? People are still discussing whether the semantic web is important; not every page is utf8 encoded, etc. IMHO small LLMS (trained specifically for this) combined wit…
Fully agree on the premise: there are X different ways to do anything on the web. But - prior to this - the solution seemed to be: everyone starts from scratch with some ad-hoc Regex, and plays a game of whackamole to cover the first n of the x different ways to do things.
Best of my knowledge there isn't anything more modern than Mozilla's readability and that's essentially a tool from the early 2010s.
Re: Reader-LM: Small Language Models for Cleaning and Converting HTML to Markdown
#24For as much as I would love for this to work, I'm not getting great results trying out the 1.5b model in their example notebook on Colab. It is impressively fast, but testing it on an arxiv.org page (specifically https://arxiv.org/abs/2306.03872 ) only gives me a short markdown file containing the abstract, the "View PDF" link and the submission history. It completely leaves out the title (!), authors and other links…
Question is why even use these small models?
When you've Google Flash which is lightening fast and cheap.
My brother implemented it in option-k : https://github.com/zerocorebeta/Option-K
It's near instant. So why waste time on small models? It's going to cost more than Google flash.
Re: Reader-LM: Small Language Models for Cleaning and Converting HTML to Markdown
#25Why Claude 3.5 Sonnet is missing from the benchmark? Even if the real reason is different and completely legitimate, or perhaps purely random, it comes across as "claude does better than our new model so we omitted it because we wanted the tallest bars on the chart to be ours". And as soon as the reader thinks that, they may start to question everything else in your work, which is genuinely awesome!
It's damn slow and overkill for such task.
Re: Reader-LM: Small Language Models for Cleaning and Converting HTML to Markdown
#26Maybe I am missing something here, but why would you run "AI" on that task when you go from formal language to formal language? I don't get the usage of "regex/heuristics" either. Why can that task not be completely handled by a classical algorithm? Is it about the removal of non-content parts?
There’s html and then there’s… html. A nicely formatted subset of html is very different from a dom tag soup that is more or less the default nowadays.
That's why the best strategy is to feed the whole page into LLM. (After removing html tags) and just ask LLM to give you the date you need in the format you need.
If there is lots of javascript dom manipulation happening after pageload. Then just render in webdriver and screenshot, ocr and feed the result into LLM and ask it the right questions.
Re: Reader-LM: Small Language Models for Cleaning and Converting HTML to Markdown
#27Unfortunately not getting any good results for RFC 3339 (https://www.rfc-editor.org/rfc/rfc3339), such a page where I think it would be great to convert text into readable Markdown.
The end result is just like the original site but with without any headings and the a lot of whitespace still remaining (but with some non-working links inserted) :/
Using thei API link, this is what it looks like: https://r.jina.ai/https://www.rfc-editor.org/rfc/rfc3339
Re: Reader-LM: Small Language Models for Cleaning and Converting HTML to Markdown
#28Unfortunately not getting any good results for RFC 3339 ( https://www.rfc-editor.org/rfc/rfc3339 ), such a page where I think it would be great to convert text into readable Markdown. The end result is just like the original site but with without any headings and the a lot of whitespace still remaining (but with some non-working links inserted) :/ Using thei API link, this is what it looks like: https://r.jina.ai/htt…
That's their existing API (which I also tried, with... less than desirable results). This post is about a new model, `reader-lm`, which isn't in production yet.
Re: Reader-LM: Small Language Models for Cleaning and Converting HTML to Markdown
#29For as much as I would love for this to work, I'm not getting great results trying out the 1.5b model in their example notebook on Colab. It is impressively fast, but testing it on an arxiv.org page (specifically https://arxiv.org/abs/2306.03872 ) only gives me a short markdown file containing the abstract, the "View PDF" link and the submission history. It completely leaves out the title (!), authors and other links…
Question is why even use these small models? When you've Google Flash which is lightening fast and cheap. My brother implemented it in option-k : https://github.com/zerocorebeta/Option-K It's near instant. So why waste time on small models? It's going to cost more than Google flash.
Privacy, Cost, Latency, Connectivity.
Re: Reader-LM: Small Language Models for Cleaning and Converting HTML to Markdown
#30I'm curious about the dataset. What scenarios need to be covered during training?