21–30 of 52 posts
What I do is convert to markdown, that way you still get some semantic structure. Even built an Elixir library for this: https://github.com/agoodway/html2markdown
Isn't GPT-4o multimodal? Shouldn't I be able to just feed in an image of the rendered HTML, instead of doing work to strip tags out?
it is theoretically possible, but the results and bandwidth would be worse. sending an image that large would take a lot longer than sending text
By default it will strip all HTML tags and return just the text:
curl 'https://simonwillison.net/' | strip-tags
curl 'https://simonwillison.net/' | strip-tags .quote
curl 'https://simonwillison.net/' | strip-tags .quote -t div -t blockquote
Running this command:
curl 'https://simonwillison.net/' | strip-tags .quote -t div -t blockquote -m
history | tail -n
2000 | llm -s "Write aliases for my zshrc based on my
terminal history. Only do this for most common features.
Don't use any specific files or directories."
—
anjor #
3:01 pm
/ ai, generative-ai, llms, llm
Art is notoriously
hard to define, and so are the differences between good art
and bad art. But let me offer a generalization: art is
something that results from making a lot of choices. […] to
oversimplify, we can imagine that a ten-thousand-word short
story requires something on the order of ten thousand
choices. When you give a generative-A.I. program a prompt,
you are making very few choices; if you supply a hundred-word
prompt, you have made on the order of a hundred choices. If
an A.I. generates a ten-thousand-word story based on your
prompt, it has to fill in for all of the choices that you are
not making.
— Ted Chiang #
10:09 pm
/ art, new-yorker, ai, generative-ai, ted-chiang
This results in a kind of innerText you get in browsers, great and light to pass into LLMs.
defp extract_inner_text(html) do
html
|> Floki.parse_document!()
|> Floki.find("body")
|> Floki.traverse_and_update(fn
{tag, _attrs, _children} = _node when tag in ["script", "style"] ->
nil
node ->
node
end)
|> Floki.text(sep: " ")
|> String.trim()
|> String.replace(~r/\s+/, " ")
endIsn't GPT-4o multimodal? Shouldn't I be able to just feed in an image of the rendered HTML, instead of doing work to strip tags out?
ChatGPT is clearly trained on wikipedia, is there any concern about its knowledge from there polluting the responses? Seems like it would be better to try against data it didn't potentially already know.
There are a few optimizations we can make:
- trip all content in and - use Readability.js for articles - extract structured content from oEmbed
It works surprisingly well for me, even with gpt-4o-mini
name=john,age=23
name=anna,age=26
{
matches: [
{ name: "john", age: 23 },
{ name: "anna", age: 26 }
]
}