Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The last six months in LLMs, illustrated by pelicans on bicycles

simonwillison.net

Post reply on HN
Print view View original

141–150 of 244 posts

Re: The last six months in LLMs, illustrated by pelicans on bicycles

#141

Post by diggan »

zahlman wrote: post #136
diggan wrote: post #73

Earlier quoted context omitted.

Lets give it a try, if you're willing to be the experiment subject :) The prompt is "Generate an SVG of a pelican riding a bicycle" and you're supposed to write it by hand, so no graphical editor. The specification is here: https://www.w3.org/TR/SVG2/ I'm fairly certain I'd lose interest in getting it right before I got something better than most of those.

> The colors use traditional bicycle brown (#8B4513) and a classic blue for the pelican (#4169E1) with gold accents for the beak (#FFD700). The output pelican is indeed blue. I can't fathom where the idea that this is "classic", or suitable for a pelican, could have come from.

My guess would be that it doesn't see the web colors (CSS color hexes) as proper hex triplets, but because of tokenization it could be something dumb like '#8B','451','3' instead. I think the same issue happens around multiple special characters after each other too.

Re: The last six months in LLMs, illustrated by pelicans on bicycles

#142

Post by voiper1 »

simonw wrote: post #131
zahlman wrote: post #128

> If you lost interest in local models—like I did eight months ago—it’s worth paying attention to them again. They’ve got good now! > As a power user of these tools, I want to stay in complete control of what the inputs are. Features like ChatGPT memory are taking that control away from me. You reap what you sow.... > I already have a tool I built called shot-scraper, a CLI app that lets me take screenshots of web pa…

I had the LLM write a bash script for me that used my https://shot-scraper.datasette.io/ tool - on the basis that it was a neat opportunity to demonstrate another of my own projects. And honestly, even with LLM assistance getting Image Magick to output a 1200x600 image with two SVGs next to each other that are correctly resized to fill their half of the image sounds pretty tricky. Probably easier (for Claude) to achi…

Isn't "left or right" _followed_ by rationale asking it to rationalize it's 1 word answer - I thought we need to get AI to do the chain of though _before_ giving it's answer for it to be more accurate?

Re: The last six months in LLMs, illustrated by pelicans on bicycles

#143

Post by [deleted] »

azinman2 wrote: post #87
adrian17 wrote: post #81

> This was one of the most successful product launches of all time. They signed up 100 million new user accounts in a week! They had a single hour where they signed up a million new accounts, as this thing kept on going viral again and again and again. Awkwardly, I never heard of it until now. I was aware that at some point they added ability to generate images to the app, but I never realized it was a major thing (p…

Except this went very mainstream. Lots of turn myself into a muppet, what is the human equivalent for my dog, etc. TikTok is all over this. It really is incredible.

[deleted]

Re: The last six months in LLMs, illustrated by pelicans on bicycles

#144

Post by senthil_rajasek »

baq wrote: post #115
Jedd wrote: post #96

Earlier quoted context omitted.

Yeah, but so were the bored ape NFTs - none of these ephemeral fads are any indication of quality, longevity, legitimacy, or interest.

It’s hard to think of a worse analogy TBH. My wife is using ChatGPT to change photos (still is to this day), she didn’t use it or any other LLM until that feature hit. It is a fad, but it’s also a very useful tool. Ape NFTs are… ape NFTs. Useless. Pointless. Negative value for most people.

"My wife is using ChatGPT to change photos (still is to this day), she didn’t use it or any other LLM until that feature hit."

This is deja vu, except instead of ChatGPT to edit photos it was instagram a decade ago.

Re: The last six months in LLMs, illustrated by pelicans on bicycles

#145

Post by mdaniel »

big_hacker wrote: post #62

Honestly the metric which increased the most is the marketing and astroturfing budget of the major players (OpenAI, Anthropic, Google and Deepseek). Say what you want about Facebook but at least they released their flagship model fully open.

> model fully open.

uh-huh https://www.llama.com/llama4/license/

Re: The last six months in LLMs, illustrated by pelicans on bicycles

#146

Post by dotemacs »

blackhaj7 wrote: post #91
joshstrange wrote: post #3

I really enjoy Simon’s work in this space. I’ve read almost every blog post they’ve posted on this and I love seeing them poke and prod the models to see what pops out. The CLI tools are all very easy to use and complement each other nicely all without trying to do too much by themselves. And at the end of the day, it’s just so much fun to see someone else having so much fun. He’s like a kid in a candy store and that…

Same sentiment!

The same here.

Because of him, I installed a RSS reader so that I don't miss any of his posts. And I know that he shares the same ones across Twitter, Mastodon & Bsky...

Re: The last six months in LLMs, illustrated by pelicans on bicycles

#147

Post by timewizard »

nathan_phoenix wrote: post #6

My biggest gripe is that he's comparing probabilistic models (LLMs) by a single sample. You wouldn't compare different random number generators by taking one sample from each and then concluding that generator 5 generates the highest numbers... Would be nicer to run the comparison with 10 images (or more) for each LLM and then average.

My biggest gripe is he didn't include a picture of an actual pelican.

https://www.google.com/search?q=pelican&udm=2

The "closest pelican" is not even close.

Re: The last six months in LLMs, illustrated by pelicans on bicycles

#148

Post by ActorNightly »

joshuajooste05 wrote: post #135

Does anyone have any thoughts on privacy/safety regarding what he said about GPT memory. I had heard of prompt injection already. But, this seems different, completely out of humans control. Like even when you consider web search functionality, he is actually right, more and more, users are losing control over context. Is this dangerous atm? Do you think it will become more dangerous in the future when we chuck even…

Sort of. The thing is with agentic models, you are basically entering probability space where it can do real actions in the form of http requests if the statistical output leads it to it.

Re: The last six months in LLMs, illustrated by pelicans on bicycles

#149

Post by isx726552 »

> I’ve been feeling pretty good about my benchmark! It should stay useful for a long time... provided none of the big AI labs catch on.

> And then I saw this in the Google I/O keynote a few weeks ago, in a blink and you’ll miss it moment! There’s a pelican riding a bicycle! They’re on to me. I’m going to have to switch to something else.

Yeah this touches on an issue that makes it very difficult to have a discussion in public about AI capabilities. Any specific test you talk about, no matter how small … if the big companies get wind of it, it will be RLHF’d away, sometimes to the point of absurdity. Just refer to the old “count the ‘r’s in strawberry” canard for one example.

Re: The last six months in LLMs, illustrated by pelicans on bicycles

#150

Post by jauntywundrkind »

senthil_rajasek wrote: post #144
baq wrote: post #115

Earlier quoted context omitted.

It’s hard to think of a worse analogy TBH. My wife is using ChatGPT to change photos (still is to this day), she didn’t use it or any other LLM until that feature hit. It is a fad, but it’s also a very useful tool. Ape NFTs are… ape NFTs. Useless. Pointless. Negative value for most people.

"My wife is using ChatGPT to change photos (still is to this day), she didn’t use it or any other LLM until that feature hit." This is deja vu, except instead of ChatGPT to edit photos it was instagram a decade ago.

Applying some filters and adding some overlay text is something some folks did, but there's such a massive creative world that's opened up, where all we have to do is ask.
Post reply on HN