Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The last six months in LLMs in five minutes

simonwillison.net

Post reply on HN
Print view View original

381–390 of 631 posts

Re: The last six months in LLMs in five minutes

#381

Post by bluegatty »

koonsolo wrote: post #118
nl wrote: post #27

Earlier quoted context omitted.

> But 'Drawing of a Proper Duck' is almost arbitrary because it may have nothing to do with the 'Specific Duck You Wanted'. That might be the case, but Simon's case "Generate an SVG of a pelican riding a bicycle" is very different. The model actually has to understand what parts of a pelican and bicycle come together in something like an anatomically plausible way. That's a higher level of abstraction than something…

> That might be the case, but Simon's case "Generate an SVG of a pelican riding a bicycle" is very different. When it was new, sure. Right now, models can be trained on that because everybody uses it as a benchmark.

The model is trained on lines, and the word 'pelican' and not much more.

The model does not 'understand' comprehensively the relationship between anatomy, dimensionality, etc..

Re: The last six months in LLMs in five minutes

#382

Post by SlinkyOnStairs »

walthamstow wrote: post #278
Philip-J-Fry wrote: post #248

Earlier quoted context omitted.

I don't want to offend (it's AI coded anyway :)) but that does not scream "high quality" to me. The headline gif on that repo just paints a terrible picture. It can't draw a box correctly, there's random underscores all over the screen. The UI itself is just incredibly incoherent. I don't even know what I'm looking at. Like, no it doesn't seem like very high quality work... It just seems like a vibe coded tool. Edit:…

Take it up with Anthropic. It's actually their billion-dollar TUI product you're commenting on. The problem with being such a naysayer is that you're entirely disconnected from what's going on. You haven't tried an agent like Claude Code and experienced it for yourself, so you don't recognise what it looks like when it's in front of you.

There are two possibilities here:

1) This tool breaks the Claude TUI. Exactly as described by the comment.

2) The Claude TUI itself is broken. The comment is wrong, but assuming the "billion dollar TUI product" is capable of basic rendering and it's the wrapper that broke it, that is an entirely reasonable assumption

The fun here is that both of these softwares were made extensively using AI. No matter which of our options is the case here, the point stands. An AI-built product was shown, it looks obviously ass.

Re: The last six months in LLMs in five minutes

#383

Post by epolanski »

hollowturtle wrote: post #209

> The coding agents got really good It's since november 2025, the so called "inflection point", that I'm still wondering for who coding agents become "really good". All I observe they got better at tool call and answering questions about big codebases, especially if the question has a vague pattern to search, and they're superuseful for that! For generating production code even with a lot of steering and baby sitting…

> But we should stop talking about 1s and 0s

I agree, but you contradicted yourself just one line above.

> For generating production code even with a lot of steering and baby sitting? Absolutely not

Moreover this is further in contradiction with several facts:

1. the majority of this industry has always been composed by mediocre/bad developers, often unable to write a fizz buzz

2. the majority of work in this industry is implementing mundane CRUDs to move and transform trivial data across the organization's stakeholders and/or customer or third parties

3. there's lots of stellar and respected engineers leveraging the tools on a regular basis even on problems that are far from trivial and outputting quality code much faster than they would've done otherwise. Mitchell Hashimoto has blogged about it in his work on Ghostty, Sanfilippo has blogged about it in his work on Redis and so did plenty of others. I know several open source stellar developers who benefitted greatly from these tools, yet you think it cannot improve the quality and output of the most mundane tasks out there?

Re: The last six months in LLMs in five minutes

#384

Post by iammjm »

koonsolo wrote: post #118
nl wrote: post #27

Earlier quoted context omitted.

> But 'Drawing of a Proper Duck' is almost arbitrary because it may have nothing to do with the 'Specific Duck You Wanted'. That might be the case, but Simon's case "Generate an SVG of a pelican riding a bicycle" is very different. The model actually has to understand what parts of a pelican and bicycle come together in something like an anatomically plausible way. That's a higher level of abstraction than something…

> That might be the case, but Simon's case "Generate an SVG of a pelican riding a bicycle" is very different. When it was new, sure. Right now, models can be trained on that because everybody uses it as a benchmark.

you can replace the pelican and the bicycle with your preferred animal and a means of locomotion. I bet you can come up with a pair that definitely wasnt in the training data

Re: The last six months in LLMs in five minutes

#385

Post by Veelox »

hollowturtle wrote: post #209

> The coding agents got really good It's since november 2025, the so called "inflection point", that I'm still wondering for who coding agents become "really good". All I observe they got better at tool call and answering questions about big codebases, especially if the question has a vague pattern to search, and they're superuseful for that! For generating production code even with a lot of steering and baby sitting…

My explanation for the lack of shared experience is very language dependent quality. I work in Go and it's gotten really really good. I have to pick the right abstraction and it can be overly verbose at times but it can make in 5 minutes what would have taken me an hour.

Re: The last six months in LLMs in five minutes

#386

Post by maccard »

Razengan wrote: post #214
hollowturtle wrote: post #209

> The coding agents got really good It's since november 2025, the so called "inflection point", that I'm still wondering for who coding agents become "really good". All I observe they got better at tool call and answering questions about big codebases, especially if the question has a vague pattern to search, and they're superuseful for that! For generating production code even with a lot of steering and baby sitting…

> It's since November 2025, the so called "inflection point", that I'm still wondering for who coding agents become "really good". You can dig up my past comments semi-arguing with simonw where I said AI just isn't good enough yet, but lately I've been using Codex mostly just to review existing Godot/GDScript code: https://github.com/InvadingOctopus/comedot and now I'd say that in this day and age one would have to b…

> I've been on the $20 ChatGPT plan for about a year now, and only started using Codex since like maybe 4 months ago, almost always on the latest model with "Extended Thinking" or "Extra High", because I want my shared code to be as correct as possible because everything else I do depends on it, and I only hit limits like 2 times in the last 3 months.

I've recently tried codex, and I have it set to plan mode with 5.5 and I'm hitting the limits on a single task on a "medium" sized codebase.

Re: The last six months in LLMs in five minutes

#387

Post by hootz »

pr337h4m wrote: post #75

Something that’s largely been ignored: DeepSeek has made context caching virtually free with V4-Flash.

I swear to god that DeepSeek V4-Flash is the most useful model available right now. It's SO FAST and is good enough for so many tasks that I run it most of the time for almost everything. Even when it messes up, it's so cheap to iterate that I can fix most problems without changing the model to a more "capable" one.

Re: The last six months in LLMs in five minutes

#388

Post by timr »

kstenerud wrote: post #361
timr wrote: post #348

Earlier quoted context omitted.

In this case, the visual display was fine -- I was instructing it to fix bad code from a previous round that happened to deliver the right results. Like I said, this is just an example that happens to be CSS. I see this stuff daily, if not hourly.

That's interesting. As I said I haven't tried using LLMs at this level, although I'm about to embark on some this week. What I've found helps (at least at the other layers) is to have principles documents and standards documents for the AI to reference when it's modifying code. Principles documents describe the why, and standards documents describe the how. So for example a few parts from my initial CSS-standards.md…

Yeah, I have those, but it's still pretty hit and miss, and obviously, it ends up being a game of whack-a-mole for everything I find.

I don't mean to over-state the importance of these little errors, just to say that agents do plenty of dumb stuff, even today, and the people who say otherwise are selling something or (hot take incoming) some combination of stupid, lazy and/or delusional.

Re: The last six months in LLMs in five minutes

#389

Post by kstenerud »

SlinkyOnStairs wrote: post #382
walthamstow wrote: post #278

Earlier quoted context omitted.

Take it up with Anthropic. It's actually their billion-dollar TUI product you're commenting on. The problem with being such a naysayer is that you're entirely disconnected from what's going on. You haven't tried an agent like Claude Code and experienced it for yourself, so you don't recognise what it looks like when it's in front of you.

There are two possibilities here: 1) This tool breaks the Claude TUI. Exactly as described by the comment. 2) The Claude TUI itself is broken. The comment is wrong, but assuming the "billion dollar TUI product" is capable of basic rendering and it's the wrapper that broke it, that is an entirely reasonable assumption The fun here is that both of these softwares were made extensively using AI. No matter which of our o…

The issue is likely that the tmux session being generated is for some reason not propagating all term caps. Most likely it's an interop issue between tmux and docker and the image running under docker - possibly even something with the terminal client that the pipeline doesn't like somewhere.

Claude Code correctly reduces its display to 7-bit ASCII in response (still functional, although less pretty). Once I get around to fixing this, it will probably result in another section in https://github.com/kstenerud/yoloai/blob/main/docs/dev/backe...

Edit: Looks like it's the terminal. That's a rabbit hole for another day.

Running through VS Code's terminal via VSCode tunnel, it looks like it normally does.

https://freeimage.host/i/BySkkDN

Re: The last six months in LLMs in five minutes

#390

Post by timr »

sjagauanbdvva wrote: post #360
timr wrote: post #334

Earlier quoted context omitted.

While reading this thread, I literally just caught an agent putting in the following CSS selector in a rule: > .row > div > div, .alert This is fairly simple CSS, not multi-threaded systems development. A bar low enough that you could trip over it. I catch this kind of stuff all the time (literally every run), but only because I read every line. Most of it wouldn't be the end of the world for any particular task, but…

Or they don’t know CSS. Amazing how the LLM is godly with things I don’t understand, and falls over completely when it works in my domain… I wonder why that is /s

Yes, it's a mystery, isn't it?

Specifically for CSS, these bots really want to just barf out tailwind-style crap. If you deviate even slightly from the standards and practices of the modal front-end developer, you quickly see how these things are brittle, and no amount of prompting and cajoling will truly affect their behavior. In this case, you're kind of seeing the downstream affects of saying "no, do NOT do tailwind, make actual CSS with actual semantic class names please and thank you."

Perhaps ironically, this results in the quality of output I might expect if I had prompted a right-out-of-bootcamp coder to do the same. (But at least it doesn't whine about it!)

Post reply on HN