Live data from Hacker News

The last six months in LLMs in five minutes

simonwillison.net

361–370 of 631 posts

Re: The last six months in LLMs in five minutes

#361
post #348

Earlier quoted context omitted.

I haven't done any CSS/HTML/JS level work with Claude yet. I've mainly been using it for systems level stuff. LLMs have traditionally had problems with visual rendering (the good ol' pelican on the bicycle test). I wonder if this is more of the same?

In this case, the visual display was fine -- I was instructing it to fix bad code from a previous round that happened to deliver the right results. Like I said, this is just an example that happens to be CSS. I see this stuff daily, if not hourly.

That's interesting. As I said I haven't tried using LLMs at this level, although I'm about to embark on some this week.

What I've found helps (at least at the other layers) is to have principles documents and standards documents for the AI to reference when it's modifying code. Principles documents describe the why, and standards documents describe the how.

So for example a few parts from my initial CSS-standards.md (still needs a lot of revision):

    ## Utility-first discipline

    **Raw utilities everywhere by default. Never `@apply` for "components."** `@apply` exists only for
    true low-level primitives that can't live in a template (e.g., `prose` overrides, embedded
    third-party widget shells).

    Wathan's stated position: extract only on "worrisome duplication." The Tailwind team explicitly
    describes `@apply` as a tool you reach for after first reaching for templates. **Premature CSS
    abstraction is the failure mode.**

    ## Spacing

    Use only the default scale (`0, 0.5, 1, 1.5, 2, 3, 4, 6, 8, 12, 16, 24…`). **Never `p-[13px]`.** If
    you need a value, change the scale in `@theme`:

    ```css
    @theme {
      --spacing: 0.25rem;
    }
    ```

    v4 uses a single `--spacing` multiplier; everything derives from it.

    ## Anti-patterns (banned)

    - **`!` important prefix** (`!bg-red-500`). Fix specificity properly.
    - **Arbitrary values for colour** (`bg-[#1da1f2]`). Define in `@theme`.
    - **Arbitrary pixel offsets** as default (`top-[3px]`). Use the spacing scale. Tolerated only as
      rare one-offs.
    - **Nested custom CSS more than one level deep.**
    - **`@apply` for any class that wraps fewer than ~5 utilities** or appears in fewer than ~3
      templates.
    - **Dynamic class string interpolation** (`text-${level}-500`) — purger can't see these.
    - **Custom breakpoints in v1.**
    - **Inline `` blocks.** All CSS goes through `assets/css/app.css`.

Re: The last six months in LLMs in five minutes

#362

Earlier quoted context omitted.

You claim "very high quality" but can't even get the basic UI working properly. You wrap tmux and a container in 2k lines of code and claim quality, I think the comment above was aimed at this claim.

The UI is working properly. Interfering with Anthropic's UI, or any of the other agent harness' UIs it supports, would be madness incarnate. I also strongly suspect that you'd only taken a cursory glance at the top of the readme prior to passing judgment.

I did not much more than a cursory glance too, but found "./sandbox/create.go", a ~1300 lines long file with so much duplication even within just itself that I stopped counting.

Now it was a long time ago I did Go professionally, but I'm also in the camp of "That doesn't really count as high-quality", although I know for a fact you can get quality code out of LLMs, but I don't think that's a good showcase of that.

Re: The last six months in LLMs in five minutes

#363

Earlier quoted context omitted.

It can look like that in certain conditions. The question is why are you so eager to give critique on unrelated work, appearing in a demo screencap, to someone who didn't produce it?

I don't know what you're talking about. His tool wraps Claude and breaks the TUI. What's so hard to understand? That's valid critique. What world have I woke up in today?

To be honest I assumed it was the screencap software running a basic terminal env without bells and whistles that CC needs, which I've seen before. If the actual tool functions like that too, that's not great. That said, it works for them, it works for them.

Re: The last six months in LLMs in five minutes

#364

Earlier quoted context omitted.

So why has your tool completely broken the Claude Code UI then? Can't you see in the gif? It's completely broken. My Claude doesn't look like that. Neither does anyone else's.

Claude Code will automatically "dumb" the TUI down a bit when it can't properly detect certain terminal capabilities, to avoid potential font rendering issues. Likely there are some terminal caps that aren't being properly preserved inside of the sandbox. It's never bothered me since the agent itself works fine.

I think you can fix that by setting an environment variable (regarding the terminal?) but it was a while since I checked. (I was running Claude as a subprocess and had similar issues.)

Also this reminds me of a principle I learned from a mentor. "People are visual buyers. If it looks good, people will think the code is good."

Unfortunately it doesn't matter whose fault the janky TUI is, people will see that and associate it with your software.

Re: The last six months in LLMs in five minutes

#365

I asked Gemini for a video of 'pelican riding a unicycle in hyde park' - I was blown away by the output: https://gemini.google.com/share/55e250c99693

Graphically perfect, but content-wise nonsense. The pelican's center of gravity is clearly behind the wheel. It needs to be above or very slightly ahead of the wheel.

I do hope that JEPA can help resolve the nonsense from AI models.

Re: The last six months in LLMs in five minutes

#366

I asked Gemini for a video of 'pelican riding a unicycle in hyde park' - I was blown away by the output: https://gemini.google.com/share/55e250c99693

Graphically perfect, but content-wise nonsense. The pelican's center of gravity is clearly behind the wheel. It needs to be above or very slightly ahead of the wheel.

Still impressed. And, to be honest, I don't think that this problem matter much. Physical accuracy is very nice, but for example is not the most important aspect when I watch a fantasy movie. Or even a scifi one.

Re: The last six months in LLMs in five minutes

#367
post #259

Earlier quoted context omitted.

I've been a teacher (most of the time a college professor) for...a long time. Nowadays, when preparing a new course, I definitely work with AI: "Here's what I want, and who my audience is - give me a course outline". That gives me a starting point. Of course, I modify it. Maybe I bounce back and forth to the AI for further refinements and suggestions, but ultimately I have to be happy with the result. When prepping t…

> AI is a tool. Use it appropriately Yes, but no room is made for people who see no use for it. There is a forced-consensus that this technology is useful, which I have to combat against at work. We teach in a very different environment, but your use sounds typical of my colleagues. "I ask it for suggestions and pick one", but nobody seems to wonder about what is lost when we shrink the horizon of what we will teach…

> Yes, but no room is made for people who see no use for it. There is a forced-consensus that this technology is useful, which I have to combat against at work.

This is the crux of the issue -- The technology is useful. Using it appropriately is probably the thing that people are ignoring, but you're conflating one and the other in your comment.

It is not useful to you in this case, and complain that it is an overall detriment in your industry. Those are fine and reasonable statements and conditions, and I see no reason to disagree with them... But your first statement, people who see no use for it? That is, to me, as off-putting an opinion as the consequence-unaware hypebeasts who are running OpenClaw with access to their trading accounts and can't see why others aren't.

I sympathise with the idea that everyone wants to use the new hammer and so is treating every problem like a nail, but hammers are still pretty good tools. (And you can ignore the ex-NFT-fans hammering on their dicks in the corner.)

Re: The last six months in LLMs in five minutes

#369
post #42
post #22

Earlier quoted context omitted.

> So maybe the AI labs have been paying attention after all! > I think this mainly demonstrates that the pelican on the bicycle has firmly exceeded its limits as a useful benchmark. As acknowledged in the article.

Gemini 3.1 basically takes it home on that benchmark, anyway, it's done.

Gemini is heavily benchmaxxed and sucks in agentic coding so no surprise.

Re: The last six months in LLMs in five minutes

#370

Earlier quoted context omitted.

Feel free to open a bug report if it bothers you. Or a PR. Or feel free to avoid the tool entirely if this UI issue shakes your faith in its overall quality down to its very foundations. This is hardly a hill to die on.

You’re missing the point. You claimed high quality and provided a repo. Did you not expect someone to actually look and critique it? Whether the visual bugs are a deal breaker or not isn’t the point. The point is that’s not high quality code, it may work. But it’s not code I would ship at my job and therefore it’s not high enough quality for anyone serious

Hey that's fine. You're free to make whatever judgment you wish.

But I still stand by the quality of my code, including here. You and I don't need to agree.

What decades of managing codebases (public and private, huge and small) has taught me is that there will always be an endless list of bugs and feature ideas and nice-to-haves and technical debt pressures in any given project. You'll never get to them all, so you prioritize (as I have done here). Functional bugs usually trump visual ones unless they're actually interfering with work.

Will I fix this bug? Probably, now that I'm aware of it. But there are more important matters to attend to first.

Edit: Turns out the bug comes from a mismatch with the terminal I'm using. With other terminals it looks fine. Term caps are surprisingly complicated, especially when you have multiple layers!

Post reply on HN