Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Second Half

ysymyth.github.io

Post reply on HN
Print view View original

31–38 of 38 posts

Re: The Second Half

#31

Post by fullstackchris »

dceddia wrote: post #24
armchairhacker wrote: post #23

RL doesn't completely "work" yet, it still has a scalability problem. Claude can write a small project, but as it becomes larger, Claude gets confused and starts making mistakes. I used to think the problem was that models can't learn over time like humans, but maybe that can be worked around. Today's models have large enough context windows to fit a medium sized project's complete code and documentation, and tomorro…

I wonder how much of this is that code is less explicit than written language in some ways. With English, the meaning of a sentence is mostly self-contained. The words have inherent meaning, and if they’re not enough on their own, usually the surrounding sentences give enough context to infer the meaning. Usually you don’t have to go looking back 4 chapters or look in another book to figure out the implications of th…

This is super insightful, and I think there is at least part of what you are thinking of: an abstract syntax tree! Or at the very least one could include metadata about the token under scrutiny (similar to how most editors can show you git blame / number of references / number of tests passing in the current code you are looking at...)

It makes me think about things like... "what if we also provided not just the source code, but the abstract syntax tree or dependency graph", or at least the related nodes relevant to what code the LLM wants to change. In this way, you potentially have the true "full" context of the code, across all files / packages / whatever.

Re: The Second Half

#32

Post by fullstackchris »

daveguy wrote: post #30
jarbus wrote: post #12

I largely agree, and this is actually something I've been thinking for a while. The problem was never the algorithm; it's the game the algorithm is trying to solve. It's not clear to me what extent we can push this to aside from math, coding. Robotics should be ripe for this, though.

Unfortunately the feedback loop for robotics is many many orders of magnitude slower than math / coding problems. And when you get to artificial environments, you are learning artificial dynamics -- same limitations as the benchmarks.

Moravec's paradox

Re: The Second Half

#33

Post by dceddia »

debone wrote: post #26
dceddia wrote: post #24

Earlier quoted context omitted.

I wonder how much of this is that code is less explicit than written language in some ways. With English, the meaning of a sentence is mostly self-contained. The words have inherent meaning, and if they’re not enough on their own, usually the surrounding sentences give enough context to infer the meaning. Usually you don’t have to go looking back 4 chapters or look in another book to figure out the implications of th…

Not really true. You can have a book where in the last chapter you have a phrase "She was not his kid." Knowing nothing else, you can only infer the self-contained details. But in the book context this could be the phrase which turns everything upside down, and it could refer to a lot of context.

The whole book could be the surrounding context, not just a sentence or two, and I think that still fits with the point I wanted to make - that written words are more linear or in the same plane compared to code which is more "multidimensional" in a sense, when you start to consider the reasons behind the code, the order of execution, things being executed multiple times, etc.

Re: The Second Half

#34

Post by dceddia »

fullstackchris wrote: post #31
dceddia wrote: post #24

Earlier quoted context omitted.

I wonder how much of this is that code is less explicit than written language in some ways. With English, the meaning of a sentence is mostly self-contained. The words have inherent meaning, and if they’re not enough on their own, usually the surrounding sentences give enough context to infer the meaning. Usually you don’t have to go looking back 4 chapters or look in another book to figure out the implications of th…

This is super insightful, and I think there is at least part of what you are thinking of: an abstract syntax tree! Or at the very least one could include metadata about the token under scrutiny (similar to how most editors can show you git blame / number of references / number of tests passing in the current code you are looking at...) It makes me think about things like... "what if we also provided not just the sour…

Yeah! I think an AST is sort of what I'm envisioning here, but with much broader metadata, including requirements and implicit assumptions and stuff.

As a concrete example, a random bit of code from the minih264 encoder:

    /**
    *   Quantized/dequantized representation for 4x4 block
    */
    typedef struct
    {
        int16_t qv[16];     // quantized coefficient
        int16_t dq[16];     // dequantized
    } quant_t;
Someone who's built an encoder or studied h264 probably knows what this is for (I have a very fuzzy idea). But even with the comment there's lots of questions. Are these arrays restricted to certain values? Can they span the full int16, or are there limits, or are the bits packed in an interesting way? Can they be negative? Why would you want to store these 2 numbers together in a struct, why not separately? Do they get populated at the same time, or at different phases of the pipeline, or are they built up over multiple passes? Are all of these questions ridiculous because I don't really understand enough about how h264 works (probably)?

LLMs already have a lot of this knowledge, and could probably answer if prompted, but my point is more that the code doesn't explicitly lay out all of these things unless you carefully trace the execution, and even then, some of the requirements might not be evident. Maybe negative numbers aren't valid here (I don't actually know) but the reason that invariant gets upheld is an abs() call 6 levels up the call stack, or the data read from the file is always positive so we just don't have to worry about it. I dunno.

Anyway I imagine LLMs could be even more useful if they knew more about all this implicit context somehow, and I think this is the kind of stuff that just piles up as a codebase gets larger.

Re: The Second Half

#35

Post by [deleted] »

thetrustworthy wrote: post #21

For those who are knowledgeable about the field but not yet the author of this post, it is worth mentioning that Shunyu Yao has played a huge role in the development of LLM-based AI agents, including being an author / contributor to: - ReAct - Reflexion - SWE-bench - OpenAI Deep Research - OpenAI Operator

[deleted]

Re: The Second Half

#36

Post by conartist6 »

"solving" Dota is a huge huge HUGE overstatement of the kind you are pointing out.

The players it played against had never played against something that behaved so weirdly. It had lightning reflexes and it clearly wasn't human. It was playing a toy game mode requiring about 5% of skills needed for a full match. I'm other words, they engineered it to look good at they toy task, and it did. But they didn't give the pros any time at all to learn their opponent -- after all they might have figured out how to play against it!

Re: The Second Half

#37

Post by jebarker »

conartist6 wrote: post #36

"solving" Dota is a huge huge HUGE overstatement of the kind you are pointing out. The players it played against had never played against something that behaved so weirdly . It had lightning reflexes and it clearly wasn't human. It was playing a toy game mode requiring about 5% of skills needed for a full match. I'm other words, they engineered it to look good at they toy task, and it did. But they didn't give the pr…

Not arguing that they solved DOTA, but "The players it played against had never played against something that behaved so weirdly." seems like a feature, not a bug. We want AI to find unexpected new ways of accomplishing tasks.

Re: The Second Half

#38

Post by bionhoward »

armchairhacker wrote: post #23

RL doesn't completely "work" yet, it still has a scalability problem. Claude can write a small project, but as it becomes larger, Claude gets confused and starts making mistakes. I used to think the problem was that models can't learn over time like humans, but maybe that can be worked around. Today's models have large enough context windows to fit a medium sized project's complete code and documentation, and tomorro…

Claude and 4o aren’t RL trained IIRC? Also, who’s using these for code? You’re cool not being able to train on your chat logs used to develop your own codebase? Sounds pretty sus
Post reply on HN