Is that really true? I would have expected by now that AI companies nowadays are doing RL on git histories , not just on the HEAD.
I also expected this. Please run some experiments and maybe other models are different
LLMs learn what programmers create, not how programmers work
11–20 of 31 posts
Re: LLMs learn what programmers create, not how programmers work
#12The novice came to the master. "I have figured it out, the rules for how LLMs understand CLIs. It gives the right commands, but adds colons. It was trained on the visual shape of terminals, not keystrokes." "Clear the session," the master said. "Run the same prompt again." The novice pressed return. The model output: `ls -R /tmp` "The colons are gone," the novice said. "But my theory explained them perfectly." "You b…
I ran tests of 100 attempts with different prompt/scenario combinations. Each "attempt"/theory had 3 different system prompts wordings. Most of the prompts did not mention a colon, but it kept appearing. When I added negative instructions against using a colon, the quality went down (most of the tool calls were malformed, one common issue was markdown ticks in front)
It was only when my system prompt acted like colons were normal that I kept getting 100/100 perfect expected tool calls. I ranked my system prompts by which returned the most consistent commands.
Re: LLMs learn what programmers create, not how programmers work
#13I know even we hit the same thing building internal security tooling. our model kept formatting output like documentation, not like how we would or any person in place of us would read in a terminal at 2am during an incident. I am a bit curious, did you find this behavior consistent across models or is it more pronounced with certain ones?
I ran into it while building - I should have tested different temps too - I was just trying to get cli style tool calls to be more reliable
Re: LLMs learn what programmers create, not how programmers work
#14Is that really true? I would have expected by now that AI companies nowadays are doing RL on git histories , not just on the HEAD.
Claude definitely does
Re: LLMs learn what programmers create, not how programmers work
#15I know even we hit the same thing building internal security tooling. our model kept formatting output like documentation, not like how we would or any person in place of us would read in a terminal at 2am during an incident. I am a bit curious, did you find this behavior consistent across models or is it more pronounced with certain ones?
Literate programming is about to become mainstream in the funniest way possible.
Re: LLMs learn what programmers create, not how programmers work
#16[dead]
Re: LLMs learn what programmers create, not how programmers work
#17Great observation. The brain of a programmer is still a "black box" to the feed-forward network of nodes . But in theory, if you pumped a lot of the live-coding videos from something like youtube into the process, you could get a bit of that "what's your approach"-erism to bleed into the model. There might not be enough material there to truly "train it to think" but it would be interesting to try and "fill the gaps"…
[dead]
Re: LLMs learn what programmers create, not how programmers work
#18[dead]
Re: LLMs learn what programmers create, not how programmers work
#19not just for coding, for any profession, LLMs only learned what people created but not how professionals work.
Re: LLMs learn what programmers create, not how programmers work
#20Is that really true? I would have expected by now that AI companies nowadays are doing RL on git histories , not just on the HEAD.
[flagged]