Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning Programming in an Age of LLMs

blog.ploeh.dk

Post reply on HN
Print view View original

181–190 of 190 posts

Re: Learning Programming in an Age of LLMs

#181

Post by ActorNightly »

js8 wrote: post #58

I answered this to myself - stop worrying about LLMs. It's pretty simple: due to Curry-Howard isomorphism, programming languages are just notations for some type of formal logic. Now ask yourself a question, what language do you want to maintain the programs in? Do you think natural language is going to be easier and more maintainable than formal logic? The answer is no. So you need programmers, people who can read t…

Your analogy is poor, and you are missing a very important fact.

Most of the human written code, in places where that code needs to make money, is decidable either entirely or in large parts. I.e without running the code, you can take a domain of inputs and build a complete range of outputs solely by looking at the code.

The way that works in your head is that you are effectively doing a compilation to a logical like structure, which then you can use to infer what the output will be from what the input is, and its a direct mapping that is invertible and separable, so if you know what the output should be, you know what the input is, you can pinpoint the exact location where it breaks. Thats how humans write code.

If thats not clear, imagine a piece of code that splits strings by spaces, deletes the empty strings, and returns the number of words in a string. The fact that you can say that if you want 3 words, there should be maximum 2 sequences of continous spaces between words, is you effectively transpiling that program into a latent space inside your brain neurons and inverting it.

LLMs essentially do this, with the added advantage of having been trained on a HUGE number of codebases, so they can recognize patterns that a human cant.

Where LLMs struggle is complex behavior - they can't simulate things like a human can and choose the best course of action. Even harnesses for agentic loops that can auto run and debug code can't match what a human can do in this regard (hence why self driving still sucks rn).

So moving forward, being a good coder isn't going to be about writing code, or even about prompting LLMs. Its going to be all about whether or not you can design good custom agentic loops, which necessarily involves knowledge of the model at hand (i.e what words you have to use to get it to do the right thing). This will be especially true as investment into "private" inference grows where companies will be using smaller models that have less detailed RL and thus will need much more guidance to do the right thing.

Re: Learning Programming in an Age of LLMs

#183

Post by BatchJob »

My earnest, non-sarcastic advice to anyone considering learning to code is to find something else that interests you.

While there is possibly some time left before the software industry implodes, hobbyist software will also get steamrolled so I would in the strongest terms possible guide someone to find something that makes them happy and to pursue it.

The software industry as it exists today is a sad bag of regret and disappointment being papered over by opportunists of the worst kind.

Save yourself the trouble.

Re: Learning Programming in an Age of LLMs

#184

Post by jeremyjh »

js8 wrote: post #79
jeremyjh wrote: post #75

Earlier quoted context omitted.

LLMs do not execute natural language. Natural language describes a problem or request, and LLMs generate and test formal logic they predict will satisfy the request.

I disagree with each sentence for a different reason. LLMs interpret (so, "execute" in a way) natural language in the sense they have internal logic that assigns to the sequence of tokens in context a next token. If we delineate the input and output into a series of logical statements, we can think of it as a program that builds a logical statement from a list of input statements. So it encodes derivation in some log…

> LLMs interpret (so, "execute" in a way) natural language

You could say the same thing about human programmers, but I've never heard anyone say they think that programmers "execute" Jira tickets.

> they have internal logic that assigns to the sequence of tokens in context a next token.

I don't think this means what you think it means, because it has almost no information content relevant to what we're discussing. The probabilities that are most relevant at the level we're discussing are satisfying a reward function from post-training, which approximates to:

"What is the likelihood the solution the agent is pursuing will be marked correct by the automated grader based on the full prompt and other context provided?"

It still has to predict the next token but that isn't based on a likelihood of that token appearing in a corpus of internet text consumed in pretraining. That was eons ago. Every predicted token is shaped by the probabilities of the predicted solution, which must already be very specific and shaped completely by the request and associated context that is built during investigation of the same.

Re: Learning Programming in an Age of LLMs

#185

Post by jeremyjh »

layer8 wrote: post #142
jeremyjh wrote: post #75

Earlier quoted context omitted.

LLMs do not execute natural language. Natural language describes a problem or request, and LLMs generate and test formal logic they predict will satisfy the request.

When you want to change the resulting program (“formal logic”), you generally don’t change the prompt and regenerate everything (like you would do with source code). Instead, you provide new natural-language instructions to have the LLM perform the change. This means that you don’t have a reproducible specification on the level where you are describing the specification . Therefore the thing that you are maintaining…

> we need people that understand the thing that is being maintained

We do for a lot of software today, but not all of it. I think in a year we'll need them for less, but I'm not sure how much less.

This claim isn't supported or justified by talking about natural language versus formal language. For the last 50 years or so the people shaping a lot of software in the most important ways are often using only natural language.

The product manager doesn't understand the code today. They write PRDs and Jira tickets and comments in Slack, and software comes out. They have people test the software, report bugs, more software comes out. Eventually they decide its close enough to their vision to ship, without ever understanding or looking at any of the code. They can do that with human programmers or agent programmers. The former holds up better in larger systems, but I don't see any evidence for the proposition that this is due to the limits of natural language.

Re: Learning Programming in an Age of LLMs

#186

Post by Terr_ »

juvvel wrote: post #119
bbmatryoshka wrote: post #61

Earlier quoted context omitted.

non deterministic abstraction are absolutely useful, outside of software sector they have been used since the start of civilization ("a worker" is a very very non deterministic abstraction, outside from the most basic tasks)

> "a worker" is a very very non deterministic abstraction, outside from the most basic tasks Sure but we (usually) don't fool ourselves into believing otherwise. The problem with AI is that it suggests near-perfect intelligence with reproducible results. So many people are not acknowledging the fundamental limitations of LLMs, whereas we all have an intuition for the limitations of humans.

In addition to the reliability/security aspect, there's the claim to expertise and unwarranted confidence. We also don't (usually) fool ourselves into believing that the manager has gained the skills and expertise of the worker, simply because they gave abstract orders and a good result was achieved.

In other words, "I can code because I prompt an LLM" is like "I can do metalworking because I sign the paychecks of a blacksmith."

Re: Learning Programming in an Age of LLMs

#187

Post by pferde »

FuckButtons wrote: post #157
pferde wrote: post #146

Earlier quoted context omitted.

That's just it. you didn't "build so much", you didn't build anything. You asked someone (or something) to build it for you. How can people look at LLM-generated code and think "this is mine, I made this" is beyond me.

I’m in two minds about this, in some sense they did and it’s the same way in which the C code I write is mine, but the assembly underneath is an artifact of my intent, but I owe the compiler authors for it. On the other hand the cognitive distance between my C and the assembly is likely lower, but that’s because I’ve spent time staring at what was generated in order to figure out why my code was misbehaving. Which is…

If you ask LLM to code something for you, and you then work on the code, then yes, there is partial ownership. Just like if you had asked another, maybe junior, developer to write a piece of code, and then you helped them make it better.

Re: Learning Programming in an Age of LLMs

#188

Post by purpleflashing »

Finally a thread to traumadump my anxieties!

I have been trying to learn programming unsuccessfully for many years (I still ocasionally get a burst of energy, try something, then eventually abandon it halfway). In the end, I just accept that I don't have the kind of thinking that makes this effort enjoyable and I cannot achieve anything good.

But, to be fair, LLMs also don't solve any problems for me in any meaningful way. I have to do a little bit of very simple coding at work and I would say that I struggle with LLMs as much as I do when I try to solve them by myself. The code works in a moment but when it gets extended, it breaks, I spend a lot of time troubleshooting. Last week I had to rewrite the code myself as my attempt to prompt fixes with LLMs just kept adding bugs -- I can't say my rewrite is any good (I need to traverse graphs and I don't know how to do it properly so I just used whatever algorithm I found in a book lol), it will probably break the next week when I need to change it or add something but I hope at least I'll be able to fix it faster as I have roughly an idea what's happening.

I haven't measured how much time I spend on things with or without LLMs but the subjective experience is pretty poor -- I work in an "engineering-heavy" company, the constant feeling of not understanding my own work among people who have master things that are 10x more complex is pretty depressing. When I program something poorly, I can at least explain what I did and what my idea was, even it was a bad one, waking up to build failures and spending an entire day relearning my own project is not good, man.

As for the obvious argument -- "you're using it wrong" -- I might be? I did try to learn prompting and I don't think I do any obviously stupid stuff like specifying a persona. At this point I feel half-insane as a frequent sentiment is that non-programmers can generate code for functioning software with LLMs, perhaps it's low quality code but it works. Except for me -- a non-programmer who fumbles it every day! I don't even need anything complicated -- they're Python extensions for a Sphinx project!

Re: Learning Programming in an Age of LLMs

#189

Post by layer8 »

jeremyjh wrote: post #185
layer8 wrote: post #142

Earlier quoted context omitted.

When you want to change the resulting program (“formal logic”), you generally don’t change the prompt and regenerate everything (like you would do with source code). Instead, you provide new natural-language instructions to have the LLM perform the change. This means that you don’t have a reproducible specification on the level where you are describing the specification . Therefore the thing that you are maintaining…

> we need people that understand the thing that is being maintained We do for a lot of software today, but not all of it. I think in a year we'll need them for less, but I'm not sure how much less. This claim isn't supported or justified by talking about natural language versus formal language. For the last 50 years or so the people shaping a lot of software in the most important ways are often using only natural lan…

Developers are making a lot of important decisions in detail that the product manager has little understanding of, and these often lie exactly in the disparity between natural and formal language. What the product manager writes isn’t a full specification of the system.

Re: Learning Programming in an Age of LLMs

#190

Post by rgoulter »

japhyr wrote: post #5

I'm the author of Python Crash Course, and I got this exact same email this week. I was thinking of writing a public response as well, because any attempt to sincerely answer these questions takes something along the lines of a full post. It's also worth a public response because many people who are getting into programming for the first time right now are asking variations of these same questions. > Do I think that…

> Now, if you can steer an LLM reasonably well, you can quickly build an MVP that goes well beyond your own understanding of the implementation.

Pre-LLM, I'd distinguish between e.g. "I know Python" and "I know this codebase". So if I wrote a codebase in Python I'd be familiar with it, if someone else wrote a codebase in Python I'd be familiar with the Python. -- An LLM coding agent can give a codebase in Python very quickly.

With a newbie, they'd be familiar with neither; but an LLM coding agent can give them a full solution written in Python.

I'd say that this power from LLM coding agents blurs the distinction, in some sense. But to an extent it's always been the case that abstractions allow programming without a full understanding of everything down to atoms.

People 'can' learn faster than they used to. But I'd think those who are curious to learn will be able to have better results than those who only have a shallow understanding.

Post reply on HN