Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is understanding code becoming "optional"?

news.ycombinator.com

Post reply on HN
Print view View original

21–30 of 42 posts

Re: Ask HN: Is understanding code becoming "optional"?

#21

Post by mikaelaast »

adamzwasserman wrote: post #20
mikaelaast wrote: post #15

Earlier quoted context omitted.

How do you verify the code without actually looking at it?

Although I write very little code myself anymore, I don't trust AI code at all. My default assumption: every line is the most mid possible implementation, every important architecture constraint violated wantonly. Your typical junior programmer. So I run specialized compliance agents regularly. I watch the AI code and interrupt frequently to put it back on track. I occasionally write snippets as few-shot examples. Ve…

I like this. The few-shot example snippet method is something I’d like to incorporate in my workflow, to better align generated code with my preferences.

Re: Ask HN: Is understanding code becoming "optional"?

#23

Post by adamzwasserman »

mikaelaast wrote: post #21
adamzwasserman wrote: post #20

Earlier quoted context omitted.

Although I write very little code myself anymore, I don't trust AI code at all. My default assumption: every line is the most mid possible implementation, every important architecture constraint violated wantonly. Your typical junior programmer. So I run specialized compliance agents regularly. I watch the AI code and interrupt frequently to put it back on track. I occasionally write snippets as few-shot examples. Ve…

I like this. The few-shot example snippet method is something I’d like to incorporate in my workflow, to better align generated code with my preferences.

I have written a research paper on another interesting prompting technique that I call axiomatic prompting. On objectively measurable tasks, when an AI scores below 70%, including clear axioms in the prompt systematically increases success.

In coding this would convert to: when trying to impose a pattern or architecture that is different enough from the "mid" programming approach that the AI is compelled to use, including axioms about the approach (in a IF this THEN than style, as opposed to few shot examples) will improve success.

The key is the 70% threshold: if the model already has enough training data, axioms hurt. If the model is underperforming because the training set did -not- have enough examples (for example hyperscript), axioms helps.

Re: Ask HN: Is understanding code becoming "optional"?

#24

Post by giantg2 »

If anything, you have to understand code more now.

Before you (or your devs) could write code a couple different ways and understand it. Now you have to look a code generated by an agent that is not necessary writing code in the same way as the culture at your company. There might be a thousand different ways a feature gets written. You have to spend more time reviewing and thinking it about it in my opinion.

Re: Ask HN: Is understanding code becoming "optional"?

#25

Post by giantg2 »

raw_anon_1111 wrote: post #17
mikaelaast wrote: post #15

Earlier quoted context omitted.

How do you verify the code without actually looking at it?

How do you verify the compiler without looking at the assembled code? How do you verify code that links against binary libraries? You run it and check for your desired behavior.

Compilers have a finite set of inputs and outputs that should generate reproducible results. There's a larger amount of possible outputs for the same question with AI and very little reproducbility.

Re: Ask HN: Is understanding code becoming "optional"?

#26

Post by raw_anon_1111 »

giantg2 wrote: post #25
raw_anon_1111 wrote: post #17

Earlier quoted context omitted.

How do you verify the compiler without looking at the assembled code? How do you verify code that links against binary libraries? You run it and check for your desired behavior.

Compilers have a finite set of inputs and outputs that should generate reproducible results. There's a larger amount of possible outputs for the same question with AI and very little reproducbility.

Yes but once the code is written it’s not going to magically change. I am going to test the code just like I would test something I wrote - again like I’ve been doing for 40 years when writing my code by hand.

Re: Ask HN: Is understanding code becoming "optional"?

#27

Post by dapangzi »

giantg2 wrote: post #24

If anything, you have to understand code more now. Before you (or your devs) could write code a couple different ways and understand it. Now you have to look a code generated by an agent that is not necessary writing code in the same way as the culture at your company. There might be a thousand different ways a feature gets written. You have to spend more time reviewing and thinking it about it in my opinion.

Made a similar comment.

It's great for tenured engineers, when we use it.

When juniors use LLM, because they don't have experience, it becomes a nightmare for tenured engineers, and we just end up "mopping the slop", as I tend to say.

I also have issue with how LLM do testing.

Re: Ask HN: Is understanding code becoming "optional"?

#28

Post by taurath »

dapangzi wrote: post #27
giantg2 wrote: post #24

If anything, you have to understand code more now. Before you (or your devs) could write code a couple different ways and understand it. Now you have to look a code generated by an agent that is not necessary writing code in the same way as the culture at your company. There might be a thousand different ways a feature gets written. You have to spend more time reviewing and thinking it about it in my opinion.

Made a similar comment. It's great for tenured engineers, when we use it . When juniors use LLM, because they don't have experience, it becomes a nightmare for tenured engineers, and we just end up "mopping the slop", as I tend to say. I also have issue with how LLM do testing.

Just as with an LLM, a detailed style and format guide helps an incredible amount both for the LLMs and juniors. If you have standards and they’re not written down, you either require everyone to go teach them to anyone new, or you don’t have standards.

Re: Ask HN: Is understanding code becoming "optional"?

#29

Post by moomoo11 »

mikaelaast wrote: post #15
raw_anon_1111 wrote: post #12

When I first started coding, I knew how my code worked down to assembly language because that was the only way I could get anything to run at a sufficient speed on a 1Mhz computer, I then graduated to C and C++ with some VB and then C#, JavaScript and Python Back in 2000 I knew every server and network switch in our office and eventually our self hosted server room with a SAN and a whopping 3TB of RAM before I left.…

How do you verify the code without actually looking at it?

"Let's check that we can do X, Y, Z"

"Create documentation and then write tests"

a few moments later...

"There's a bug where we cannot do Y. Investigate the code and then let's discuss the best fix"

"Update the documentation and tests"

Re: Ask HN: Is understanding code becoming "optional"?

#30

Post by dapangzi »

taurath wrote: post #28
dapangzi wrote: post #27

Earlier quoted context omitted.

Made a similar comment. It's great for tenured engineers, when we use it . When juniors use LLM, because they don't have experience, it becomes a nightmare for tenured engineers, and we just end up "mopping the slop", as I tend to say. I also have issue with how LLM do testing.

Just as with an LLM, a detailed style and format guide helps an incredible amount both for the LLMs and juniors. If you have standards and they’re not written down, you either require everyone to go teach them to anyone new, or you don’t have standards.

> you don’t have standards.

The problem is that LLM mess up things as basic as math and dates, and that's before the context gets too large and it starts making other mistakes.

Edit: Also LLM over mock tests and juniors trust that...

Post reply on HN