Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is understanding code becoming "optional"?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 42 posts

Re: Ask HN: Is understanding code becoming "optional"?

#31

Post by taurath »

dapangzi wrote: post #30
taurath wrote: post #28

Earlier quoted context omitted.

Just as with an LLM, a detailed style and format guide helps an incredible amount both for the LLMs and juniors. If you have standards and they’re not written down, you either require everyone to go teach them to anyone new, or you don’t have standards.

> you don’t have standards. The problem is that LLM mess up things as basic as math and dates, and that's before the context gets too large and it starts making other mistakes. Edit: Also LLM over mock tests and juniors trust that...

Not very often, and most of the time it shouldn't be generating those but rather formatting code to test that. If you accept the non-determinism and use some of the more recent models, you'll find it can do 99% of it very fast, and with some guardrails and testing it can fairly reliably produce working solutions.

Re: Ask HN: Is understanding code becoming "optional"?

#32

Post by dapangzi »

taurath wrote: post #31
dapangzi wrote: post #30

Earlier quoted context omitted.

> you don’t have standards. The problem is that LLM mess up things as basic as math and dates, and that's before the context gets too large and it starts making other mistakes. Edit: Also LLM over mock tests and juniors trust that...

Not very often, and most of the time it shouldn't be generating those but rather formatting code to test that. If you accept the non-determinism and use some of the more recent models, you'll find it can do 99% of it very fast, and with some guardrails and testing it can fairly reliably produce working solutions.

> Not very often

> testing

This does not match my experience, have been working with LLM since 2023. We presently use the latest models, I assure you. We can definitely afford it.

I am not saying LLM is worthless, but being able to check its outputs is still necessary at this stage, because as you said, it is non-deterministic.

We have had multiple customer impacting events from code juniors committed without understanding it. Please read my top level comment in this post for context.

I genuinely hope you do not encounter issues due to your confidence in LLM, but again, my experience does not match yours.

Edit: Would also add that LLM is not good at determining line numbers in a code file, another flaw that causes a lot of confusion.

Re: Ask HN: Is understanding code becoming "optional"?

#33

Post by al_borland »

It’s important to remember these people you mention who work for Anthropic have a vested interest in selling Claude Code to the world. They are not an impartial third-party, so I would take anything they say with a grain of salt.

> Less code because code equals responsibility.

This is true. The problem with AI is that while someone may personally write less code, they are still responsible for it and have to answer questions about the minutiae of what it does. One of my least favorite things is being responsible for, or having to answer for, work that isn’t mine. I’m not sure why I’d willing make that my whole job.

Re: Ask HN: Is understanding code becoming "optional"?

#34

Post by tstrimple »

I've used Claude Code a lot over the last year and I've generally been very happy with it. I have a lot of experience writing code both professionally and for personal projects. I've found that for things like basic APIs and websites and database operations, I don't have to pay attention to the code being produced much at all anymore. It Just Works for the most part as long as you adequately describe what you're trying to build. There are only so many ways you can write a CRUD app after all, and generally the implementation isn't "special" just necessary.

But my experience on 3d game dev in particular has been quite different. I've been able to get good results for basic 2d games and basic features in 3d worlds, but have been struggling to build more complicated scenarios with Claude Code without laying out every specific detail. I have to tell it to use quaternions for a particular rotation because I know about issues with gimble lock. I have to suggest a ray traced solution in another area because relative mouse position isn't good enough when accounting for resolution and aspect ratio. If I didn't know about ray tracing or quaternions and how they are used and fit into game development I wouldn't have been able to interrupt Claude Code and guide it down a better path. I think claude code is particularly weak in spatial reasoning and I suspect the context required for some GPU operations are pushing other parts of instructions out of context. It's forgetting "the basics" far more than I've experienced in any other project. Building a 3d world simulation featuring a bastardization of plate tectonics and weather systems is the first thing I've tried to do with Claude Code that I could have probably written myself faster. If it wasn't for the crippling adhd.

Re: Ask HN: Is understanding code becoming "optional"?

#35

Post by taurath »

dapangzi wrote: post #32
taurath wrote: post #31

Earlier quoted context omitted.

Not very often, and most of the time it shouldn't be generating those but rather formatting code to test that. If you accept the non-determinism and use some of the more recent models, you'll find it can do 99% of it very fast, and with some guardrails and testing it can fairly reliably produce working solutions.

> Not very often > testing This does not match my experience, have been working with LLM since 2023. We presently use the latest models, I assure you. We can definitely afford it. I am not saying LLM is worthless, but being able to check its outputs is still necessary at this stage, because as you said, it is non-deterministic. We have had multiple customer impacting events from code juniors committed without underst…

I haven’t run into that problem but I do also hold agents on a tight leash!

Re: Ask HN: Is understanding code becoming "optional"?

#36

Post by giantg2 »

raw_anon_1111 wrote: post #26
giantg2 wrote: post #25

Earlier quoted context omitted.

Compilers have a finite set of inputs and outputs that should generate reproducible results. There's a larger amount of possible outputs for the same question with AI and very little reproducbility.

Yes but once the code is written it’s not going to magically change. I am going to test the code just like I would test something I wrote - again like I’ve been doing for 40 years when writing my code by hand.

But your thought process during coding influences your testing. At least for most of us, we find edge cases or point of concern during coding that we place extra focus on in test.

This is different than what you've done for the past 40 years becuase you're not testing your code. This would be analogous to you testing someone else's code. The vast majority of people and places have not followed that paradigm until AI showed up.

Re: Ask HN: Is understanding code becoming "optional"?

#37

Post by giantg2 »

dapangzi wrote: post #30
taurath wrote: post #28

Earlier quoted context omitted.

Just as with an LLM, a detailed style and format guide helps an incredible amount both for the LLMs and juniors. If you have standards and they’re not written down, you either require everyone to go teach them to anyone new, or you don’t have standards.

> you don’t have standards. The problem is that LLM mess up things as basic as math and dates, and that's before the context gets too large and it starts making other mistakes. Edit: Also LLM over mock tests and juniors trust that...

I've never had an LLM create a robust, meaningful test file. I end up rewriting at least half of it.

Re: Ask HN: Is understanding code becoming "optional"?

#38

Post by giantg2 »

dapangzi wrote: post #32
taurath wrote: post #31

Earlier quoted context omitted.

Not very often, and most of the time it shouldn't be generating those but rather formatting code to test that. If you accept the non-determinism and use some of the more recent models, you'll find it can do 99% of it very fast, and with some guardrails and testing it can fairly reliably produce working solutions.

> Not very often > testing This does not match my experience, have been working with LLM since 2023. We presently use the latest models, I assure you. We can definitely afford it. I am not saying LLM is worthless, but being able to check its outputs is still necessary at this stage, because as you said, it is non-deterministic. We have had multiple customer impacting events from code juniors committed without underst…

I had a mid-level submit a PR implementing caching. I had to reject it multiple times. They were using Copilot and it couldn't implement it right and the developer couldn't understand it. Stuff like always retrieving from the API instead of the cache, or never storing the object in the cache after retrieving it.

They promoted that guy over me because he started closing more stories than me and faster after he started using Copilot. No wonder that team has 40% of its capacity used for rework and tech debt...

Re: Ask HN: Is understanding code becoming "optional"?

#39

Post by raw_anon_1111 »

giantg2 wrote: post #36
raw_anon_1111 wrote: post #26

Earlier quoted context omitted.

Yes but once the code is written it’s not going to magically change. I am going to test the code just like I would test something I wrote - again like I’ve been doing for 40 years when writing my code by hand.

But your thought process during coding influences your testing. At least for most of us, we find edge cases or point of concern during coding that we place extra focus on in test. This is different than what you've done for the past 40 years becuase you're not testing your code. This would be analogous to you testing someone else's code. The vast majority of people and places have not followed that paradigm until AI…

My thought process during my architecture influences my testing.

Since AI has been a thing, I’ve been in a customer facing cloud consulting role - working full time at consulting departments (AWS ProServe) and now a third party company - specializing in app dev.

Before my hands actually write a line of code or infrastructure as code, I’ve already spoken to sales to get a high level idea of what the customer wants, read over the contract (SoW) to see what questions I have, done discovery sessions/requirements analysis, created architecture diagrams, done a design review, created detailed stories/workstreams (epics), thought about all the way things can go wrong etc.

I very much keep my hands on the wheel and treat AI as a junior coder that might not follow my instructions. I can answer any question about architectural decisions, repo structure, what any Lambda does the naming conventions etc.

I’ve also intuited “these are the things that I need to think about and test for from my 30 years of professional experience as a developer and 8 years of experience across literally dozens of AWS implementations”.

In the before times, if I were doing this without AI, I would have to have two or three more junior people doing the work just because I couldn’t physically do it in 40 hours a week. Even then I would be focused on how it works and look for corner cases.

I don’t have to think about what I need to test for. I did specifically call out concurrency because there are subtle bugs.

Ironically, what I am working on now had a subtle concurrent locking bug that Codex wrote. I threw the code into ChatGPT thinking mode and it found it immediately and suggested better alternatives. I also have Claude and Codex cross check each other.

Re: Ask HN: Is understanding code becoming "optional"?

#40

Post by giantg2 »

raw_anon_1111 wrote: post #39
giantg2 wrote: post #36

Earlier quoted context omitted.

But your thought process during coding influences your testing. At least for most of us, we find edge cases or point of concern during coding that we place extra focus on in test. This is different than what you've done for the past 40 years becuase you're not testing your code. This would be analogous to you testing someone else's code. The vast majority of people and places have not followed that paradigm until AI…

My thought process during my architecture influences my testing. Since AI has been a thing, I’ve been in a customer facing cloud consulting role - working full time at consulting departments (AWS ProServe) and now a third party company - specializing in app dev. Before my hands actually write a line of code or infrastructure as code, I’ve already spoken to sales to get a high level idea of what the customer wants, re…

"I don’t have to think about what I need to test for."

Good luck then. The business process flow including edge cases should arguably be top of mind for what to test. Testing shouldn't be an afterthought but rather an integral thought when writing the code that needs to be tested.

"I would have to have two or three more junior people doing the work"

Yeah, and they're the ones thinking about testing the code they write. Architects (which it sounds like you are an architect and not a dev) don't get into thay much detail.

Post reply on HN