Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If you are good at code review, you will be good at using AI agents

seangoedecke.com

Post reply on HN
Print view View original

191–200 of 203 posts

Re: If you are good at code review, you will be good at using AI agents

#191

Post by QuantumNoodle »

jmcodes wrote: post #142
kiitos wrote: post #127

Earlier quoted context omitted.

the time spent literally typing code into an editor is never the bottleneck in any competently-run project if the act of writing code is something you consider a burden rather than a joy then my friend you are in the wrong profession

Been doing it for ten years still love the profession as much if not more than when I started, but the joy of software development for me was always in seeing my idea come to life, in exploring all the clever ways people had solved so many problems, in trying to become as good at the craft as they were, and in sharing those solutions and ideas with like-minded peers. I care deeply about the code quality that goes int…

While reading your comment it occured to me that people code at different abstraction levels. I do systems programming in golang and rust and I - like you - enjoy seeing my ideas come to life not so much the typing. The final result (how performant, how correct, how elegant and not complex) is in my control instead of an agent's; I enjoy having the creativity in the implementation. I can imagine other flavors of the profession working at higher abstraction layers and using more frameworks, where their result is dependant on how the framework executes. At that point, you might just want to connect all the frameworks/systems and get the feature out the door. And it is definitely a spectrum of languages, tools, frameworks that are more or less involved.

The creativity in implementing (e.g an indexed array that, when it grows to large, gets reformated to a less performance hashmap) is what I imagine being lost and bring people satisfaction. Pulling that off in a clean and not in a complex way... well there is a certain reward in that. I don't have any long term proof but I also hypothesize it helps with maintainability.

But I also see your point, sometimes I need a tool that does a function and I don't care to write it and giving the agent requirements and having it implemented is enough. But typically these tools are used and discarded.

Re: If you are good at code review, you will be good at using AI agents

#192

Post by egberts1 »

Asking AI to stay true to my requested parameters is hard, THEY ALL DRIFT AWAY, RANDOMLY

When working on nftables syntax highlighters, I have 230 tokens, 2,500 state, and 50,000+ state transitions.

Some firm guidelines given to AI agents are:

1. Fully-deterministic LL(1) full syntax tree.

2. No use of Vim 'syntax keyword' statement

3. Use long group names in snake_case whose naming starts with 'nft_' prefix (avoids collision with other Vim namespaces)

4. For parts of the group names, use only nftables/src/parser_bison.y semantic action and token names as-is.

5. For each traversal down the syntax tree, append that non-terminal node name from parser_bison.y to its group names before using it.

With those 5 "simple" user-requested requirements, all AI agents drift away from at least each of the rules at seemingly random interval.

At the moment, it is dubiois to even trust the bit-length of each packet field.

Never mind their inability to construct a simple Vimscript.

I use AI agents mainly as documentation.

On the bright side, they are getting good at breaking down 'rule', 'chain_block stmt', and 'map_stmt_expr' (that '.' period we see at chaining header expressions together; just use the quoted words and paste in one of your nft rule statements.

Re: If you are good at code review, you will be good at using AI agents

#193

Post by krageon »

EdwardDiego wrote: post #145
Manfred wrote: post #48

Earlier quoted context omitted.

Reviewing code from less experienced or unmotivated people is also very taxing, both in a cognitive and emotional sense. It will never approach a really good level of quality because you just give up after 4 rounds of reviews on the same feature.

Except humans learn from your PR comments and in other interactions with more experienced people, and so inexperienced devs become experienced devs eventually. LLMs are not so trainable.

If they're unmotivated enough to not get there after four review rounds for a junior-appropriate feature, they're not going to get better. It's a little impolite to say, but if you spend any significant amount of time coaching juniors you'll encounter exactly what I'm talking about.

Re: If you are good at code review, you will be good at using AI agents

#194

Post by jmcodes »

QuantumNoodle wrote: post #191
jmcodes wrote: post #142

Earlier quoted context omitted.

Been doing it for ten years still love the profession as much if not more than when I started, but the joy of software development for me was always in seeing my idea come to life, in exploring all the clever ways people had solved so many problems, in trying to become as good at the craft as they were, and in sharing those solutions and ideas with like-minded peers. I care deeply about the code quality that goes int…

While reading your comment it occured to me that people code at different abstraction levels. I do systems programming in golang and rust and I - like you - enjoy seeing my ideas come to life not so much the typing. The final result (how performant, how correct, how elegant and not complex) is in my control instead of an agent's; I enjoy having the creativity in the implementation. I can imagine other flavors of the…

Agreed 100% and I enjoy that part too, I just don't really see how that is being taken away.

The way I see it these tools allow me to use my actual brainpower mostly on those problems. Because all the rote work can now be workably augmented away, I can choose which problems to actually focus on "by hand" as it were. I'd never give those problems to an LLM to solve. I might however ask it to search the web for papers or articles or what have you that have solved similar problems and go from there.

If someone is giving that up then I'd question why they're doing that.. No one is forcing them to.

It's the problem solving itself that is fun, the "layer" that it's in doesn't really make a difference to me.

Re: If you are good at code review, you will be good at using AI agents

#195

Post by legacynl »

egberts1 wrote: post #192

Asking AI to stay true to my requested parameters is hard, THEY ALL DRIFT AWAY, RANDOMLY When working on nftables syntax highlighters, I have 230 tokens, 2,500 state, and 50,000+ state transitions. Some firm guidelines given to AI agents are: 1. Fully-deterministic LL(1) full syntax tree. 2. No use of Vim 'syntax keyword' statement 3. Use long group names in snake_case whose naming starts with 'nft_' prefix (avoids c…

Im a dev (who knows nothing about nftables) and I don't understand your instructions. I think maybe you could improve your situation by formulating them as "when creating new groupnames use the semantic actions and token names as defined in parser_bison.y" I.e. with if conditions so that the correct rules apply to the correct situations. Because your rules are written as if to apply to every line of code, it might unnecessarily try to incorporate context even when it's not applicable.

Re: If you are good at code review, you will be good at using AI agents

#196

Post by legacynl »

jmcodes wrote: post #55
QuantumNoodle wrote: post #10

Code review is part of the job, but one of the least enjoyable parts. Developers like _writing_ and that gives the most job satisfaction. AI tools are helpful, but inherently increases the amount of code we have to review with more scrutiny than my colleagues because of how unpredictable - yet convincing - it can be. Why did we create tools that do the fun part and increase the non-fun part? Where are the "code-revie…

Maybe I'm weird but I don't actually enjoy the act of _writing_ code. I enjoy problem solving and creating something. I enjoy decomposing systems and putting them back together in a better state, but actually manually typing out code isn't something I enjoy. When I use an LLM to code I feel like I can go from idea to something I can work with in much less time than I would have normally. Our codebase is more type-saf…

If natural language was an efficient way to write software we would have done it already. Fact is that it's faster to write class X { etc }; Than it is to write "create a class named X with behavior etc". If you want to think and solve problems yourself, it doesn't make sense to then increase your workload by putting your thoughts in natural language, which will be more verbose.

I therefore think it makes the most sense to just feed it requirements and issues, and telling it to provide a solution.

Also unless you're starting a new project or big feature with a lot of boiler plate, in my experience it's almost never necessary to make a lot of files with a lot of text in it at once.

Re: If you are good at code review, you will be good at using AI agents

#197

Post by keeda »

dingnuts wrote: post #117
keeda wrote: post #115

Earlier quoted context omitted.

1. The flaw in this premise is that the process is bad. Aside from the countless anecdotal reports about how AI and agents are improving productivity, there are actual studies showing 25 - 55% boosts. Yes, RCTs at larger size than the METR one that keeps getting bandied about: https://news.ycombinator.com/item?id=44860577 and many more on Google Scholar: https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&as_ylo...…

I picked one of the studies in the search (!) you linked. First of all, it's a bullshit debate tactic to try to overwhelm your opponents with vague studies -- a search is complete bullshit because it puts the onus on the other person to discredit the gargantuan amount of data you've flooded them with. Many of the studies in that search don't have anything to do with programming at all. So right off the bat, I don't t…

> a search is complete bullshit because it puts the onus on the other person to discredit the gargantuan amount of data you've flooded them with.

Alternatively, a search is a way to show that basing your opinions on limited personal experience or a single source is silly given the vast amount of other research out there that largely contradicts it. Worse if that single source itself happens to have flaws that are not sufficiently discussed, e.g. at least one of the 16 participants from the METR study deliberately filtered out large tasks that he strongly prefered do only with AI -- what does that mean for its results?

https://xcancel.com/ruben_bloom/status/1943536052037390531

> Many of the studies in that search don't have anything to do with programming at all.

That's fair, but unfortunately due to the limits of keyword search. For instance "medical coding" is not programming-related at all, but is being impacted by LLMs and gets caught in the keyword search ¯\_ (ツ)_/¯

Anyway if your preference is "real-world projects", here are a couple specific studies (including one that I had already separately mentioned in the linked comment) at much larger scales that show significant productivity boosts of LLM-assisted programming at doing their regular, day-job tasks:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4945566 (4867 developers across 3 large companies including Microsoft)

https://www.bis.org/publ/work1208.pdf (1219 programmers at a Chinese BigTech)

There are many more, but I left them out as they are based on other methodologies such as the use of standardized tasks for better comparability, or empirical analysis of open-source commits, or developer surveys, or student projects, which may get dismissed as "not an RCT on real-world tasks." Interestingly they all show comparable, positive results, so consider that it's not as straightforward to dismiss other studies as being irrelevant.

Re: If you are good at code review, you will be good at using AI agents

#198

Post by EdwardDiego »

300hoogen wrote: post #188
EdwardDiego wrote: post #145

Earlier quoted context omitted.

Except humans learn from your PR comments and in other interactions with more experienced people, and so inexperienced devs become experienced devs eventually. LLMs are not so trainable.

retarded take

Thanks for the insightful reply that showed me where I went astray.

Re: If you are good at code review, you will be good at using AI agents

#199

Post by EdwardDiego »

krageon wrote: post #193
EdwardDiego wrote: post #145

Earlier quoted context omitted.

Except humans learn from your PR comments and in other interactions with more experienced people, and so inexperienced devs become experienced devs eventually. LLMs are not so trainable.

If they're unmotivated enough to not get there after four review rounds for a junior-appropriate feature, they're not going to get better. It's a little impolite to say, but if you spend any significant amount of time coaching juniors you'll encounter exactly what I'm talking about.

I have spent plenty, rest assured.

Re: If you are good at code review, you will be good at using AI agents

#200

Post by kiitos »

theshrike79 wrote: post #177
kiitos wrote: post #127

Earlier quoted context omitted.

the time spent literally typing code into an editor is never the bottleneck in any competently-run project if the act of writing code is something you consider a burden rather than a joy then my friend you are in the wrong profession

But it's not exactly rewarding to add one more CRUD endpoint. It's a shit-ton of typing in multiple layers. An LLM can do it in two minutes while I fetch coffee, then I can proceed to add the complex bits (if there are any)

> But it's not exactly rewarding to add one more CRUD endpoint. It's a shit-ton of typing in multiple layers.

i don't disagree with you but if "adding one more CRUD endpoint" and similar rote tasks represent any significant amount of your engineering hours, especially in the context of business impact, then something is fundamentally broken in your team, engineering org, or company overall

time spent typing code into an editor is usually, hopefully!, approximately statistically 0% of overall engineering time

Post reply on HN