Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The highest quality codebase

gricha.dev

Post reply on HN
Print view View original

361–370 of 409 posts

Re: The highest quality codebase

#361

Post by urxvtcd »

pdntspa wrote: post #43
xnorswap wrote: post #18

Claude is really good at specific analysis, but really terrible at open-ended problems. "Hey claude, I get this error message: ", and it'll often find the root cause quicker than I could. "Hey claude, anything I could do to improve Y?", and it'll struggle beyond the basics that a linter might suggest. It suggested enthusiastically a library for and it was all " Recommended " about it, but when I pointed out that the…

That's why you treat it like a junior dev. You do the fun stuff of supervising the product, overseeing design and implementation, breaking up the work, and reviewing the outputs. It does the boring stuff of actually writing the code. I am phenomenally productive this way, I am happier at my job, and its quality of work is extremely high as long as I occasionally have it stop and self-review it's progress against the…

Few weeks ago I'd disagree with you, but recently I've been struggling with concentration and motivation and now I kind of try to embrace coding with AI. I guide it pretty strictly, try to stick with pure functions, and always read the output thoroughly. In a couple of places requiring some carefulness I coded them in executable pseudocode (Python) and made AI translate it to the more boilerplate-y target language.

I don't know if I'm any faster than I would be if I was motivated, but I'm A LOT more productive in my current state. I still hope for the next AI winter though.

Re: The highest quality codebase

#362

Post by [deleted] »

skydhash wrote: post #360
Dylan16807 wrote: post #357

Earlier quoted context omitted.

Context. That wasn't the original question. That's a short restatement of the real question which is up in an earlier post: >> Such questions may be decided by personal preferences, but their impact can easily be demonstrated. > I really don't think this is true. What was the demonstrated impact of writing Terraform in Go rather than Rust? Would writing Terraform in Rust have resulted in a better product? Would rewri…

And I’ve already answered that question. One of the main impact is that if you want a contributor to the codebase, the person have to learn Go. Even if they have good knowledge of the domain and are proficient in Rust. There would be some cost associated to training that person in Go (it may be small). Rewriting from Go to another language wouldn’t solve the problem better. Because Go is an implementation choice, not…

[deleted]

Re: The highest quality codebase

#363

Post by Dylan16807 »

skydhash wrote: post #360
Dylan16807 wrote: post #357

Earlier quoted context omitted.

Context. That wasn't the original question. That's a short restatement of the real question which is up in an earlier post: >> Such questions may be decided by personal preferences, but their impact can easily be demonstrated. > I really don't think this is true. What was the demonstrated impact of writing Terraform in Go rather than Rust? Would writing Terraform in Rust have resulted in a better product? Would rewri…

And I’ve already answered that question. One of the main impact is that if you want a contributor to the codebase, the person have to learn Go. Even if they have good knowledge of the domain and are proficient in Rust. There would be some cost associated to training that person in Go (it may be small). Rewriting from Go to another language wouldn’t solve the problem better. Because Go is an implementation choice, not…

> And I’ve already answered that question.

You somewhat answered it in a way that doesn't really get to why they asked it (you can't make every decision based on "demonstrated impact").

But you did that in a different comment than the one I replied to. The one I replied to was just answering the wrong question entirely. Which is why I replied.

> Rewriting from Go to another language wouldn’t solve the problem better. Because Go is an implementation choice, not a design choice. There’s nothing in Go that make Terraform better. It could be in C and a lot of people wouldn’t notice.

I'm sorry, are you arguing that using feel to decide how to structure a piece of code is a "huge red flag", but the choice of entire programming language is unimportant?

Re: The highest quality codebase

#364

Post by Dilettante_ »

mainmailman wrote: post #57
hazmazlaz wrote: post #48

Well of course it produced bad results... it was given a bad prompt. Imagine how things would have turned out if you had given the same instructions to a skilled but naive contractor who contractually couldn't say no and couldn't question you. Probably pretty similar.

Yeah I don't see the utility in doing this hundreds of times back to back. A few iterations can tell us some things about how Claude optimizes code, but an open ended prompt to endlessly "improve" the code sounds like a bad boss making huge demands. I don't blame the AI for adding BS down the line.

I don't think the question "will the AI add BS" was what drove this experiment. The very first thing the author references is re-feeding and degrading the same image 100 times, which similarly is not about improving the image.

This was more about seeing in what interesting ways the LLM will "fail", to get a little glimpse into how the black-box "thinks".

Re: The highest quality codebase

#365

Post by Dylan16807 »

agumonkey wrote: post #352
Dylan16807 wrote: post #335

Earlier quoted context omitted.

If you produce the same product, then you get to ask for the same pay. That's not a scam. If enough people can make the product faster, then competition will drive the price down. But the ability to charge less is not at all an obligation to charge less.

it's paradoxical, the llm is not helping consumers, it's not helping the experienced engineer, it's helping a new class of devs that just want the easy way out. and ultimately this wave will make the price go down to the point the skilled dev won't be able to sustain long term growth because learning more and more advanced will not be valued by the economy.. just a thought but i don't see a nice path ahead now

Why is it not helping the experienced engineer? I don't fully understand your scenario.

If the experienced engineer is already faster than the LLM, their job is not at risk.

If the LLM is faster then the experienced engineer at making some kind of code product, then the experienced engineer can use it to save time. And in the short term they can spend even more time learning! Maybe it's a net negative because it helps the "new class of devs that just want the easy way out" more, but it's still helping the experienced engineer.

And if increased competition drops the price then the LLM's influence is helping customers.

Re: The highest quality codebase

#366

Post by smt88 »

swid wrote: post #330
smt88 wrote: post #200

Earlier quoted context omitted.

It isn't, and it reflects how deeply LLMs are misunderstood, even by technical people

It surely is different. If you set the temp to 0 and do the test with slightly different wording, there is no guarantee at all the scores would be consistent. And if an LLM is consistent, even with a high temp, it could give the same PR the same grade while choosing different words to say. The tokens are still chosen from the distribution, so a higher probability of the same grade will result in the same grade being…

I think you're restating (in a longer and more accurate way) what I understood the original criticism to be, that this grading test isn't testing what's it's supposed to, partly because a grade is too few tokens.

The model could "assess" the code qualitatively the same and still give slightly different letter grades.

Re: The highest quality codebase

#368

Post by [deleted] »

mbesto wrote: post #83
xnorswap wrote: post #18

Claude is really good at specific analysis, but really terrible at open-ended problems. "Hey claude, I get this error message: ", and it'll often find the root cause quicker than I could. "Hey claude, anything I could do to improve Y?", and it'll struggle beyond the basics that a linter might suggest. It suggested enthusiastically a library for and it was all " Recommended " about it, but when I pointed out that the…

> There's a significant blind-spot in current LLMs related to blue-sky thinking and creative problem solving. It can do structured problems very well, and it can transform unstructured data very well, but it can't deal with unstructured problems very well. While this is true in my experience, the opposite is not true. LLMs are very good at helping me go through a structure processing of thinking about architectural a…

[deleted]

Re: The highest quality codebase

#369

Post by Dilettante_ »

krupan wrote: post #325
havkom wrote: post #60

Earlier quoted context omitted.

There are fundamental differences. Many people expect a positive gradient of quality from AI overhaul of projects. For translating back and forth, it is obvious from the outset that there is a negative gradient of quality (the Chinese whispers game).

200 times?? No, I don't think anybody expected that to produce something good. It's just a other attention grabbing, "I did a thing a ridiculous amount of times!" stunt.

You're assuming the author did this only for audience engagement(dozens, nay, scores of blog hits!), and not to gratify their own intellectual curiosity? Sometimes you just want to see what happens.

Re: The highest quality codebase

#370

Post by agumonkey »

Dylan16807 wrote: post #365
agumonkey wrote: post #352

Earlier quoted context omitted.

it's paradoxical, the llm is not helping consumers, it's not helping the experienced engineer, it's helping a new class of devs that just want the easy way out. and ultimately this wave will make the price go down to the point the skilled dev won't be able to sustain long term growth because learning more and more advanced will not be valued by the economy.. just a thought but i don't see a nice path ahead now

Why is it not helping the experienced engineer? I don't fully understand your scenario. If the experienced engineer is already faster than the LLM, their job is not at risk. If the LLM is faster then the experienced engineer at making some kind of code product, then the experienced engineer can use it to save time. And in the short term they can spend even more time learning! Maybe it's a net negative because it help…

you may have a point, i'm fuzzy in my perception right now but there are non linear factor imo, here's how i see things

- a market needs a certain kind of product (feature set, complexity, performance)

- good engineer could apply skills to deliver that

- lazy engineers couldn't, but with llm they can, it gives them solutions without understanding much, which is irrelevant for them, they want to ship

- i myself don't enjoy having code spilled out for me, and the time savings from llm won't bring much more joy (unlike the lazy engineer who is happy)

- a llm might help me do more advanced things but the market might not care for it. say the average user wants a dashboard with a bunch of data points and a few actions. the llm answer will match that perfectly. I could ask the llm to produce a more complex dashboard with more customization, more feature.. but the user will not want it because it's beyond its needs

so yeah it's a matter of ratio, it seems that lazy devs will get a 10x improvement while a skilled one will only get 1.5 and might be squeezed out of the market

Post reply on HN