Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I trusted an LLM, now I'm on day 4 of an afternoon project

nemo.foo

Post reply on HN
Print view View original

31–40 of 215 posts

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#31

Post by talldayo »

potsandpans wrote: post #10

Counterexample: Ive been able to complete more side projects in the last month leveraging llms than i have ever in my life. One of which I believe to have potential as a viable product, and another which involved complicated rust `no_std` and linker setup for compiling rust code onto bare metal RISCV from scratch. I think the key to being successful here is to realize that you're still at the wheel as an engineer. Th…

> which involved complicated rust `no_std` and linker setup for compiling rust code onto bare metal RISCV from scratch.

That's complicated, but I wouldn't say the resulting software is complex. You gave an LLM a repetitive, translation-based job, and you got good results back. I can also believe that an LLM could write up a dopey SAAS in half the time it would take a human to do the same.

But having the right parameters only takes you so far. Once you click generate, you are trusting that the model has some familiarity with your problem and can guide you without needing assistance. Most people I've seen rely entirely on linting and runtime errors to debug AI code, not "solid fundamentals" that can fact-check a problem they needed ChatGPT to solve first place. And the "experience" required to iterate and deploy AI-generated code basically boils down to your copy-and-paste skills. I like my UNIX knowledge, but it's not a big enough gate to keep out ChatGPT Andy and his cohort of enthusiastic morons.

We're going to see thousands of AI-assisted success stories come out of this. But we already had those "pennies on the dollar" success stories from hiring underpaid workers out of India and Pakistan. AI will not solve the unsolved problems of our industry and in many ways it will exacerbate the preexisting issues.

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#32

Post by NitpickLawyer »

KronisLV wrote: post #16

In my experience LLMs will help you with things that have been solved thousands of times before and are just a matter of finding some easily researched solution. The very moment when you try to go off the beaten path and do something unconventional or stuff that most people won't have written a lot about, it gets more tricky. Just consider how many people will know how to configure some middleware in a Node.js projec…

> vs most things related to hardware or low level work.

counter point:

https://github.com/ggerganov/llama.cpp/pull/11453

> This PR provides a big jump in speed for WASM by leveraging SIMD instructions for qX_K_q8_K and qX_0_q8_0 dot product functions.

> Surprisingly, 99% of the code in this PR is written by DeekSeek-R1. The only thing I do is to develop tests and write prompts (with some trials and errors)

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#33

Post by baxtr »

potsandpans wrote: post #10

Counterexample: Ive been able to complete more side projects in the last month leveraging llms than i have ever in my life. One of which I believe to have potential as a viable product, and another which involved complicated rust `no_std` and linker setup for compiling rust code onto bare metal RISCV from scratch. I think the key to being successful here is to realize that you're still at the wheel as an engineer. Th…

Is it reasonable to assume that more senior devs benefit more from LLMs?

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#34

Post by dogma1138 »

potsandpans wrote: post #10

Counterexample: Ive been able to complete more side projects in the last month leveraging llms than i have ever in my life. One of which I believe to have potential as a viable product, and another which involved complicated rust `no_std` and linker setup for compiling rust code onto bare metal RISCV from scratch. I think the key to being successful here is to realize that you're still at the wheel as an engineer. Th…

Indeed LLMs are useful as an intern, they are at the “cocky grad” stage of their careers. If you don’t understand the problem and can’t steer the solution and worse has only limited understanding of the code they produce you are unlikely to be productive.

On the other hand if you understand what needs to be done, and how to direct the work the productivity boost can be massive.

Claude 3.5 sonnet and O1 are awesome at code generation even with relatively complex tasks and they have a long enough context and attention windows that the code they produce even on relatively large projects can be consistent.

I also found a useful method of using LLMs to “summarize” code in an instructive manner which can be used for future prompts. For example summarizing a large base class that may be reused in multiple other classes can be more effective than having to overload a large part of your context window with a bunch do code.

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#35

Post by dogma1138 »

baxtr wrote: post #33
potsandpans wrote: post #10

Counterexample: Ive been able to complete more side projects in the last month leveraging llms than i have ever in my life. One of which I believe to have potential as a viable product, and another which involved complicated rust `no_std` and linker setup for compiling rust code onto bare metal RISCV from scratch. I think the key to being successful here is to realize that you're still at the wheel as an engineer. Th…

Is it reasonable to assume that more senior devs benefit more from LLMs?

It depends it think it’s less about how senior they are and how good they are at writing requirements, and knowing what directives should be explicitly stated and what can be safely inferred.

Basically if they are good at utilizing junior developers and interns or apprentices they probably will do well with an LLM assistant.

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#36

Post by transcriptase »

I’ve been able to do more far complex things with ESP32s and RPis in an evening without knowing the first thing about python or c++.

I can also tell when it’s stuck in some kind of context swamp and won’t be any more help, because it will just keep making the same stupid mistakes over and over and generally forgetting past instructions.

At that point I take the last working code and paste it into a new chat.

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#37

Post by nrb »

pieix wrote: post #7

> AI isn’t a co-pilot; it’s a junior dev faking competence. Trust it at your own risk. This is a good take that tracks with my (heavy) usage of LLMs for coding. Leveraging productive-but-often-misguided junior devs is a skill every dev should actively cultivate!

> Leveraging productive-but-often-misguided junior devs is a skill every dev should actively cultivate!

Feels like this is only worthwhile because the junior dev learns from the experience; an investment that yields benefits all around, in the broad sense. Nobody wants a junior around that refuses to learn in perpetuity, serving only as a drag on productivity and eventually your sanity.

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#38

Post by alfalfasprout »

NitpickLawyer wrote: post #32
KronisLV wrote: post #16

In my experience LLMs will help you with things that have been solved thousands of times before and are just a matter of finding some easily researched solution. The very moment when you try to go off the beaten path and do something unconventional or stuff that most people won't have written a lot about, it gets more tricky. Just consider how many people will know how to configure some middleware in a Node.js projec…

> vs most things related to hardware or low level work. counter point: https://github.com/ggerganov/llama.cpp/pull/11453 > This PR provides a big jump in speed for WASM by leveraging SIMD instructions for qX_K_q8_K and qX_0_q8_0 dot product functions. > Surprisingly, 99% of the code in this PR is written by DeekSeek-R1. The only thing I do is to develop tests and write prompts (with some trials and errors)

A single PR doesn't really "prove" anything. Optimization passes on well-tested narrowly scoped code are something that LLMs are already pretty good at.

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#40

Post by ravroid »

One strategy I've been experimenting with is maintaining a 'spec' document, outlining all features and relevant technical notes about a project. I include the spec with all relevant source files in my prompt before asking the LLM to implement a new change or feature. This way it doesn't have to do as much guessing as to what my code is doing, and I can avoid relying on long-running conversations to maintain context. Instead, for each big change I include an up-to-date spec and all of the relevant source files. I update the spec to reflect the current state of the project as changes are made to give the LLM context about my project (this also doubles as documentation).

I use an NPM script to automate concatenating the spec + source files + prompt, which I then copy/paste to o1. So far this has been working somewhat reliably for the early stages of a project but has diminishing returns.

Post reply on HN