Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We might all be AI engineers now

yasint.dev

Post reply on HN
Print view View original

271–280 of 384 posts

Re: We might all be AI engineers now

#271

Post by 20k »

cdetrio wrote: post #268
20k wrote: post #254

Earlier quoted context omitted.

Except you have no idea if what the LLM is telling you is true I do a lot of astrophysics. Universally LLMs are wrong about nearly every astrophysics questions I've asked them - even the basic ones, in every model I've ever tested. Its terrifying that people take these at face value For research at a PhD level, they have absolutely no idea what's going on. They just make up plausible sounding rubbish

Astrophysicist David Kipping had a podcast episode a month ago reporting that LLMs are working shockingly well for him, as well as for the faculty at the IAS.[1] It's curious how different people come to very different conclusions about the usefulness of LLMs. https://youtu.be/PctlBxRh0p4

The problem with these long videos is that what I really want to see is what questions were asked of it, and the accuracy of the results

Every time I ask LLMs questions I know the answers to, its results are incomplete, inaccurate, or just flat out wrong much of the time

The idea that AI is an order of magnitude superior to coders is flat out wrong as well. I don't know who he's talking to

Re: We might all be AI engineers now

#272

Post by 20k »

simonw wrote: post #265
20k wrote: post #251

Earlier quoted context omitted.

But you have to check the answer against the documentation anyway though, to validate that its actually correct! Unless you're just taking the LLM answers at face value?

For most code stuff you don't check the answer against the documentation - you write the code and run it and see if it works. That's always a better signal than anything that official documentation might tell you.

That seems like a strong error, you have no idea if it works or if it just happens to work

Re: We might all be AI engineers now

#273

Post by 20k »

mgraczyk wrote: post #257
20k wrote: post #252

Earlier quoted context omitted.

I mean, is it really that hard to find information in the docs? Like, if I want to find out what, I don't know, "GetQueuedCompletionStatus" does. I google GetQueuedCompletionStatus Find this page: https://learn.microsoft.com/en-us/windows/win32/api/ioapiset... Bam, that's the single source of truth right there. Microsoft's docs are pretty great If I use an LLM, I have to ask it for the documentation about "GetQueuedC…

How about we do the following. I have not done win32 programming in 12 years. Maybe you've done it more recently. I'll use an LLM and you look up things manually. We can see, who can build a win32 admin UI that shows a realtime view of every open file by process with sorting, filtering and search on both the files and process/command names. I estimate this will take me 5 minutes Would you like to race?

This mentality is fundamentally why I think AI is not that useful, it completely underscores everything that's wrong with software engineering and what makes a very poor quality senior developer

I'll write an application without AI that has to be maintained for 5 years with an ever evolving featureset, and you can write your own with AI, and see which codebase is easiest to maintain, the most productive to add new features to, and has the fewest bugs and best performance

Re: We might all be AI engineers now

#274

Post by overgard »

_dwt wrote: post #123
noemit wrote: post #3

Not a day goes by that a fellow engineer doesn't text me a screenshot of something stupid an AI did in their codebase. But no one ever mentions the hundreds of times it quietly wrote code that is better than most engineers can write. The catch about the "guided" piece is that it requires an already-good engineer. I work with engineers around the world and the skill level varies a lot - AI has not been able to bridge…

I am going to try to put this kindly: it is very glib, and people will find it offensive and obnoxious, to implicitly round off all resistance or skepticism to incuriosity. Perhaps to alienate AI critics even further is the goal, in which case - carry on. But if you are genuinely confused by the attitudes of your peers, try asking not "what do I have that they lack" ("curiosity"?) but "what do they see that I don't"…

My feeling is that the code it generates is locally ok, but globally kind of bad. What I mean is, in a diff it looks ok. But when you start comparing it to the surrounding code, there's a pretty big lack of coherency and it'll happily march down a very bad architectural path.

In fairness, this is true of many human developers too.. but they're generally not doing it at a 1000 miles per hour and they theoretically get better at working with your codebase and learn. LLMs will always get worse as your codebase grows, and I just watched a video about how AGENTS.md actually usually results in worse outcomes so it's not like you can just start treating MD files as memory and hope it works out.

Re: We might all be AI engineers now

#275

Post by norir »

prescriptivist wrote: post #255
archagon wrote: post #247

Earlier quoted context omitted.

A new technology comes out — admittedly one that’s extraordinarily capable at some things — and suddenly conventional software engineering is “more or less obviated at this point”? I’m sorry, but that’s really fucking dumb. Do you think LLMs are actually intelligent? Do you think their capabilities exceed the quality of their training corpus? Is there no longer any need to think about new software paradigms, build ne…

> Is there no longer any need to think about new software paradigms, build new frameworks, study computer science, because the regurgitated statistical version of programming is entirely good enough? All I'm saying is you're gonna have to figure out how to do this with an agent. It's not that I don't see value in the craft; it's just that value is less important. As far as the new paradigms, the new frameworks, new s…

> In the meantime, there's nothing stopping you from using the agent to write the code that is every bit as high quality as if you sat down and typed it in yourself.

You can only speak for yourself.

Re: We might all be AI engineers now

#276

Post by overgard »

prescriptivist wrote: post #193
_dwt wrote: post #123

Earlier quoted context omitted.

I am going to try to put this kindly: it is very glib, and people will find it offensive and obnoxious, to implicitly round off all resistance or skepticism to incuriosity. Perhaps to alienate AI critics even further is the goal, in which case - carry on. But if you are genuinely confused by the attitudes of your peers, try asking not "what do I have that they lack" ("curiosity"?) but "what do they see that I don't"…

I don't think that people who don't want to use these tools or clean old ways are incurious. But I think these developers should face the fact that those skills and those ways they are reticent to give up are more or less obviated at this point. Not in the future, but now. It's just that the adoption of these tools isn't evenly distributed yet. I think there's a place for thoughtful dialogue around what this means fo…

I'm using Claude every day, and it definitely makes me faster but.. I'm also able to give it a lot of very specific instructions and correct a lot of mistakes quickly because I look at the code and understand what it's doing; and I'm also asking it to write code in domains I understand. So I don't think these skills are obsolete at all. If anything, keeping them sharp is the only differentiator we have. "Agentic Engineering" is as much as joke as "Vibe Coding" is in my mind. The tools are powerful, but they don't make up for knowing how to code, and if you're just blindly trusting it it's going to end badly.

Re: We might all be AI engineers now

#277

Post by mgraczyk »

20k wrote: post #273
mgraczyk wrote: post #257

Earlier quoted context omitted.

How about we do the following. I have not done win32 programming in 12 years. Maybe you've done it more recently. I'll use an LLM and you look up things manually. We can see, who can build a win32 admin UI that shows a realtime view of every open file by process with sorting, filtering and search on both the files and process/command names. I estimate this will take me 5 minutes Would you like to race?

This mentality is fundamentally why I think AI is not that useful, it completely underscores everything that's wrong with software engineering and what makes a very poor quality senior developer I'll write an application without AI that has to be maintained for 5 years with an ever evolving featureset, and you can write your own with AI, and see which codebase is easiest to maintain, the most productive to add new fe…

Sure let's do it. I am pretty confident mine will be more maintainable, because I am an extremely good software engineer, AI is a powerful tool, and I use AI very effectively

I would literally claim that with AI I can work faster and produce higher quality output than any other software engineer who is not using AI. Soon that will be true for all software engineers using AI.

Re: We might all be AI engineers now

#278

Post by lazarus01 »

Sounds like the author is confused or trying too hard to please the audience. I feel software engineering has higher expectation to move faster now, which makes it more difficult as a discipline.

I personally code data structures and algorithms for 1 - 2 hrs a day, because I enjoy it. I find it also helps keeps me sharp and prevents me from building too much cognitive debt with AI generated code.

I find most AI generated code is over engineered and needs a thorough review before being deployed into production. I feel you still have to do some of it yourself to maintain an edge. Or at least I do at my skill level.

Re: We might all be AI engineers now

#279

Post by overgard »

_dwt wrote: post #227
prescriptivist wrote: post #193

Earlier quoted context omitted.

I don't think that people who don't want to use these tools or clean old ways are incurious. But I think these developers should face the fact that those skills and those ways they are reticent to give up are more or less obviated at this point. Not in the future, but now. It's just that the adoption of these tools isn't evenly distributed yet. I think there's a place for thoughtful dialogue around what this means fo…

Well, no, not with that attitude there won’t! I am not trying to insinuate that there is a conspiracy, or that posts like yours are part of it, but there has been a huge wave of posts and comments since February which narrow the Overton window to the distance between “it’s here and it’s great” and “I’m sad but it’s inevitable”. Humanity has possessed nuclear weapons for 80 years and has used them exactly twice in ang…

> We can in fact just NOT do things!

100% this. I don't know why we think that pouring trillions of dollars into something we barely understand to create an economic revolution that is almost certainly awful is at all "inevitable". We just need leaders that aren't complete idiots. I'm generally cynical, but I do see that normies (ie not in tech) are waking up a bit. I don't think the technology is inherently a bad thing, but the people that think that we should just do this as fast as possible to win "the race" should be shot into space as far as I'm concerned. To start with, we need a working SEC that can actually punish the grifting CEO's that are using fear to manipulate markets.

Re: We might all be AI engineers now

#280

Post by simonw »

20k wrote: post #272
simonw wrote: post #265

Earlier quoted context omitted.

For most code stuff you don't check the answer against the documentation - you write the code and run it and see if it works. That's always a better signal than anything that official documentation might tell you.

That seems like a strong error, you have no idea if it works or if it just happens to work

If you're good at programming you can usually tell exactly why it worked or didn't work. That's how we've all worked before coding agents came along too - you don't blindly assume the snippet you pasted off StackOverflow will work, you try it and poke at it and use it to build a firm mental model of whether it's the right thing or not.
Post reply on HN