Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It's not empowering to hand off the details

davidnicholaswilliams.com

Post reply on HN
Print view View original

101–110 of 154 posts

Re: It's not empowering to hand off the details

#101

Post by Barrin92 »

iepathos wrote: post #98
Barrin92 wrote: post #95

Earlier quoted context omitted.

>We don't need to understand them to be able to verify they work correctly I don't think that's the case and it's becoming more obvious every day. From people who cannot distinguish truth or fiction on online platforms, good code from bad, to insecure software that causes supply chain attacks, to investment vehicles or fake medicine peddled that does not work, the idiotic MBA manager who does not understand the worke…

If I commission a part specified as 10.00 ± 0.01 mm, I can verify it with a calibrated micrometer without understanding the CNC machine, its software, or how to manufacture the part. Likewise, a function with a finite input space can be exhaustively tested against its specification without understanding its implementation. In these cases, I need to understand the requirement and the test, not the production details,…

the real world doesn't consist of people sitting at home measuring square metal blocks. Judgement of just about anything sufficiently complex requires discernment, a critical eye, intuition and even taste.

A codebase and its long term health is never judged by someone going down lines of code with a ruler, but with experience that only comes from intimacy with software development. If you don't have that kind of living immediate judgement (in German we have the word Fingerspitzengefühl, lit. tip-of-the-finger-felling) you end up with people measuring your progress in commit frequency or token consumption because they in some autistic sense have nothing else left but to measure and quantify random things. We end up with so many idiotic products because the kind of people who had intuition and would immediately throw the thing out were fired, and now you have people who mistake random noise and metrics for the real thing.

Re: It's not empowering to hand off the details

#102

Post by johnfn »

But surely at some point you need to stop, right? Most web developers are perfectly capable without understanding the intricacies of the file system or OS, even though the browser sits on top them. Most engineers don’t know assembly, or transistors or electrical engineering. Steve Jobs didn’t know every line of Objective-C that went into OSX. I don’t think Michelangelo collected all his paint materials by hand when painting the Sistine Chapel. In some sense I don’t understand a lot of the panic around AI when the entire career progression of what it means to be a software engineer has simply been to deal with ever-increasing levels of abstraction.

Re: It's not empowering to hand off the details

#103

Post by Izkata »

cwnyth wrote: post #74
jameshart wrote: post #65

Earlier quoted context omitted.

One problem is that agents are notoriously bad at handling ‘me’ vs ‘you’ over an extended conversation. ‘Without my approval’ far back in the context of an extended session gets fuzzy about who ‘me’ is.

This has always seemed strange to me. It will offer a suggestion, I'll make a comment about a suggestion, and it returns with, "Your suggestion..." The further distance from the suggestion, the more likely it confuses who made it.

Possibly just a general issue, rather than a long conversation one: https://news.ycombinator.com/item?id=48799439

Re: It's not empowering to hand off the details

#104

Post by Izkata »

stavros wrote: post #89
ButlerianJihad wrote: post #87

Earlier quoted context omitted.

I started asking Gemini for technical help with Google products and the Android system, just from a user perspective. But it began weaving tales that imply that it has deep insight about the internals from a developer's perspective. Sooner or later, I began calling it out. I said "that sounds more like a Star Trek episode than technical support!" and it would go "you're absolutely right to call me out like that!" Tha…

Yeah hm, that sounds a lot like random technobabble. Why would the system randomly drop the thumbnail cache because of a swipe, and why would that corrupt the system?

The description is out of order and I'm just going to assume "corruption" actually means "invalid data":

Memory leak -> dropped cache. Then because something that was dropped in that cache is still being viewed it has to regenerate it and that triggers a re-render. Then based on something I've seen in a game, a re-render drops the scroll location and renders at the top, and if your finger moves at the exact right moment it repopulates x1 and y1 at the same timestamp it populates x2 and y2, resulting in infinite velocity.

Re: It's not empowering to hand off the details

#105

Post by bigstrat2003 »

sharts wrote: post #64

Why people continue to treat AI as though it’s not just an intern or recent grad? That alone will solve 90% of problems people have.

For one thing, it's not actually that useful to have such a person at your fingertips. You put up with the drain a junior places upon you because with training, he'll be able to come into his own and pay off the time invested. An LLM can't do that. Also, a lot of people incorrectly believe that an LLM is smarter than a new grad, and of course the companies making them have a vested financial interest in lying to you that their products are smarter than they are.

Re: It's not empowering to hand off the details

#106

Post by SyneRyder »

Sharlin wrote: post #76
existencebox wrote: post #50

Earlier quoted context omitted.

I expect there's a probabilistic factor to it, among other things. I use the almost the same addendum as you ("do not make, commit, or push any changes without my approval") and it broadly seems to work in my day-to-day dev. However, I also set up some agents that trigger on incoming requests from my dev teams to automate things that used to be rote/manual tasks. _That_ is where I see the issues, I suspect simply bec…

I haven’t used these tools that permit LLMs to make changes on their own, but I can’t help being baffled if there’s no big red switch that changes the system prompt and disables any and all write access to the outside world.

There is - in the Claude Code harness there's a dedicated Plan Mode that doesn't edit any files, or a mode where you can manually approve each edit.

If you're running Claude via -p automated mode, you can set all kinds of permissions and access to each individual editing tool, much like you would with firewall software.

It doesn't edit the system prompt, it blocks access to the tools itself. Just make sure you haven't given the AI the ability to edit its own configuration files, or you'll potentially have an OpenAI v Hugging Face situation.

Re: It's not empowering to hand off the details

#107

Post by protocolture »

RGS1811 wrote: post #19

I've been vibecoding a ton for the past 9 months, built a bunch of cool little apps for myself with AI, ran experiments, built an entire SDLC on skills, did the agent orchestration harness thing, etc. In the past few weeks I've hit a wall where I'm just tired of it. Each model becomes more independent but also harder to direct in detail. They produce massive, tedious, sloppy text outputs with very little input. They'…

12 months ago I had ChatGPT successfully not to mention quickly, exctract elements of a PNG as SVG. About 6 or 7 times.

I jumped in to perform the same thing again, and it spat out a PNG. And then when I asked it generated a little python script to generate the SVG. The Result was garbage. It took me a while to get the result I had gotten consistently last time, and the results included it hallucinating a few svgs in the process.

I do wonder if we have already passed the best possible models and are over specialising them at this point.

Re: It's not empowering to hand off the details

#108

Post by manwe150 »

raincole wrote: post #31
RGS1811 wrote: post #19

I've been vibecoding a ton for the past 9 months, built a bunch of cool little apps for myself with AI, ran experiments, built an entire SDLC on skills, did the agent orchestration harness thing, etc. In the past few weeks I've hit a wall where I'm just tired of it. Each model becomes more independent but also harder to direct in detail. They produce massive, tedious, sloppy text outputs with very little input. They'…

> as an assistant that helps me make decisions, and ensures that I'm in the driver's seat This... quite describes what the current AI feels like to me. So I really don't see a problem. If AI does too much for you, you should, uh, just prompt it to do less. Everyone talks about how their AI escapes sandbox and virtualization or delete their entire repos or something. Perhaps they're using a skynet I'm not aware of. Fr…

That’s not really the issue. The issue I have (sometimes) is that it might discover it has read-only access to most places (.git, github) and so sometimes when it thinks it “finished” it will discover those are read only and try to find workarounds (erase the read-only files, hand edit the .git folder, make new forks to push to those instead)—none of which a human would normally consider reasonable necessary effort. Most of those attempts do fail, but it turns out that sometimes just being maximally helpful in following orders strictly can be quite surprisingly adversarial, entirely by accident

Re: It's not empowering to hand off the details

#109

Post by QuercusMax »

Barrin92 wrote: post #95
iepathos wrote: post #78

This post ignores that verifying something works doesn't require you to fully understand it. This is easily observable in products we use everyday. We don't need to understand them to be able to verify they work correctly. The cost of verification is often cheaper than the cost of production. It present delegation as absence of power/agency. While I think there is some kernel of truth in here, the claim "it's not emp…

>We don't need to understand them to be able to verify they work correctly I don't think that's the case and it's becoming more obvious every day. From people who cannot distinguish truth or fiction on online platforms, good code from bad, to insecure software that causes supply chain attacks, to investment vehicles or fake medicine peddled that does not work, the idiotic MBA manager who does not understand the worke…

Blackbox software testing has been a thing for a very long time. Running manual tests doesn't require reading the source code.

For personal side projects I spend orders of magnitude more time testing than I do reading the code.

Re: It's not empowering to hand off the details

#110

Post by novaleaf »

lelanthran wrote: post #62
lifeisstillgood wrote: post #58

It reminds me of “magic”. I say “expelliamus” and the wand flies out the other guys hand, not his hat, or wallet or the person behind’s sandwiches. The details always matter. There is a (fairly) good book series where it turns out we are living in the matrix and someone finds a config file and effectively becomes a Wizard, and of course problems ensue with details (like flying by adjusting one’s position one feet up…

> There is a (fairly) good book series where it turns out we are living in the matrix and someone finds a config file and effectively becomes a Wizard, and of course problems ensue with details (like flying by adjusting one’s position one feet up every second is just vibrating a lot) This is very interesting. Which book series is this?

as others mentioned, it sounds like "Off to Be the Wizard". Unfortunately I wouldn't call it a great series. It starts off strong but drags. IIRC I got through half of the second book before stopping.
Post reply on HN