Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI: Accelerated Incompetence

slater.dev

Post reply on HN
Print view View original

161–170 of 287 posts

Re: AI: Accelerated Incompetence

#161

Post by Mtinie »

jvanderbot wrote: post #115
Artgor wrote: post #65

You know, sometimes I feel that all this discourse about AI for coding reflects the difference between software engineers and data scientists / machine learning engineers. Both often work with unclear requirements, and sometimes may face floating bugs which are hard to fix, but in most cases, SWE create software that is expected to always behave in a certain way. It is reproducible, can pass tests, and the tooling is…

This has been about 50% of the time my experience as well. There are very good SWE who know how to use ML in real systems, and then there are the others who believe through and through it will replace well understood systems developed by subdomain experts. As a concrete example, when I worked at Amazon, there were several really good ML-based solutions for very real problems that didn't have classical approaches to l…

Your stationary plane example highlights a divide I've seen across my work experience in different domains; teams defaulting to ML when fundamental engineering would work better.

I'm curious: do you think there's any amount of high-quality data that could make the learning-based approach viable for orientation estimation? Or would it always be solving the wrong problem, regardless of data volume and delivery speed?

My sense is that effective solutions need the right confluence of problem understanding, techniques, data, and infrastructure. Missing any one piece makes things suboptimal, though not necessarily unsolvable.

Re: AI: Accelerated Incompetence

#162

Post by twak »

Artgor wrote: post #65

You know, sometimes I feel that all this discourse about AI for coding reflects the difference between software engineers and data scientists / machine learning engineers. Both often work with unclear requirements, and sometimes may face floating bugs which are hard to fix, but in most cases, SWE create software that is expected to always behave in a certain way. It is reproducible, can pass tests, and the tooling is…

i agree; but perhaps also it is the difference between managers and SWE? The former (SWE team leaders included) can see that engineers aren't perfect. The latter are often highly focused on determinism (this works/doesn't) and struggle with conflicting goals.

Through a career SWEs start rigid and overly focused on the immediate problem and become flexible/error-tolerant[1] as they become system (mechanical or meat) managers. this maps to an observation that managers like AI solutions - because they compare favourably to the new hire - and because they have the context to make this observation.

[1] https://grugbrain.dev/#:~:text=grug%20note%20humourous%20gra...

Re: AI: Accelerated Incompetence

#163

Post by pixl97 »

bgwalter wrote: post #131
davidclark wrote: post #108

Earlier quoted context omitted.

> The overall average ability of people being able to get from Point A to Point B safely and reliably, especially in areas they are unfamiliar with, has certainly increased dramatically. Is there evidence for this?

No, I've never heard of someone getting into a unsafe situation because of using paper maps. When there were only paper maps, people managed. Then there is this Google Maps accident: https://www.independent.co.uk/tv/news/driver-bridge-google-m... Which tells you that following directions of a computer makes people more stupid.

>I've never heard of someone getting into a unsafe situation because of using paper maps

Simply because the media didn't report on it....

Re: AI: Accelerated Incompetence

#164

Post by dematz »

sensanaty wrote: post #21

> Input Risk. An LLM does not challenge a prompt which is leading ... (Emphasis mine) This has been the biggest pain point for me, and the frustrating part is that you might not even realize you're leading it a particular way at all. I mean it makes sense with how LLMs work, but a single word used in a vague enough way is enough to skew the results in a bad direction, sometimes contrary to what you actually wanted to…

Definitely share your feeling that people move the goalposts from "AI can do it" to "well it would have been able to do it if you used model o2.7 in an IDE with RAG and also told it how to do it in the prompt" ...ok, at some point it's less value for the effort than writing the code myself, thanks

That said, AI does make some things easier today, like if you have an example to use for "make me a page like this but with data from x instead of y". Often it's faster than searching documentation, even with the caveat that it might hallucinate. And ofc it will probably improve over time.

The particular improvement I'd like to see is (along with in general doing things right) finding the simplest solution without constantly having to be told to do so. My experience is the biggest drawback to letting chatgpt/claude/etc loose is quickly churning out a bunch of garbage, never stopping to say this will be too complex to do anything with in the future. TFA claims only humans can resist entropy by understanding the overall design; again idk if that will never improve but it feels like the big problem right now.

Re: AI: Accelerated Incompetence

#165

Post by sklarsa »

sensanaty wrote: post #21

> Input Risk. An LLM does not challenge a prompt which is leading ... (Emphasis mine) This has been the biggest pain point for me, and the frustrating part is that you might not even realize you're leading it a particular way at all. I mean it makes sense with how LLMs work, but a single word used in a vague enough way is enough to skew the results in a bad direction, sometimes contrary to what you actually wanted to…

> but a single word used in a vague enough way is enough to skew the results in a bad direction

I'm glad I'm not the only one who feels this way. It seems like these models latch on to a particular keyword somewhere in my prompt chain and throw traditional logic out the window as they try to push me down more niche paths that don't even really solve the original problem. Which just leads to higher levels of frustration and unhappiness for the human involved.

> Anecdotally, I've felt my skills quickly regressing because of AI tooling

To combat this, I've been trying to use AI to solve problems that I normally would with StackOverflow results: for small, bite-sized and clearly-defined tasks. Instead of searching "how to do X?", I now ask the model the same question and use its answer as a guide to solving the problem instead of a canonical answer.

Re: AI: Accelerated Incompetence

#166

Post by viraptor »

gwbas1c wrote: post #91
karn97 wrote: post #45

Earlier quoted context omitted.

[flagged]

I don't get the downvotes. When I used the free trials of FSD, It kept doing dangerous things and I kept needing to intervene. It seems a lot of people conflate the fact that the technology is impressive with the fact that it's still quite dangerous. > FSD is way more unreliable than an average drunk driver. I'm thinking you mean something like someone who blows a 0.1 or 0.12, or possibly drank 2-3 beers. (IE, someon…

It's a troll, or someone who actually doesn't understand the difference between an individual example and statistics. The claim is so bad it's not worth discussing.

Re: AI: Accelerated Incompetence

#168

Post by bsaul »

mjburgess wrote: post #68
bsaul wrote: post #41

"[AI] is not capable of working at a conceptual level". I wonder where did author got that feeling. What recent LLMs proved time and time again is that they are definitely able to work at conceptual level (by correctly translating concepts from one language to another depending on the context for example). Saying it doesn't "understand" the concepts as humans do is a different thing. It wouldn't "understand" pain, be…

They work in a token space whose metrical structure is given by proxies for concepts. So at a point in this space I can "walk towards" points which cluster around the token "dog". This is a weak model of some features of concepts, eg., association: "dog" is associated with "cat", etc. But it, e.g., does not model composition, nor intension, nor the role of the term in counterfactuals. (See my comment elsewhere in thi…

As always with definitive assertions regarding LLMs incapacities, i would be more convinced if one could demonstrate those assertions with an illustrative example, on a real LLM.

So far, the abilities of LLM to manipulate concepts, in practice, has been indistinguishable in practice from "true" human-level concept manipulation. And not just for scientific, "narrow" fields.

Re: AI: Accelerated Incompetence

#169

Post by pixl97 »

throw4847285 wrote: post #148
bearjaws wrote: post #23

Let's be real, 70% of staff are phoning in their jobs so badly that an AI often is just as good if not better. The real struggle will be, the people phoning it in are still going to be useless, but with AI. The rest will learn and grow with AI.

That's an extremely self-serving narrative. I assume you're part of the 30 percent?

It's probably better to look at a group from outside. Every company of any size seems to accumulate at least some people that could be replaced with a small shell script. Where I work there are a few people that seem so questionable at their job (even though most are good) I wonder how they keep their positions. I'd rather work with AI for the rest of my life then have to deal with them again.

Re: AI: Accelerated Incompetence

#170

Post by pixl97 »

pydry wrote: post #81
silvestrov wrote: post #47

Earlier quoted context omitted.

This will require companies to be able to survive for some time using the old code and to recognise the importance of cleaning up. Has this ever happened besides the Y2K fixes? Wouldn't it be much more likely that the companies will simply go under? Or that they will make a team that writes a completely new version of the code, somewhat like Mac OS X was a replacement of MacOS 9 and not a cleanup.

Yeah, it happened at the end of the last outsourcing boom. It's often a very quiet transformation though - a tacit recognition that a fashion failed doesn't get the same PR as "groundbreaking new business fad" because to do so too explicitly would humiliate all of the executives who were the driving force the original fashion. Success has many fathers, failure is an orphan. Publications like the Economist or the WSJ…

>benefits of tech unions

Careful speaking such heresy around here, you might get burned at the stake.

Post reply on HN