Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI: Accelerated Incompetence

slater.dev

Post reply on HN
Print view View original

101–110 of 287 posts

Re: AI: Accelerated Incompetence

#101

Post by tomrod »

Artgor wrote: post #65

You know, sometimes I feel that all this discourse about AI for coding reflects the difference between software engineers and data scientists / machine learning engineers. Both often work with unclear requirements, and sometimes may face floating bugs which are hard to fix, but in most cases, SWE create software that is expected to always behave in a certain way. It is reproducible, can pass tests, and the tooling is…

SWE use probability all the time. Rearchitect around that race condition or reduce its footprint? How long will this database call make, p99? A/B tests. Etc.

Re: AI: Accelerated Incompetence

#102

Post by [deleted] »

mjburgess wrote: post #68
bsaul wrote: post #41

"[AI] is not capable of working at a conceptual level". I wonder where did author got that feeling. What recent LLMs proved time and time again is that they are definitely able to work at conceptual level (by correctly translating concepts from one language to another depending on the context for example). Saying it doesn't "understand" the concepts as humans do is a different thing. It wouldn't "understand" pain, be…

They work in a token space whose metrical structure is given by proxies for concepts. So at a point in this space I can "walk towards" points which cluster around the token "dog". This is a weak model of some features of concepts, eg., association: "dog" is associated with "cat", etc. But it, e.g., does not model composition, nor intension, nor the role of the term in counterfactuals. (See my comment elsewhere in thi…

[deleted]

Re: AI: Accelerated Incompetence

#103

Post by rhubarbtree »

mjburgess wrote: post #43
paradite wrote: post #25

> LLMs as they currently exist cannot master a theory, design, or mental construct because they don't remember beyond their context window. Only humans can can gain and retain program theory. False. > An LLM is a token predictor. It works only at the level of text. It is not capable of working at a conceptual level: it doesn't reason about ideas, diagrams, or requirements specifications. False. Anyone who have spent…

The problem with people "who have spent time in machine learning or reinforcement learning" is that they've spent no time, literally none, understanding what a concept is. There is no such thing as a higher dimensional concept, nor can they be projected into a weight space, because they aren't quantities. The concept, say, "Dog" composes with the concept, "Happy" to form "Happy Dog". The extension(Dog) is all possibl…

If the effectiveness of LLMs has taught me anything, it’s that concepts are quantities, at least in the sense that they are vectors.

Good job too, otherwise the brain would have to magically create some entirely new physics to represent concepts.

Re: AI: Accelerated Incompetence

#104

Post by macintux »

iamacyborg wrote: post #82
anyonecancode wrote: post #73

I think you could make similar arguments about mapping technology like Google and Apple Maps -- that using them decreases people's skills in navigating the physical world, atrophying our sense of direction and geography. And actually, that's not wrong. People really do often struggle to navigate these days if they don't have the crutch of something like Google Maps. It really has changed our relationship to the physi…

> I think you could make similar arguments about mapping technology like Google and Apple Maps -- that using them decreases people's skills in navigating the physical world, atrophying our sense of direction and geography. And actually, that's not wrong. People really do often struggle to navigate these days if they don't have the crutch of something like Google Maps. It really has changed our relationship to the phy…

I think you’re both right, but the key is walking vs driving. Walking gives you time to look around, GPS reduces stress, and typically you’re walking in an urban location with landmarks.

Driving still requires careful attention to other drivers, the world goes by rapidly, and most roads look like other roads.

Re: AI: Accelerated Incompetence

#105

Post by rhubarbtree »

monero-xmr wrote: post #9

I use LLMs for annoying shit that used to take an inordinate amount of time. For example an analyst gives me a CSV, it has some data clean up issues, eventually it needs to become a series of SQL inserts. What was a day of script writing becomes 15 minutes of prompt engineering to clean up the CSV and convert it into the proper statements. Massive time savings! I believe one can vibe code a basic app or script. Like…

How do you check to make sure that the LLM hasn’t deleted any data, or changed something you didn’t want to be changed?

A colleague did this recently and found it had removed crucial data.

Better to get it to help you author scripts or difficult parts of scripts, and then review the code carefully.

Re: AI: Accelerated Incompetence

#106

Post by intended »

sensanaty wrote: post #21

> Input Risk. An LLM does not challenge a prompt which is leading ... (Emphasis mine) This has been the biggest pain point for me, and the frustrating part is that you might not even realize you're leading it a particular way at all. I mean it makes sense with how LLMs work, but a single word used in a vague enough way is enough to skew the results in a bad direction, sometimes contrary to what you actually wanted to…

I consider this hype the preparatory phase for the era where we rediscover the laws of quality control.

A GREAT example is good old Coke vs Pepsi.

Re: AI: Accelerated Incompetence

#107

Post by klabb3 »

iamacyborg wrote: post #82
anyonecancode wrote: post #73

I think you could make similar arguments about mapping technology like Google and Apple Maps -- that using them decreases people's skills in navigating the physical world, atrophying our sense of direction and geography. And actually, that's not wrong. People really do often struggle to navigate these days if they don't have the crutch of something like Google Maps. It really has changed our relationship to the physi…

> I think you could make similar arguments about mapping technology like Google and Apple Maps -- that using them decreases people's skills in navigating the physical world, atrophying our sense of direction and geography. And actually, that's not wrong. People really do often struggle to navigate these days if they don't have the crutch of something like Google Maps. It really has changed our relationship to the phy…

Also worth mentioning that tools have stable output. An LLM is not a tool in that sense – it’s not reproducible. Changing the model, retraining, input phrasing etc can change dramatically the output.

The best tools are transparent. They are efficient, fast and reliable, yes, but they’re also honest about what they do! You can do everything manually if you want, no magic, no hidden internal state, and with internal parts that can be broken up and tested in isolation.

With LLMs even the simple act of comparing them side by side (to decide which to use) is probabilistic and ultimately based partly on feelings. Perhaps it comes with the territory, but this makes me extremely reluctant to integrate it into engineering workflows. Even if they had amazing abilities, they lower the bar significantly from a process perspective.

Re: AI: Accelerated Incompetence

#108

Post by davidclark »

anyonecancode wrote: post #73

I think you could make similar arguments about mapping technology like Google and Apple Maps -- that using them decreases people's skills in navigating the physical world, atrophying our sense of direction and geography. And actually, that's not wrong. People really do often struggle to navigate these days if they don't have the crutch of something like Google Maps. It really has changed our relationship to the physi…

> The overall average ability of people being able to get from Point A to Point B safely and reliably, especially in areas they are unfamiliar with, has certainly increased dramatically.

Is there evidence for this?

Re: AI: Accelerated Incompetence

#109

Post by klabb3 »

90s_dev wrote: post #32
bearjaws wrote: post #23

Let's be real, 70% of staff are phoning in their jobs so badly that an AI often is just as good if not better. The real struggle will be, the people phoning it in are still going to be useless, but with AI. The rest will learn and grow with AI.

It's almost like people outsourcing their job to AI are asking to get fired, not only by proving that a computer program can do their job better, but even paving the way for it!

Don’t forget that they give away the data the AI needs for training.

Re: AI: Accelerated Incompetence

#110

Post by osigurdson »

People would do better to think about new problems that can be solved with LLMs that couldn't have been solved by either humans or machines previously. People are overly focused on the tiny area of the Venn diagram where human and machine competency intersect.
Post reply on HN