Live data from Hacker News

AI: Accelerated Incompetence

slater.dev

61–70 of 287 posts

Re: AI: Accelerated Incompetence

#61

I disagree with the author here. His point can be summarized as "LLMs produce bad code, LLMs lead to the teams's average skills decreasing and they can't maintain large codebases over time". Yet nowhere he addresses the #1 flaws to his position: rate of improvement of the technology, and its promise to deliver on saved money and gained speed. In all the companies I've seen engineering leadership hardly really gives a…

[flagged]

Re: AI: Accelerated Incompetence

#62

I disagree with the author here. His point can be summarized as "LLMs produce bad code, LLMs lead to the teams's average skills decreasing and they can't maintain large codebases over time". Yet nowhere he addresses the #1 flaws to his position: rate of improvement of the technology, and its promise to deliver on saved money and gained speed. In all the companies I've seen engineering leadership hardly really gives a…

> Assume in [N] years these AIs are 10x better

Why has OpenAI been acquiring "application" layer companies for large financial sums, instead of improving their own tools to build application layer codebases?

> 30% of new lines of code is AI-generated

"Watching AI drive Microsoft employees insane", 500 comments, https://news.ycombinator.com/item?id=44050152

Re: AI: Accelerated Incompetence

#63
post #44
post #33

Earlier quoted context omitted.

Definitely. Honestly I was going to write a custom forum in the style of those perl bboards but with https://90s.dev aesthetics (the website, not the os). But maybe first and foremost I need a mailing list so people can be notified of things like this when they're announced/released?

I'm interested. I don't think it should just be an older coders club though. It should be for anyone of any age that's into the craft of fine, well built software, and who wants to learn from people who have been into it a long time. I also like /r/tinycode for its spirit. My #1 saying in coding is "simplicity is harder than complexity." It's gone downhill like the rest of Reddit though. I'm also not totally anti-AI.…

Yeah it wouldn't be age-based. It would be 90s-themed, but the focus would entirely be on the cultivating the craft itself.

To clarify the AI stance, I meant it in the context of the article: it would encourage cultivating our skills so it both grows and doesn't atrophy.

Re: AI: Accelerated Incompetence

#64
post #12

Earlier quoted context omitted.

'human engineering' here isn't social engineering. I think they're referring to aspects of engineering that LLMs are still incapable of - like design simplification.

I follow... it's just, I much prefer to work with the computer than the humans. Simplification usually involves a lot of talking. edit: The [currently] Dead reply below is a touch ironic. I rest my case. Great sell, worth the squeeze.

[flagged]

Re: AI: Accelerated Incompetence

#65
You know, sometimes I feel that all this discourse about AI for coding reflects the difference between software engineers and data scientists / machine learning engineers.

Both often work with unclear requirements, and sometimes may face floating bugs which are hard to fix, but in most cases, SWE create software that is expected to always behave in a certain way. It is reproducible, can pass tests, and the tooling is more established.

MLE work with models that are stochastic in nature. The usual tests aren't about models producing a certain output - they are about metrics, that, for example, the models produce the correct output in 90% cases (evaluation). The tooling isn't as developed as for SWE - it changes more often.

So, for MLE, working with AI that isn't always reliable, is a norm. They are accustomed to thinking in terms of probabilities, distributions, and acceptable levels of error. Applying this mindset to a coding assistant that might produce incorrect or unexpected code feels more natural. They might evaluate it like a model: "It gets the code right 80% of the time, saving me effort, and I can catch the 20%."

Re: AI: Accelerated Incompetence

#66
post #45
post #30

Earlier quoted context omitted.

For large companies this is true, and this may be the best AI coding take I have read. It's similar with full self drive. FSD is better than a bad, drunk, or texting human driver, and that's a lot of the drivers on the road.

[flagged]

> FSD is way more unreliable than an average drunk driver.

Than an average drunk driver, or you drunk driving? One of my first drinking buddies when I was still an alcoholic (decades ago) was actually an excellent driver when drunk, but most people I knew at that time couldn't hardly even walk let alone drive when drunk.

Re: AI: Accelerated Incompetence

#67

> Input Risk. An LLM does not challenge a prompt which is leading ... (Emphasis mine) This has been the biggest pain point for me, and the frustrating part is that you might not even realize you're leading it a particular way at all. I mean it makes sense with how LLMs work, but a single word used in a vague enough way is enough to skew the results in a bad direction, sometimes contrary to what you actually wanted to…

[flagged]

Re: AI: Accelerated Incompetence

#68
post #41

"[AI] is not capable of working at a conceptual level". I wonder where did author got that feeling. What recent LLMs proved time and time again is that they are definitely able to work at conceptual level (by correctly translating concepts from one language to another depending on the context for example). Saying it doesn't "understand" the concepts as humans do is a different thing. It wouldn't "understand" pain, be…

They work in a token space whose metrical structure is given by proxies for concepts. So at a point in this space I can "walk towards" points which cluster around the token "dog".

This is a weak model of some features of concepts, eg., association: "dog" is associated with "cat", etc. But it, e.g., does not model composition, nor intension, nor the role of the term in counterfactuals. (See my comment elsewhere in this comments section on this issue).

However you can always brute force your way to apparent performance in some apparently conceptual skill if the kinds of questions you ask are similar to the trainign data. So eg., if someone has asked, "if dogs played on mars, would they be happy?" etc. or similar-enough-families-of-questions... then that allows you to have a "dog" cluster around "literal facts" and a "dog" cluster around some subset of preknown counterfactuals.

If you want to see the difference between this and genuine mental capabilities, note that there are an infinite combination of concepts of abitary depth, which can be framed in an infinite number of counterfactauls, and so on. And a child armed with only those basic components, and the capacity for imagination, can evaluate this infinite variety.

This is why we see LLMs being used most by narrow fields (esp. software engineers) where the kinds of "conceptual work" that they need has been extremely well documented and is sufficiently stable to provide some utiltiy.

Re: AI: Accelerated Incompetence

#69
post #41

"[AI] is not capable of working at a conceptual level". I wonder where did author got that feeling. What recent LLMs proved time and time again is that they are definitely able to work at conceptual level (by correctly translating concepts from one language to another depending on the context for example). Saying it doesn't "understand" the concepts as humans do is a different thing. It wouldn't "understand" pain, be…

> But humans constantly talk about thing they've never personally experienced

On the extreme, we can talk about things like Aphantasia, Synesthesia and colour blindness and understand the concepts even if we never experienced them.

Re: AI: Accelerated Incompetence

#70

Is AI perhaps an opportunity for years of paid cleanup and re-design work that those of us who have dedicated decades to learning the craft of software engineering will be able to cash in on?

This will require companies to be able to survive for some time using the old code and to recognise the importance of cleaning up. Has this ever happened besides the Y2K fixes? Wouldn't it be much more likely that the companies will simply go under? Or that they will make a team that writes a completely new version of the code, somewhat like Mac OS X was a replacement of MacOS 9 and not a cleanup.

> Or that they will make a team that writes a completely new version of the code

Isn't that one way of the "cleanup and re-design work"?

Post reply on HN