Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

241–250 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#241

Post by troupo »

surgical_fire wrote: post #80
xutopia wrote: post #4

The most likely danger with AI is concentrated power, not that sentient AI will develop a dislike for us and use us as "batteries" like in the Matrix.

"AI will take over the world". I hear that. Then I try to use AI for simple code task, writing unit tests for a class, very similar to other unit tests. If fails miserably. Forgets to add an annotation and enters in a death loop of bullshit code generation. Generates test classes that tests failed test classes that test failed test classes and so on. Fascinating to watch. I wonder how much CO2 it generated while fryi…

"Just one more prompt, bro", and your problems will be solved.

Re: Beliefs that are true for regular software but false when applied to AI

#243

Post by nl »

alwa wrote: post #178
genghisjahn wrote: post #167

Earlier quoted context omitted.

Apparently? From what? Where did this information come from that they had two competing teams?

I feel like I hear people referring to Wayne Ma’s reporting for The Information to that effect. https://www.theinformation.com/articles/apple-fumbled-siris-... > Distrust between the two groups got so bad that earlier this year one of Giannandrea’s deputies asked engineers to extensively document the development of a joint project so that if it failed, Federighi’s group couldn’t scapegoat the AI team. > It didn’t hel…

https://archive.is/Ncefp

This is a pretty good article, and worth reading if you aren't aware that Apple has seemingly mostly abandoned the vision of on-device AI (I wasn't aware of this)

Re: Beliefs that are true for regular software but false when applied to AI

#244

Post by drpixie »

coliveira wrote: post #38
drsupergud wrote: post #11

> bugs are usually caused by problems in the data used to train an AI This also is a misunderstanding. The LLM can be fine, the training and data can be fine, but because the LLMs we use are non-deterministic (at least in regard to their being intentional attempts at entropy to avoid always failing certain scenarios) current algorithms are inherently by-design not going to always answer every question correctly that…

This is not necessarily a problem. Any programming or mathematical question has several correct answers. The problem with LLMs is that they don't have a process to guarantee that a solution is correct. They will give a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way. That's why LLMs generate so many bugs in software and in anything related to logical t…

>> a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way

Not quite ... LLMs are not HAL (unfortunately). They produce something that is associated with the same input, something that should look like an acceptable answer. A correct answer will be acceptable, and so will any answer that has been associated with similar input. And so will anything that fools some of the people, some of the time ;)

The unpredictability is a huge problem. Take the geoguess example - it has come up with a collection of "facts" about Paramaribo. These may or may-not be correct. But some are not shown in the image. Very likely the "answer" is derived from completely different factors, and the "explanation" in spurious (perhaps an explanation of how other people made a similar guess!)

The questioner has no way of telling if the "explanation" was actually the logic used. (It wasn't!) And when genuine experts follow the trail of token activation, the answer and the explanation are quite independent.

Re: Beliefs that are true for regular software but false when applied to AI

#245

Post by huhkerrf »

eru wrote: post #212
remexre wrote: post #205

Earlier quoted context omitted.

It does feel like somebody forgot that "from the first sentence or two of the email, you can tell what it's about" was already a rule of good writing...

You sometimes need to want to quickly learn what's in an email that was written by someone less helpful. Eg sometimes the writer is outright antagonistic, because they have some obligation to tell you something, but don't actually want you to know.

This is a pretty damning example of backwards product thinking. How often, truly, does this happen?

Re: Beliefs that are true for regular software but false when applied to AI

#246

Post by Eisenstein »

conception wrote: post #68
toomuchtodo wrote: post #19

Earlier quoted context omitted.

Labor competes for compensation, CEOs compete for status (above a certain enterprise size, admittedly). Show me a CEO willingly stepping down to be replaced by generative AI. Jamie Dimon will be so bold to say AI will bring about a 3 day week (because it grabs headlines [1]) but he isn't going to give up the status of running JPMC; it's all he has besides the wealth, which does not appear to be enough. The feeling of…

Spoiler there’s no reason we couldn’t work three days a week now. And 100 might be pushing it, but having life expectancy to 90 as well within our grass today as well. We have just decided not to do that.

The reason we don't have 3 day weeks is because the system rewards revenue, not worker satisfaction.

Re: Beliefs that are true for regular software but false when applied to AI

#247

Post by Eisenstein »

Animats wrote: post #48
toomuchtodo wrote: post #19

Earlier quoted context omitted.

Labor competes for compensation, CEOs compete for status (above a certain enterprise size, admittedly). Show me a CEO willingly stepping down to be replaced by generative AI. Jamie Dimon will be so bold to say AI will bring about a 3 day week (because it grabs headlines [1]) but he isn't going to give up the status of running JPMC; it's all he has besides the wealth, which does not appear to be enough. The feeling of…

That's the market's job. Once AI CEOs start outperforming human CEOs, investment will flow to the winners. Give it 5-10 years. (Has anyone tried an LLM on an in-basket test? [1] That's a basic test for managers.) [1] https://en.wikipedia.org/wiki/In-basket_test

Not if CEOs use their political power to make it illegal.

Re: Beliefs that are true for regular software but false when applied to AI

#248

Post by smokel »

scuff3d wrote: post #225
themanmaran wrote: post #18

> Because eventually we’ll iron out all the bugs so the AIs will get more reliable over time Honestly this feels like a true statement to me. It's obviously a new technology, but so much of the "non-deterministic === unusable" HN sentiment seems to ignore the last two years where LLMs have become 10x as reliable as the initial models.

Ironically I came to the comments to point out that all over hackernews you see this sentiment repeated, and that's by a group I would consider to be far more technically competent then your average person. And very helpfully there is one just a few comments down from the top.

Technical competence and an interest in sociological development do not always coincide. Technology often seeks simplicity, whereas sociology examines inherently complex human behavior.

Re: Beliefs that are true for regular software but false when applied to AI

#249

Post by mattmanser »

ako wrote: post #236
wrs wrote: post #36

My current method for trying to break through this misconception is informing people that nobody knows how AI works. Literally. Nobody knows. (Note that knowing how to make something is not the same as knowing how it works. Take humans as an obvious example.)

I think there are people who know exactly how it works, they know how neural networks work, they know how the transformer architecture works, how attention works, embeddings, tokenization, etc. We just can’t define the weights of the connections between the neurons.

That's a bit like saying knowing how a pipe works is enough to explain a combustion engine. You're just listing part of how LLMs work.

Those mechanisms only explain next word prediction, not LLM reasoning.

That's an emergent property that no person, as far as I understand it, can explain past hand waving.

Happy to be corrected here.

Re: Beliefs that are true for regular software but false when applied to AI

#250

Post by Yoric »

tkgally wrote: post #188
Yoric wrote: post #116

Earlier quoted context omitted.

By the time a junior dev graduates to senior, I expect that they'll be more reliable. In fact, at the end of each project, I expect the junior dev to have grown more reliable. LLMs don't learn from a project. At best, you learn how to better use the LLM. They do have other benefits, of course, i.e. once you have trained one generation of Claude, you have as many instances as you need, something that isn't true with h…

> LLMs don't learn from a project. How long do you think that will remain true? I've bootstrapped some workflows with Claude Code where it writes a markdown file at the end of each session for its own reference in later sessions. It worked pretty well. I assume other people are developing similar memory systems that will be more useful and robust than anything I could hack together.

For LLMs? Mostly permanently. This is a limitation of the architecture. Yes, there are workarounds, including ChatGPT's "memory" or your technique (which I believe are mostly equivalent), but they are limited, slow and expensive.

Many of the inventors of LLMs have moved on to (what they believe are) better models that would handle such learnings much better. I guess we'll see in 10-20 years if they have succeeded.

Post reply on HN