Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

321–330 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#321

Post by Rygian »

generic92034 wrote: post #93
wrs wrote: post #36

My current method for trying to break through this misconception is informing people that nobody knows how AI works. Literally. Nobody knows. (Note that knowing how to make something is not the same as knowing how it works. Take humans as an obvious example.)

Nobody knows (full scope and on every level) how human brains work. Still bosses rely on their employees' brains all the time.

If I need to manage an AI as I would manage an employee's brain, I'm going to need quite a few non-technical resources to actually achieve that: time, willpower to babysit it, ability to motivate it, leverage in the form of incentives (and reprimands), to name a few.

AI sits at a weird place where it can't be analyzed as software, and it can't be managed as a person.

My current mental model is that AGI can only be achieved when a machine experiences pleasure, pain, and "bodily functions". Otherwise there's no way to manage it.

Re: Beliefs that are true for regular software but false when applied to AI

#322

Post by skydhash »

gwd wrote: post #304
karuko24 wrote: post #301

> bugs are usually caused by problems in the data used to train an AI I think a fundamental problem is that many people assume that an LLM's failure to correctly perform a task is a bug that can be fixed somehow. Often times, the reason for that failure is simply a property of the AI systems we have at the moment. When you accidentally drop a glass and it breaks, you don't say that it's a bug in gravity. Instead, you…

But it's worse than that. Even if in theory the system could be fixed, we don't actually know how to fix it for real , the way we can fix a normal computer program. The reason we can't fix them is because we have no idea how they work; and the reason we have no idea how they work is this: 1. The "normal" computer program, which we do understand, implement a neural network 2. This neural network is essentially a diffe…

That’s a very poetic description. Mine is simpler. It’s a generator. You train it to give it the parameters (weights) of the formula thag generate stuff (the formula is known). Then you give it some input data, and it will gives you an output.

Both the weights and the formula is known. But the weight are meaningless in a human fashion. This is unlike traditional software where everything from encoding (the meaning of the bits) to how the state machine (the cpu) was codified by humans.

The only ways to fix it (somewhat) is to come up with better training data (hopeless), a better formula, or tacking something on top to smooth the worst errors (kinda hopeless).

Re: Beliefs that are true for regular software but false when applied to AI

#323

Post by formerly_proven »

rldjbpin wrote: post #286
freetime2 wrote: post #122

For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…

> to perform up to Apple's typical standards of polish and control. i no longer believe they have kept on to the standards in general. the ux/ui used to be a top priority, but the quality control has certainly gone down over the years [1]. the company is now driven by supply chain and business-minded optimizations than what to give to the end user. at the same time, what one can do using AI has large correlation with…

With Apple it's incredibly obvious that most software product development is nowadays handled by outsourced/offshored contractors who simply do not use the products. At least I hope that's the case, it would be disastrous if the state of iOS/watchOS is the result of their in-house on-shore talent.

Re: Beliefs that are true for regular software but false when applied to AI

#324

Post by tsimionescu »

immibis wrote: post #282
huhkerrf wrote: post #245

Earlier quoted context omitted.

This is a pretty damning example of backwards product thinking. How often, truly, does this happen?

Never heard of terms of service?

No one cares about the terms of service. And if they actually do, they will need to read every word very carefully to know if they are in legal trouble. A possibly wrong summary of a terms of service document is entirely and completely useless.

Re: Beliefs that are true for regular software but false when applied to AI

#325

Post by IshKebab »

veunes wrote: post #268

The "just find the bug and patch it" mindset is so deeply ingrained in engineering culture that it's easy to forget it doesn't apply here

And the public. Look at this article: https://www.bbc.co.uk/news/articles/cj07ley3jnpo

> "Oh my goodness, it worked, it's amazing it's finally been updated," she tells the BBC. "This is a great step forward."

She thinks someone noticed the bug about not being able to show one-armed people, figured out why it wasn't working and wrote a fix.

Re: Beliefs that are true for regular software but false when applied to AI

#326

Post by whiplash451 »

jeremyscanvic wrote: post #291

The part about AI being very sensitive to small perturbations of their input is actually a very active research topic (and coincidentally the subject of my PhD). Most vision AIs suffer from poor spatial robustness [1], you can drastically lower their accuracy simply by translating the inputs by well-chosen (adversarial) translations of a few pixels! I don't know much about text processing AIs but I can imagine their…

Is it fair to call this a robustness problem when you need access to the model to generate a failure case?

Many non-AI based systems lack robustness by the same standard (including humans)

Re: Beliefs that are true for regular software but false when applied to AI

#327

Post by 3abiton »

> nobody knows precisely what to do to ensure an AI writes formal emails correctly or summarises text accurately.

This is a bit of a hyperbole, a lot of the recent approaches rely on MoE, that are specialized. This makes it much more usuable for simple usecases.

Re: Beliefs that are true for regular software but false when applied to AI

#328

Post by blibble »

harrisonjackson wrote: post #218
teeray wrote: post #194

Earlier quoted context omitted.

> minor tools for making emojis, summarizing notifications, and proof reading. The notification / email summaries are so unbelievably useless too: it’s hardly more work to skim the notification / email that I do anyway.

The Ring app notification summaries still scare me. > "A bunch of people right outside your house!!!" because it aggregates multiple single person walking by notifications that way...

the advertising of those spy doorbells is entirely based on paranoia

so ramping it up the rhetoric doesn't really hurt them...

Re: Beliefs that are true for regular software but false when applied to AI

#329

Post by Gepsens »

mock-possum wrote: post #255
freetime2 wrote: post #122

For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…

Which is ironic, given all I really want from Siri is an advanced-voice-chat-level chat gpt experience - being able to carry on about 90% of a natural conversation with gpt, while Siri vacillates wildly between 1) simply not responding 2) misunderstanding and 3) understand but refusing to engage - feels awful .

That tells you AAPL didn't have the staff necessary to make this happen.

Re: Beliefs that are true for regular software but false when applied to AI

#330

Post by erichocean »

AdieuToLogic wrote: post #162

I found this statement particularly relevant: While it’s possible to demonstrate the safety of an AI for a specific test suite or a known threat, it’s impossible for AI creators to definitively say their AI will never act maliciously or dangerously for any prompt it could be given. This possibility is compounded exponentially when MCP[0] is used. 0 - https://github.com/modelcontextprotocol

> it’s impossible for AI creators to definitively say their AI will never act maliciously or dangerously for any prompt it could be given

This is false, AI doesn't "act" at all unless you, the developer, use it for actions. In which case it is you, the developer, taking the action.

Anthropomorphizing AI with terms like "malicious" when they can literally be implemented with a spreadsheet—first-order functional programming—and the world's dumbest while-loop to append the next token and restart the computation—should be enough to tell you there's nothing going on here beyond next token prediction.

Saying an LLM can be "malicious" is not even wrong, it's just nonsense.

Post reply on HN