Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

351–360 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#351

Post by jvanderbot »

8organicbits wrote: post #346
jvanderbot wrote: post #319

Earlier quoted context omitted.

We should move well beyond human analogies. I have never met a human that would straight up lie about something, or build up so much deceptive tests that it might as well be lying. Granted this is not super common in these tools, but it is essentially unheard of in junior devs.

> I have never met a human that would straight up lie about something This doesn't match my experience. Consider high profile things like the VW emissions scandal, where the control system was intentionally programmed to only engage during the emissions test. Dictators. People are prone to lie when it's in their self interest, especially for self preservation. We have entire structures of government, courts, that try…

There is an enormous amount of difference between planned deception as part of a product, and undermining your own product with deceptive reporting about its quality. The difference is collaboration and alignment. You might have evil goals, but if your developers are maliciously incompetent, no goal will be accomplished.

Re: Beliefs that are true for regular software but false when applied to AI

#353

Post by tremon »

nedt wrote: post #316
AdieuToLogic wrote: post #162

I found this statement particularly relevant: While it’s possible to demonstrate the safety of an AI for a specific test suite or a known threat, it’s impossible for AI creators to definitively say their AI will never act maliciously or dangerously for any prompt it could be given. This possibility is compounded exponentially when MCP[0] is used. 0 - https://github.com/modelcontextprotocol

Yeah in that regard we should always treat it like a junior something. Very much like you can't expect your own kids to never do something dangerous even if tell it for years to be careful. I got used to getting my kid from the Kindergarten with a new injury at least once a month.

I think it's very dangerous to use the term "junior" here because it implies growth potential, where in fact it's the opposite: you are using a finished product, it won't get any better. AI is an intern, not a junior. All the effort you're spending into correcting it will leave the company, either as soon as you close your browser or whenever the manufacturer releases next year's model -- and that model will be better regardless of how much time you waste on training this year's intern, so why even bother? Thinking of AI as a junior coworker is probably the least productive way of looking at it.

Re: Beliefs that are true for regular software but false when applied to AI

#354

Post by vladms »

whatevertrevor wrote: post #263
vladms wrote: post #84

Earlier quoted context omitted.

> Any programming or mathematical question has several correct answers. Huh? If I need to sort the list of integer number of 3,1,2 in ascending order the only correct answer is 1,2,3. And there are multiple programming and mathematical questions with only one correct answer. If you want to say "some programming and mathematical questions have several correct answers" that might hold.

I think what they meant is something along the lines of: - In Math, there's often more than one logically distinct way of proving a theorem, and definitely many ways of writing the same proof, though the second applies more to handwritten/text proofs than say a proof in Lean. - In programming, there's often multiple algorithms to solve a problem correctly (in the mathematical sense, optimality aside), and for the sam…

I find it quite ironical that while discussing the topic of logic and correct answers the OP talks rather "approximately" leaving the reader to imagine what he meant and others (like you) to spell it out.

Yes, I thought as well of your interpretation, but then I read the text again, and it really does not say that, so I choose to answer to the text...

Re: Beliefs that are true for regular software but false when applied to AI

#356

Post by kelnos »

teeray wrote: post #194
freetime2 wrote: post #122

For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…

> minor tools for making emojis, summarizing notifications, and proof reading. The notification / email summaries are so unbelievably useless too: it’s hardly more work to skim the notification / email that I do anyway.

I find it weird that we even think we need notification summaries. If the notification body text is long or complex enough to benefit from summarizing, then the person who wrote that text has failed at the job. Notifications are summaries.

Re: Beliefs that are true for regular software but false when applied to AI

#357

Post by xp84 »

freetime2 wrote: post #122

For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…

> get LLMs to perform up to Apple's typical standards of polish and control.

I reject this spin (which is the Apple PR explanation for their failure). LLMs already do far better than Apple’s 2025 standards of polish. Contrast things built outside Apple. The only thing holding Siri back is Apple’s refusal to build a simple implementation where they expose the APIs to “do phone things” or “do home things” as a tool call to a plain old LLM (or heck, build MCP so LLM can control your device). It would be straightforward for Apple to negotiate with a real AI company to guarantee no training on the data, etc. the same way that business accounts on OpenAI etc. offer. It might cost Apple a bunch of money, but fortunately they have like 1000 bunches of money.

Re: Beliefs that are true for regular software but false when applied to AI

#358

Post by emp17344 »

mattmanser wrote: post #293
ako wrote: post #273

Earlier quoted context omitted.

There's no magic involved, the LLM creators can go anywhere and rebuild an LLM with pretty much the same outcome, if they have the same training data. With unlimited time you could even reproduce the output of an LLM manually as it is just a lot of mathematics. Including reasoning, as that is mostly adding words in the context that will steer the word predication to include reasoning. As this is a useful LLM behavior…

Yes, but we can't inspect, reproduce or explain the emergent property independently. We can't pick out the "math reasoning" part or the "programming" part, or inspect how it's working, or selectively change any of it. You can't turn any dials or twiddle any knobs. You can't replace one part with another, or pick out components. You can't peek inside and say: "hey it's got an irrational preference for naming its varia…

It’s not clear that those “emergent properties” even exist.

Re: Beliefs that are true for regular software but false when applied to AI

#359

Post by beyarkay »

freetime2 wrote: post #122

For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…

Apple is a good example. I kinda still can't believe they've done basically nothing, despite investing so heavily in apple silicon and MLX.

Also kinda crazy that all the "native" voice assistants are still terrible, despite the tech having been around for years by now.

Re: Beliefs that are true for regular software but false when applied to AI

#360

Post by beyarkay »

xp84 wrote: post #357
freetime2 wrote: post #122

For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…

> get LLMs to perform up to Apple's typical standards of polish and control. I reject this spin (which is the Apple PR explanation for their failure). LLMs already do far better than Apple’s 2025 standards of polish. Contrast things built outside Apple. The only thing holding Siri back is Apple’s refusal to build a simple implementation where they expose the APIs to “do phone things” or “do home things” as a tool cal…

I could also imagine that Apple execs might be too proud to use someone else's AI, and so wanted to train their own from scratch, but ultimately failed to do this. Totally agree that this smells like a people failure rather than a technology failure
Post reply on HN