Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

171–180 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#171

Post by pezezin »

Yoric wrote: post #105
noir_lord wrote: post #75

Earlier quoted context omitted.

Agreed. Or debuggers that would take out the entire OS. Or a bad driver crashing everything multiple times a week. Or a misbehaving process not handing control back to the OS. I grew up in the era of 8 and 16 bit micros and early PCs, they where hilariously less stable than modern machines while doing far less, there wasn’t some halcyon age of near perfect software, it’s always been a case of things been good enough…

I grew up in the same era and I recall crashes being less frequent. There were plenty of other issues, including the fact that you had to adjust the right IRQ and DMA for your Sound Blaster manually, both physically and in each game, or that you needed to "optimize" memory usage, enable XMS or EMS or whatever it was at the time, or that you spent hours looking at the nice defrag/diskopt playing with your files, etc.…

You are looking back with rosy glasses indeed.

Last year I assembled a retro PC (Pentium 2, Riva TNT 2 Ultra, Sound Blaster AWE64 Gold) running Windows 98 to remember my childhood, and it is more stable than what I remembered, but still way worse than modern systems. There are plenty of games that will refuse to work for whatever reason, or that will crash the whole OS, specially when existing, and require a hard reboot.

Oh and at least in the '90s you could already ship patches, we used to get them with the floppies and later CDs provided by magazines.

Re: Beliefs that are true for regular software but false when applied to AI

#172

Post by api »

alfalfasprout wrote: post #156
freetime2 wrote: post #122

For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…

> I think Apple execs genuinely underestimated how difficult it would be to get LLMs to perform up to Apple's typical standards of polish and control Not only Apple, this is happening across the industry. Executives' expectations of what AI can deliver are massively inflated by Amodei et al. essentially promising human-level cognition with every release. The reality is aside from coding assistants and chatbot interfa…

Standard hype cycle. We are probably creating the top of the peak of inflated expectations.

Re: Beliefs that are true for regular software but false when applied to AI

#173

Post by marcosdumay »

skywhopper wrote: post #121

Not the point, but I’m confused by the Geoguessr screenshot. Under the reasoning for its decision, it mentions “traffic keeps to the left” but that is not apparent from the photo. Then it says the shop sign looks like a “Latin alphabet business name rather than Spanish or Portuguese”. Uhhh… what? Spanish and Portuguese use the Latin alphabet.

It's an LLM.

It decided on the first line first (the place name), and then made the reasons on the rest of the text.

So the answer is more important on the justifications than the actual picture, and the reasoning that led it there doesn't enter the frame at all.

Re: Beliefs that are true for regular software but false when applied to AI

#174

Post by DonHopkins »

protocolture wrote: post #166
__loam wrote: post #140

Earlier quoted context omitted.

I'm happy they ate shit here because I like my mac not getting co-pilot bullshit forced into it, but apparently Apple had two separate teams competing against each other on this topic. Supposedly a lot of politics got in the way of delivering on a good product combined with the general difficulty of building LLM products.

Mac LLM vs Lisa LLM?

Apple ][ LLM Forever!

https://paleotronic.com/2025/08/03/connect-ai-to-microm8-app...

Re: Beliefs that are true for regular software but false when applied to AI

#175

Post by giancarlostoro »

kube-system wrote: post #67
giancarlostoro wrote: post #25

Earlier quoted context omitted.

Age of Code does not automatically equal quality of code, ever. Good code is maintained by good developers. A lot of bad code is pushed out by management, and other situations, or just bad devs. This is a can of worms you're talking your way into.

The author didn't mean that an older commit date on a file makes code better. The author is talking about the maturity of a project. Likewise, as AI technologies become more mature we will have more tools to use them in a safer and more reliable way.

I've seen too many old projects that are not by any means better no matter how much they get updates because management define priorities. I'm not alone in saying I've been in a few projects where the backlog is rather large. When your development is driven by marketing people trying to pump up sales, all the "non critical" bugs begin to stack up.

Re: Beliefs that are true for regular software but false when applied to AI

#176

Post by cantrevealname »

Gigachad wrote: post #155
cantrevealname wrote: post #146

Earlier quoted context omitted.

This sci-fi thing goes as far back as the 1983 movie WarGames, where they wanted to pull the plug on a rogue computer, but there was a reason you couldn’t do that: McKittrick: General, the machine has locked us out. It's sending random numbers to the silos. Pat Healy: Codes. To launch the missiles. General Beringer: Just unplug the goddamn thing! Jesus Christ! McKittrick: That won't work, General. It would interpret…

In real life it won’t be that the computer prevents you from turning it off. It’ll be that the computer is guarded by cultists who think its god, and unstoppable market forces that require it to keep running.

When AI ends up running everything essential to survival and society, it’ll be preposterous to even suggest pulling the plug just because it does something bad.

Can you imagine the chaos of completely turning off GPS or Gmail today? Now imagine pulling the plug on something in the near future that controls all electric power distribution, banking communications, and Internet routing.

Re: Beliefs that are true for regular software but false when applied to AI

#177

Post by kstrauser »

lossyalgo wrote: post #163
binarymax wrote: post #94

Earlier quoted context omitted.

You know, I had spent a good amount of years not having even a single thought about rational rose, and now that’s all over.

It could definitely be worse. I have the privilege of using it weekly :(

What? How? I thought we stamped it out in the Purge of 2007.

Re: Beliefs that are true for regular software but false when applied to AI

#178

Post by alwa »

genghisjahn wrote: post #167
__loam wrote: post #140

Earlier quoted context omitted.

I'm happy they ate shit here because I like my mac not getting co-pilot bullshit forced into it, but apparently Apple had two separate teams competing against each other on this topic. Supposedly a lot of politics got in the way of delivering on a good product combined with the general difficulty of building LLM products.

Apparently? From what? Where did this information come from that they had two competing teams?

I feel like I hear people referring to Wayne Ma’s reporting for The Information to that effect.

https://www.theinformation.com/articles/apple-fumbled-siris-...

> Distrust between the two groups got so bad that earlier this year one of Giannandrea’s deputies asked engineers to extensively document the development of a joint project so that if it failed, Federighi’s group couldn’t scapegoat the AI team.

> It didn’t help the relations between the groups when Federighi began amassing his own team of hundreds of machine-learning engineers that goes by the name Intelligent Systems and is run by one of Federighi’s top deputies, Sebastien Marineau-Mes.

Re: Beliefs that are true for regular software but false when applied to AI

#179

Post by dylan604 »

krior wrote: post #106
malfist wrote: post #85

Earlier quoted context omitted.

Remember BSODs? Used to be a regular occurrence, now they're so infrequent they're gone from windows 11

Gone? I had two last year, lets not overstate things.

Daily+ occurrences to two in a year pretty much rounds to zero. Kind of like we said measles were eradicated because there was <X per year cases.

Re: Beliefs that are true for regular software but false when applied to AI

#180

Post by mrkmarron »

AdieuToLogic wrote: post #162

I found this statement particularly relevant: While it’s possible to demonstrate the safety of an AI for a specific test suite or a known threat, it’s impossible for AI creators to definitively say their AI will never act maliciously or dangerously for any prompt it could be given. This possibility is compounded exponentially when MCP[0] is used. 0 - https://github.com/modelcontextprotocol

[flagged]
Post reply on HN