Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

121–130 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#121
Not the point, but I’m confused by the Geoguessr screenshot. Under the reasoning for its decision, it mentions “traffic keeps to the left” but that is not apparent from the photo.

Then it says the shop sign looks like a “Latin alphabet business name rather than Spanish or Portuguese”. Uhhh… what? Spanish and Portuguese use the Latin alphabet.

Re: Beliefs that are true for regular software but false when applied to AI

#122
For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a while for being wildly "out of control". [1] And in this year's iPhone launch the AI marketing was toned down significantly.

I think Apple execs genuinely underestimated how difficult it would be to get LLMs to perform up to Apple's typical standards of polish and control.

[1] https://www.bbc.com/news/articles/cge93de21n0o

Re: Beliefs that are true for regular software but false when applied to AI

#123
post #15

Earlier quoted context omitted.

The reality is that the CEO/executive class already has developed a dislike for us and is trying to use us as “batteries” like in the Matrix.

CEOs (even most VCs) are labor too

Almost everyone is "labor" to some extent. There is always a huge customer or major investor that you are beholden to. If you are independently wealthy then you are the exception.

Re: Beliefs that are true for regular software but false when applied to AI

#124
post #117
post #114

Earlier quoted context omitted.

It's an unhelpful one because it implies the danger is somehow the result of irrational or impulsive thought, and making the AI smarter will avoid it.

That's not how I read it. Perhaps because most of the smartest people I know are regularly irrational or impulsive :)

I think most people don't get that; look at how often even Star Trek script writers write Straw Vulcans*.

* https://tvtropes.org/pmwiki/pmwiki.php/Main/StrawVulcan

Re: Beliefs that are true for regular software but false when applied to AI

#125
post #7
post #4

The most likely danger with AI is concentrated power, not that sentient AI will develop a dislike for us and use us as "batteries" like in the Matrix.

I'm not so sure it will be that either, it would be having multiple AIs essentially at war with each other over access to GPUs/energy or whatever the materials are needed to grow if/when that happens. We will end up as pawns in this conflict.

Given that even fairly mediocre human intelligences can run countries into the ground and avoid being thrown out in the process, it's certainly possible for an AI to be in the intelligence range where it's smart enough to win vs humans but also dumb enough to turn us into pawns rather just go to space and blot out the sun with a Dyson swarm made from the planet Mercury.

But don't count on it.

I mean, apart from anything else, that's still a bad outcome.

Re: Beliefs that are true for regular software but false when applied to AI

#126
post #4

The most likely danger with AI is concentrated power, not that sentient AI will develop a dislike for us and use us as "batteries" like in the Matrix.

The power concentration is already massive, and a huge problem indeed. The ai is just a cherry on top. The ai is not the problem.

Re: Beliefs that are true for regular software but false when applied to AI

#127
post #87

Earlier quoted context omitted.

"AI will take over the world". I hear that. Then I try to use AI for simple code task, writing unit tests for a class, very similar to other unit tests. If fails miserably. Forgets to add an annotation and enters in a death loop of bullshit code generation. Generates test classes that tests failed test classes that test failed test classes and so on. Fascinating to watch. I wonder how much CO2 it generated while fryi…

Regularly trying to use LLMs to debug coding issues has convinced me that we're _nowhere_ close to the kind of AGI some are imagining is right around the corner.

Sure, but also the METR study showed the rate of change is t doubles every 7 months where t ~= «duration of human time needed to complete a task, such that SOTA AI can complete same with 50% success»: https://arxiv.org/pdf/2503.14499

I don't know how long that exponential will continue for, and I have my suspicions that it stops before week-long tasks, but that's the trend-line we're on.

Post reply on HN