Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

291–300 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#291

Post by jeremyscanvic »

The part about AI being very sensitive to small perturbations of their input is actually a very active research topic (and coincidentally the subject of my PhD). Most vision AIs suffer from poor spatial robustness [1], you can drastically lower their accuracy simply by translating the inputs by well-chosen (adversarial) translations of a few pixels! I don't know much about text processing AIs but I can imagine their semantic robustness is also studied.

[1] https://arxiv.org/abs/1712.02779

Edit: typo

Re: Beliefs that are true for regular software but false when applied to AI

#292

Post by 1718627440 »

eru wrote: post #212
remexre wrote: post #205

Earlier quoted context omitted.

It does feel like somebody forgot that "from the first sentence or two of the email, you can tell what it's about" was already a rule of good writing...

You sometimes need to want to quickly learn what's in an email that was written by someone less helpful. Eg sometimes the writer is outright antagonistic, because they have some obligation to tell you something, but don't actually want you to know.

I think humans are quite well capable of skimming text and reading multiple lines at once.

Re: Beliefs that are true for regular software but false when applied to AI

#293

Post by mattmanser »

ako wrote: post #273
mattmanser wrote: post #249

Earlier quoted context omitted.

That's a bit like saying knowing how a pipe works is enough to explain a combustion engine. You're just listing part of how LLMs work. Those mechanisms only explain next word prediction, not LLM reasoning. That's an emergent property that no person, as far as I understand it, can explain past hand waving. Happy to be corrected here.

There's no magic involved, the LLM creators can go anywhere and rebuild an LLM with pretty much the same outcome, if they have the same training data. With unlimited time you could even reproduce the output of an LLM manually as it is just a lot of mathematics. Including reasoning, as that is mostly adding words in the context that will steer the word predication to include reasoning. As this is a useful LLM behavior…

Yes, but we can't inspect, reproduce or explain the emergent property independently. We can't pick out the "math reasoning" part or the "programming" part, or inspect how it's working, or selectively change any of it. You can't turn any dials or twiddle any knobs. You can't replace one part with another, or pick out components. You can't peek inside and say:

"hey it's got an irrational preference for naming its variables after famous viking warriors, lets change that!"

But worse, it's not that you can't change it, you just don't know! All you can do is test it and guess its biases.

Is it racist, is it homophobic, is it misogynistic? There was an article here the other day about AI in recruitment and the hidden biases. And there was a recruitment AI that only picked men for a role. The job spec was entirely gender neutral. And they hadn't noticed until a researcher looked at it.

It's a black box. So if it does something incorrectly, all they can do is retrain and hope.

Again, this is my present understanding of how it all works right now.

Re: Beliefs that are true for regular software but false when applied to AI

#294

Post by throw-10-13 »

themanmaran wrote: post #18

> Because eventually we’ll iron out all the bugs so the AIs will get more reliable over time Honestly this feels like a true statement to me. It's obviously a new technology, but so much of the "non-deterministic === unusable" HN sentiment seems to ignore the last two years where LLMs have become 10x as reliable as the initial models.

I read this as "10x better at generating code that looks correct but hides nasty bugs behind a facade of sane looking slop".

Re: Beliefs that are true for regular software but false when applied to AI

#295

Post by Ray20 »

bobsmooth wrote: post #208
surgical_fire wrote: post #80

Earlier quoted context omitted.

"AI will take over the world". I hear that. Then I try to use AI for simple code task, writing unit tests for a class, very similar to other unit tests. If fails miserably. Forgets to add an annotation and enters in a death loop of bullshit code generation. Generates test classes that tests failed test classes that test failed test classes and so on. Fascinating to watch. I wonder how much CO2 it generated while fryi…

Most reasonable AI alarmists are not concerned with sentient AI but an AI attached to the nukes that gets into one of those repeating death loops and fires all the missiles.

In reality, this isn't a very serious threat. Rather, we're concerned about AI as a tool for strengthening totalitarian regimes.

Re: Beliefs that are true for regular software but false when applied to AI

#296

Post by Deestan »

skywhopper wrote: post #121

Not the point, but I’m confused by the Geoguessr screenshot. Under the reasoning for its decision, it mentions “traffic keeps to the left” but that is not apparent from the photo. Then it says the shop sign looks like a “Latin alphabet business name rather than Spanish or Portuguese”. Uhhh… what? Spanish and Portuguese use the Latin alphabet.

What is 12+12?

> The answer is 24! See the ASCII values of '1' is 49, '2' is 50, and '+' is 43. Adding all that together we get 3. Now since we are doing this on a computer with a 8-bit infrastructure we multiply by 3 and so the answer is 24.

Cool! I didn't understand any of that but it was correct and you sound smart. I will put this thing in charge of critical parts of my business.

Re: Beliefs that are true for regular software but false when applied to AI

#297

Post by ben_w »

Pulcinella wrote: post #202
ben_w wrote: post #127

Earlier quoted context omitted.

Sure, but also the METR study showed the rate of change is t doubles every 7 months where t ~= «duration of human time needed to complete a task, such that SOTA AI can complete same with 50% success»: https://arxiv.org/pdf/2503.14499 I don't know how long that exponential will continue for, and I have my suspicions that it stops before week-long tasks, but that's the trend-line we're on.

But will it actually get better or will it just get faster and more power efficient at failing to pair parentheses/braces/brackets/quotes?

Read the linked METR study please.

Or watch the Computerphile video summary/author interview, if you prefer: https://m.youtube.com/watch?v=evSFeqTZdqs

Re: Beliefs that are true for regular software but false when applied to AI

#298

Post by crottypeter »

noir_lord wrote: post #75
kstrauser wrote: post #65

Earlier quoted context omitted.

Holy survivorship bias, Batman. If you think modern software is unreliable, let me introduce you to our friend, Rational Rose.

Agreed. Or debuggers that would take out the entire OS. Or a bad driver crashing everything multiple times a week. Or a misbehaving process not handing control back to the OS. I grew up in the era of 8 and 16 bit micros and early PCs, they where hilariously less stable than modern machines while doing far less, there wasn’t some halcyon age of near perfect software, it’s always been a case of things been good enough…

But at the time that software was "new" and unreliable.

Re: Beliefs that are true for regular software but false when applied to AI

#299

Post by mikkupikku »

remexre wrote: post #205
teeray wrote: post #194

Earlier quoted context omitted.

> minor tools for making emojis, summarizing notifications, and proof reading. The notification / email summaries are so unbelievably useless too: it’s hardly more work to skim the notification / email that I do anyway.

It does feel like somebody forgot that "from the first sentence or two of the email, you can tell what it's about" was already a rule of good writing...

Maybe they remembered that a lot of people aren't actually good writers. My brother will send 1000 word emails that meander through subjects like what he ate for breakfast to eventually get to the point of scheduling a meeting about negotiating a time for help with moving a sofa. Mind you, I see him several times a week so he's not lonely, this is just the way he writes. Then he complains endlessly about his coworkers using AI to summarize his emails. When told that he needs to change how he writes to cut right to the point, he adopts the "why should I change, they're the ones who suck" mentality.

So while Apple's AI summaries may have been poorly executed, I can certainly understand the appeal and motivation behind such a feature.

Re: Beliefs that are true for regular software but false when applied to AI

#300

Post by lossyalgo »

kstrauser wrote: post #177
lossyalgo wrote: post #163

Earlier quoted context omitted.

It could definitely be worse. I have the privilege of using it weekly :(

What? How? I thought we stamped it out in the Purge of 2007.

Some things are forged in hell and refuse to die.
Post reply on HN