Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

211–220 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#211

Post by AdieuToLogic »

mrkmarron wrote: post #180
AdieuToLogic wrote: post #162

I found this statement particularly relevant: While it’s possible to demonstrate the safety of an AI for a specific test suite or a known threat, it’s impossible for AI creators to definitively say their AI will never act maliciously or dangerously for any prompt it could be given. This possibility is compounded exponentially when MCP[0] is used. 0 - https://github.com/modelcontextprotocol

[flagged]

> The goal is to build a language and system model that allows us to reliably sandbox and support agents in constructing "Trustworthy-by-Construction AI Agents."

  1 - Reliability implies predictable behavior.
  2 - Predictable behavior implies determinism.
  3 - LLM's are non-deterministic algorithms.
In the link you kindly provided are phrases such as, "increases the likelihood of successful correct use" and "structure for the underlying LLM to key on", yet earlier state:

  In this world merely saying that a system is likely to 
  behave correctly is not sufficient.
Also, when describing "a suitable action language and specification system", what is detailed is largely, if not completely, available in RAML[0].

Are there API specification capabilities Bosque supports which RAML[0] does not? Probably, I don't know as I have no desire to adopt a proprietary language over a well-defined one supported by multiple languages and/or tools.

0 - https://github.com/raml-org/raml-spec/blob/master/versions/r...

Re: Beliefs that are true for regular software but false when applied to AI

#212

Post by eru »

remexre wrote: post #205
teeray wrote: post #194

Earlier quoted context omitted.

> minor tools for making emojis, summarizing notifications, and proof reading. The notification / email summaries are so unbelievably useless too: it’s hardly more work to skim the notification / email that I do anyway.

It does feel like somebody forgot that "from the first sentence or two of the email, you can tell what it's about" was already a rule of good writing...

You sometimes need to want to quickly learn what's in an email that was written by someone less helpful.

Eg sometimes the writer is outright antagonistic, because they have some obligation to tell you something, but don't actually want you to know.

Re: Beliefs that are true for regular software but false when applied to AI

#213

Post by chongli »

nancyminusone wrote: post #16
xutopia wrote: post #4

The most likely danger with AI is concentrated power, not that sentient AI will develop a dislike for us and use us as "batteries" like in the Matrix.

To me, the greatest threat is information pollution. Primary sources will be diluted so heavily in an ocean of generated trash that you might as well not even bother to look through any of it.

I see that as the death knell for general search engines built to indiscriminately index the entire web. But where that sort of search fails, opportunities open up for focused search and curated search.

Just as human navigators can find the smallest islands out in the open ocean, human curators can find the best information sources without getting overwhelmed by generated trash. Of course, fully manual curation is always going to struggle to deal with the volumes of information out there. However, I think there is a middle ground for assisted or augmented curation which exploits the idea that a high quality site tends to link to other high quality sites.

One thing I'd love is to be able to easily search all the sites in a folder full of bookmarks I've made. I've looked into it and it's a pretty dire situation. I'm not interested in uploading my bookmarks to a service. Why can't my own computer crawl those sites and index them for me? It's not exactly a huge list.

Re: Beliefs that are true for regular software but false when applied to AI

#214

Post by vel0city »

ponector wrote: post #134
malfist wrote: post #85

Earlier quoted context omitted.

Remember BSODs? Used to be a regular occurrence, now they're so infrequent they're gone from windows 11

I guess that is because you run it on old hardware. When I've bought my Asus ROG expensive laptop I had bsod almost daily. A year later with all updates I had bsod once in a month on the same device and windows installation.

If you have faulty hardware no amount of software is going to solve your problems (other than software that just completely deactivates said faulty hardware).

The fact you continued to have BSOD issues after a full reinstall is pretty strong evidence you probably had some kind of hardware failure.

Re: Beliefs that are true for regular software but false when applied to AI

#215

Post by w10-1 »

It's great to help people understand that AI can be both surprisingly good and disappointing, and that testing is the only way to know, but it's impossible to test everything. That sets expectations.

I think that means savvy customers will want details or control over testing, and savvy providers will focus on solutions they can validate, or where testing is included in the workflow (e.g., code), or where precision doesn't matter (text and meme generation). Knowing that in depth is gold for AI advocates.

Otherwise, I don't think people really know or care about bugs or specifications or how AI breaks prior programmer models.

But people will become very hostile and demand regulatory frenzies if AI screws things up (e.g., influencing elections or putting people out of work). Then no amount of sympathy or understanding will help the industry, which has steadily been growing its capability for evading regulation via liability disclaimers, statutory exceptions, arbitration clauses, pitting local/regional/national governments against each other, etc.

To me that's the biggest risk: we won't get the benefits and generational investments will be lost in cleaning up after a few (even accidental) bad actors at scale.

Re: Beliefs that are true for regular software but false when applied to AI

#216

Post by vel0city »

Yoric wrote: post #105
noir_lord wrote: post #75

Earlier quoted context omitted.

Agreed. Or debuggers that would take out the entire OS. Or a bad driver crashing everything multiple times a week. Or a misbehaving process not handing control back to the OS. I grew up in the era of 8 and 16 bit micros and early PCs, they where hilariously less stable than modern machines while doing far less, there wasn’t some halcyon age of near perfect software, it’s always been a case of things been good enough…

I grew up in the same era and I recall crashes being less frequent. There were plenty of other issues, including the fact that you had to adjust the right IRQ and DMA for your Sound Blaster manually, both physically and in each game, or that you needed to "optimize" memory usage, enable XMS or EMS or whatever it was at the time, or that you spent hours looking at the nice defrag/diskopt playing with your files, etc.…

It truly depends on the quality of the software you were using at the time. Maybe the software you used didn't result in many issues. I know a lot of the games I played as a kid on my family's or friend's Win95 machines resulted in system lockups or blue screens practically every time we used them.

As I mess around with these old machines for fun in my free time, I encounter these kinds of crashes pretty dang often. Its hard to tell if its just the old hardware is broken in odd ways or not so I can't fully say its the old software, but things are definitely pretty unreliable on old desktop Windows running old desktop Windows apps.

Re: Beliefs that are true for regular software but false when applied to AI

#217

Post by AdieuToLogic »

someothherguyy wrote: post #204
mrkmarron wrote: post #180

Earlier quoted context omitted.

[flagged]

why make a new language? are there no existing languages comprehensive enough for this?

> are there no existing languages comprehensive enough for this?

In my experience, RAML[0] is worth adopting as an API specification language. It is superior to Swagger/OpenAPI in both being able to scale in complexity and by supporting modularity as a first class concept:

  RAML provides several mechanisms to help modularize
  the ecosystem of an API specification:

    Includes
    Libraries
    Overlays
    Extensions[1]
0 - https://github.com/raml-org/raml-spec/blob/master/versions/r...

1 - https://github.com/raml-org/raml-spec/blob/master/versions/r...

Re: Beliefs that are true for regular software but false when applied to AI

#218

Post by harrisonjackson »

teeray wrote: post #194
freetime2 wrote: post #122

For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…

> minor tools for making emojis, summarizing notifications, and proof reading. The notification / email summaries are so unbelievably useless too: it’s hardly more work to skim the notification / email that I do anyway.

The Ring app notification summaries still scare me.

> "A bunch of people right outside your house!!!"

because it aggregates multiple single person walking by notifications that way...

Re: Beliefs that are true for regular software but false when applied to AI

#219

Post by disqard »

harrisonjackson wrote: post #218
teeray wrote: post #194

Earlier quoted context omitted.

> minor tools for making emojis, summarizing notifications, and proof reading. The notification / email summaries are so unbelievably useless too: it’s hardly more work to skim the notification / email that I do anyway.

The Ring app notification summaries still scare me. > "A bunch of people right outside your house!!!" because it aggregates multiple single person walking by notifications that way...

That is a fantastic example of blind application of AI making things worse.

Re: Beliefs that are true for regular software but false when applied to AI

#220

Post by jayd16 »

kstrauser wrote: post #65
kazinator wrote: post #2

> AIs will get more reliable over time, like old software is more reliable than new software. :) Was that a humam Freudian slip, or artificial one? Yes, old software is often more reliable than new.

Holy survivorship bias, Batman. If you think modern software is unreliable, let me introduce you to our friend, Rational Rose.

You misunderstand. They are explicitly referring to the survivors that have been iterated on and chosen for being good.

They're NOT saying all software in the past was better.

Post reply on HN