Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Keep your AI claims in check

ftc.gov

Post reply on HN
Print view View original

301–308 of 308 posts

Re: Keep your AI claims in check

#301

Post by pyuser583 »

flatline wrote: post #36

“It’s an ambiguous term with many possible definitions.” “Does the product actually use AI at all? If you think you can get away with baseless claims…” Last I checked basic optimization techniques like simulated annealing and gradient descent - as well as a host of basic statistical tools - are standard parts of an introductory AI textbook. I’ve been on the receiving end of government agency enforcement (SEC) and it…

Yeah there are company’s that just use old fashioned data scrubbing and ETLs that claim to be AI.

Re: Keep your AI claims in check

#302

Post by yccs27 »

y2kenny wrote: post #287
yccs27 wrote: post #285

Earlier quoted context omitted.

The big difference is that data scientists only work on the model architecture and data sources, whereas expert systems need people who have expertise in the subject matter itself . One of the biggest changes from 'old AI' to modern ML is that we no longer try to use human domain knowledge as much, instead getting the model itself to see the same pattern from data.

Isn't labeling still an important part of modern AI?

I'd say labels (for supervised ML) are fundamentally different from rules (for expert systems), because

  - labels are easy to decide in many cases
  - rules require humans to analyze patterns in the problem space
  - labels only concern each data point individually
  - rules generalize over a class of data points

Re: Keep your AI claims in check

#303

Post by civilized »

p-e-w wrote: post #300
inkysigma wrote: post #255

Earlier quoted context omitted.

It's almost nearly always wrong in my experience. Like if I ask it a basic technical question, it gets stuff mostly correct but if you look too much at the details, then errors start popping up. If you ask it something relatively complicated, it's nearly always wrong. Like as an easy example of the latter, I asked chatGPT "What is the S4 model in NLP?" and it gave back > The S4 model (Simple Synchronous Syntactic-Sem…

> "What is the S4 model in NLP?" 99.99% of people would be unable to answer that question (without looking it up, I mean). Such hyper-specific queries for highly technical information from niche fields say very little about the model's overall performance at natural language tasks. If you ask things like "Which of these animals doesn't live in Africa?" or "What is the most reactive chemical element?", ChatGPT's answe…

We already had Watson for Jeopardy-style general knowledge quiz questions a decade ago. It didn't revolutionize anything.

Re: Keep your AI claims in check

#304

Post by civilized »

civilized wrote: post #303
p-e-w wrote: post #300

Earlier quoted context omitted.

> "What is the S4 model in NLP?" 99.99% of people would be unable to answer that question (without looking it up, I mean). Such hyper-specific queries for highly technical information from niche fields say very little about the model's overall performance at natural language tasks. If you ask things like "Which of these animals doesn't live in Africa?" or "What is the most reactive chemical element?", ChatGPT's answe…

We already had Watson for Jeopardy-style general knowledge quiz questions a decade ago. It didn't revolutionize anything.

Update. This morning I asked ChatGPT what day today was. It answered correctly. I then asked how it could know that given that its training data ends in September 2021. It said it was based on the number of days since its training data ended. I pointed out it still had no way of knowing that number of days if it had no knowledge past September 2021. It kept apologizing and repeating the same story over and over.

Re: Keep your AI claims in check

#305

Post by strangemonad »

klyrs wrote: post #190
mathgenius wrote: post #176

Earlier quoted context omitted.

> Deterministic if/then statements This was the pinnacle of AI in the 80's. They called them "expert systems".

Principle problem being that expert systems required meticulous inputs from domain experts, codified by skilled engineers. People don't have time or startup capital for actual expertise...

Which is very much unlike today’s training set inputs meticulously labeled by domain experts and curated by engineers?

Re: Keep your AI claims in check

#307

Post by nfgrep »

Red_Leaves_Flyy wrote: post #185
nfgrep wrote: post #102

Earlier quoted context omitted.

Not ominous, threatening, and somewhat juvenile IMO.

What are ramifications of false advertising of AI capabilities in medicine? Policing? Mass surveillance? Education? Traffic control? Mass transit? Farming? Shipping? Law? News? Death. Mass death. Poverty. Disease. Mass destruction. Famine. Brush away these warnings to move fast and break things if you (Royal you) want. Hopefully, you won’t cause mass suffering, but if you do I hope you’re made an example of. When the…

I dont disagree, I was expressing that I find their tone surprisingly “yee haw, we’ll getcha’” for an org that wields such power and responsibility.

Re: Keep your AI claims in check

#308

Post by NateEag »

arionhardison wrote: post #232
NateEag wrote: post #178

Earlier quoted context omitted.

Your description sounds like you're training your reviewers to rubberstamp the system's results as correct. That is a horrible outcome. By the time something comes along they really need to question, they'll already have adapted to the system being pretty good and they won't even notice there was an issue. This problem is endemic to ML systems and cannot be patched away, so far as I can tell. We need fundamentally di…

Not rubber stamp, they can approve or reject the suggestion.

I didn't mean you were forcing them to approve it.

I meant that, over time, your system will become right more and more often, not by virtue of genuinely understanding the situations, but because it has more training data and comes up with correlations that happen to be correct.

Once it's mostly-kinda-reliable, your users will, over time, stop paying attention to it and just start trusting it, even if they don't mean to, because that's how humans work.

And then it becomes a rubberstamp, and outcomes start getting generated where no human really thought it through.

Post reply on HN