Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Keep your AI claims in check

ftc.gov

Post reply on HN
Print view View original

251–260 of 308 posts

Re: Keep your AI claims in check

#251

Post by graypegg »

flatline wrote: post #36

“It’s an ambiguous term with many possible definitions.” “Does the product actually use AI at all? If you think you can get away with baseless claims…” Last I checked basic optimization techniques like simulated annealing and gradient descent - as well as a host of basic statistical tools - are standard parts of an introductory AI textbook. I’ve been on the receiving end of government agency enforcement (SEC) and it…

I’m sure there’s a company out there who uses some linear equation in their app that they came up with by dumping the data they had into Excel and running the linear regression “data analysis” on it.

> “Before labeling your product as AI-powered, note also that merely using an AI tool in the development process is not the same as a product having AI in it.”

So is y=7.4x+5 an “AI” running inside our app or is it just the output from an “AI tool” FTC?

Re: Keep your AI claims in check

#252

Post by KRAKRISMOTT »

graypegg wrote: post #251
flatline wrote: post #36

“It’s an ambiguous term with many possible definitions.” “Does the product actually use AI at all? If you think you can get away with baseless claims…” Last I checked basic optimization techniques like simulated annealing and gradient descent - as well as a host of basic statistical tools - are standard parts of an introductory AI textbook. I’ve been on the receiving end of government agency enforcement (SEC) and it…

I’m sure there’s a company out there who uses some linear equation in their app that they came up with by dumping the data they had into Excel and running the linear regression “data analysis” on it. > “Before labeling your product as AI-powered, note also that merely using an AI tool in the development process is not the same as a product having AI in it.” So is y=7.4x+5 an “AI” running inside our app or is it just…

Replace x and y with matrices and wrap everything in a non-linearity. Swap the 7.4 and 5 constants for variables a and b and set their values by taking the partial derivative of the with respect to the difference between the ground truth value and the predicted y.

String together a bunch of these "smart cells" and observe that we can process sequences of data by linking the cells together. Further observe that if we have a separate set of cells (technically it's an attention vector, not quite a group of neurons) whose loss function is with respect to each individual token in the sequences, we can "pay attention" to specific segments in the sequences.

Add a few more gates and feedback loops, scale up the number of cells to 10^12, and you basically have a state of the art chatbot. Capiche?

Re: Keep your AI claims in check

#253

Post by Too »

flatline wrote: post #36

“It’s an ambiguous term with many possible definitions.” “Does the product actually use AI at all? If you think you can get away with baseless claims…” Last I checked basic optimization techniques like simulated annealing and gradient descent - as well as a host of basic statistical tools - are standard parts of an introductory AI textbook. I’ve been on the receiving end of government agency enforcement (SEC) and it…

Given the ambiguity of the term it would actually be better if FTC didn’t step in at all. To let the term dilute itself in its own marketing, to the point where consumers don’t care about it at all or actively avoid products with “AI”.

Re: Keep your AI claims in check

#254

Post by visarga »

mr_toad wrote: post #141
visarga wrote: post #127

Earlier quoted context omitted.

Actually, if you ask GPT3 to explain step by step how it arrived to the conclusion, it works pretty well and even has higher accuracy.

GPT is making it up. It’s cobbling together an explanation from texts that explain how to solve a problem. It’s like a student who copies an answer, then copies the explanation as well.

Chain-of-thought is not the same with direct prompting. Step by step is better.

> Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. The empirical gains can be striking.

https://arxiv.org/abs/2201.11903

Re: Keep your AI claims in check

#255

Post by inkysigma »

p-e-w wrote: post #235
civilized wrote: post #96

Earlier quoted context omitted.

GPT is a voluble, entertaining bullshitter that is occasionally correct. It is not reliable, and for now, it cannot be made reliable. It is a very impressive toy, but still just a toy for now.

> that is occasionally correct It's almost always correct, otherwise it would be worthless. Yes, it's possible to construct questions that lead to nonsensical answers, and sometimes nonsensical answers are given even to sensible questions, but saying that ChatGPT's answers are "occasionally" correct is weapons grade BS. ChatGPT is a hair's breadth from being an ultimate answer machine, and is far more likely to be co…

It's almost nearly always wrong in my experience. Like if I ask it a basic technical question, it gets stuff mostly correct but if you look too much at the details, then errors start popping up. If you ask it something relatively complicated, it's nearly always wrong.

Like as an easy example of the latter, I asked chatGPT "What is the S4 model in NLP?" and it gave back

> The S4 model (Simple Synchronous Syntactic-Semantic Sentence Simplification Model) is a model in natural language processing (NLP) that aims to simplify complex sentences while preserving their original meaning. The model was introduced in the paper "A Simple Synchronous Syntactic-Semantic Sentence Simplification Model" by Siddharthan et al. (2014).

> The S4 model consists of two main components: a synchronous grammar and a semantic model. The synchronous grammar is used to identify the syntactic structure of the input sentence and generate a simplified version of it. The semantic model is then used to ensure that the simplified sentence retains the original meaning.

> The S4 model is designed to address the problem of text simplification, which is the process of transforming complex text into simpler text that is easier to read and understand. Text simplification is important for making information more accessible to people with cognitive disabilities or limited literacy skills, as well as for improving readability for non-native speakers of a language.

> Overall, the S4 model represents an important contribution to the field of NLP, as it provides a simple yet effective approach to sentence simplification that can be used in a wide range of applications.

However, all of the above statements are like complete nonsense. The citation doesn't exist and the S4 model that the thing is saying exists doesn't.

I have found search engines to be significantly more reliable (with the ability to actually check sources) than chatGPT. Usually, when I get decent answers from chatGPT, I would also get the same from Wikipedia which is usually a top result on most search engines. Do you have an example of an entire class of technical queries where chatGPT might perform better?

Re: Keep your AI claims in check

#256

Post by graypegg »

KRAKRISMOTT wrote: post #252
graypegg wrote: post #251

Earlier quoted context omitted.

I’m sure there’s a company out there who uses some linear equation in their app that they came up with by dumping the data they had into Excel and running the linear regression “data analysis” on it. > “Before labeling your product as AI-powered, note also that merely using an AI tool in the development process is not the same as a product having AI in it.” So is y=7.4x+5 an “AI” running inside our app or is it just…

Replace x and y with matrices and wrap everything in a non-linearity. Swap the 7.4 and 5 constants for variables a and b and set their values by taking the partial derivative of the with respect to the difference between the ground truth value and the predicted y. String together a bunch of these "smart cells" and observe that we can process sequences of data by linking the cells together. Further observe that if we…

Capiche? I agree with you! Haha.

I’m just curious where the FTC would draw the line. The root commenter made a good point that they seem to make a value judgement of what AI means. We can stretch that meaning pretty far if we want :)

Re: Keep your AI claims in check

#257

Post by falsenapkin »

flatline wrote: post #36

“It’s an ambiguous term with many possible definitions.” “Does the product actually use AI at all? If you think you can get away with baseless claims…” Last I checked basic optimization techniques like simulated annealing and gradient descent - as well as a host of basic statistical tools - are standard parts of an introductory AI textbook. I’ve been on the receiving end of government agency enforcement (SEC) and it…

To quote the points you left out:

> Are you exaggerating what your AI product can do?

> Are you promising that your AI product does something better than a non-AI product?

> Are you aware of the risks?

I'm guessing everyone here has come across examples of "AI" tossed onto something that either 1) 10 years ago wouldn't have been called AI or 2) the thought of something with a more recent interpretation of "AI" being core to the function of the product is a little scary and/or feels a little unnecessary.

Maybe it is a shakedown/warning. I think that's fair. We should have better definitions so that these agencies can't overstep, and products should have a better explanation of what "AI" means in their context. Until then yeah, vague threats versus vague promises.

Re: Keep your AI claims in check

#259

Post by simple-thoughts »

danielmarkbruce wrote: post #248
genocidicbunny wrote: post #241

Earlier quoted context omitted.

You keep using the word casual here, but what exactly do you mean by that? The statement seemed anything but casual. It has likely been worked on and vetted by multiple people, and while it does have a somewhat breve and light tone, I don't think that qualifies for 'casualness'. For a statement meant for a broad audience, the tone seems appropriate enough.

It's a blog post. It has a breve and light tone. There aren't many ways to make it more casual. Perhaps a tweet? Legal threats shouldn't be made this way, especially by a government toward citizens.

Most people don’t understand how regulatory agencies actually operate (banditry), but most people do vote. So they have conflicting goals; on the one side, they need to create a paper record so that judges will approve their future banditry, but on the other side they need to disguise their intentions from the public. This blog post by the FTC is unusually well done and effectively meets both goals, so I disagree with you that the threats shouldn’t be made this way as it is clearly in their own benefit to do so.

Re: Keep your AI claims in check

#260

Post by blobbers »

psyklic wrote: post #65
flatline wrote: post #36

“It’s an ambiguous term with many possible definitions.” “Does the product actually use AI at all? If you think you can get away with baseless claims…” Last I checked basic optimization techniques like simulated annealing and gradient descent - as well as a host of basic statistical tools - are standard parts of an introductory AI textbook. I’ve been on the receiving end of government agency enforcement (SEC) and it…

I know of at least one startup that claimed to use AI (including having AI in the company name), but in actuality humans did nearly all of the work. Hoped that once they got enough customers (and supposedly "proved the concept"), they could figure out how to use AI instead. I bet this is/was somewhat common. I also see many (particularly "legacy") products say they're "AI-driven" or "powered by AI", when in actuality…

Mechanical Turk is an artificial intelligence. Just not the kind you imagine. Besides, counting cars is hard, have a human do it.
Post reply on HN