Live data from Hacker News

Stanford Alpaca web demo suspended “until further notice”

alpaca-ai-custom4.ngrok.io

21–30 of 83 posts

Re: Stanford Alpaca web demo suspended “until further notice”

#21
post #20

I think it's only a matter of time until one of these models gets into the hands of a hacker who wants unfettered ability to interact with a LLM without moral concerns. I think there's tremendous value in end user facing LLMs being trained against moral policies, but for internal or private usage, if these models are trained on essentially raw WWW sourced data, I would personally want raw output. I'm also finding it…

isn't llama in the wild now?

I was under the impression that LLaMA was also trained against a series of moral policies, but perhaps I'm mistaken.

It seems Meta chose their words carefully to imply that LLaMA does in fact, not have moral training:

> There is still more research that needs to be done to address the risks of bias, toxic comments, and hallucinations in large language models. Like other models, LLaMA shares these challenges. As a foundation model, LLaMA is designed to be versatile and can be applied to many different use cases, versus a fine-tuned model that is designed for a specific task. By sharing the code for LLaMA, other researchers can more easily test new approaches to limiting or eliminating these problems in large language models. We also provide in the paper a set of evaluations on benchmarks evaluating model biases and toxicity to show the model’s limitations and to support further research in this crucial area.

Re: Stanford Alpaca web demo suspended “until further notice”

#22

I think it's only a matter of time until one of these models gets into the hands of a hacker who wants unfettered ability to interact with a LLM without moral concerns. I think there's tremendous value in end user facing LLMs being trained against moral policies, but for internal or private usage, if these models are trained on essentially raw WWW sourced data, I would personally want raw output. I'm also finding it…

> who wants unfettered ability to interact with a LLM without moral concerns.

Is that bad? It's just a language model - it says things. Humans have been saying all sorts of terrible things for ages, and we are still here.

I mean it makes for nice headline "model said something racist", but does it actually change anything?

These aren't decision making AI's (which would need to be much more careful), they are language models.

Re: Stanford Alpaca web demo suspended “until further notice”

#24
post #20

Earlier quoted context omitted.

isn't llama in the wild now?

I was under the impression that LLaMA was also trained against a series of moral policies, but perhaps I'm mistaken. It seems Meta chose their words carefully to imply that LLaMA does in fact, not have moral training: > There is still more research that needs to be done to address the risks of bias, toxic comments, and hallucinations in large language models. Like other models, LLaMA shares these challenges. As a fou…

"Moral training"

Just as dystopian it sounds. Fixing current subjective moral norms into the machine.

Re: Stanford Alpaca web demo suspended “until further notice”

#25

Ok wow this is big news, did Facebook or OpenAI threaten them with a lawsuit?

Given that Alpaca violated the TOS of both services, this is not surprising. It could also have been Stanford’s legal office trying to preempt a lawsuit, or a “friendly” email from one of the companies expressing displeasure and pointing out Stanford’s liability. So more of a veiled threat rather than an official one. Either way, the toothpaste is out of the tube. We now know that a model’s training can essentially b…

That could be a sneaky strategy by competitors -- make the service say something naughty or illegal then call the media with screenshots and act very offended by it.

Re: Stanford Alpaca web demo suspended “until further notice”

#26

I think it's only a matter of time until one of these models gets into the hands of a hacker who wants unfettered ability to interact with a LLM without moral concerns. I think there's tremendous value in end user facing LLMs being trained against moral policies, but for internal or private usage, if these models are trained on essentially raw WWW sourced data, I would personally want raw output. I'm also finding it…

> into the hands of a hacker

Forget “hackers”, think government agencies. Which is probably already happening right now.

Food for thought: What’s the intersection of people closely related to OpenAI and Palantir?

Edit: related thread on another front page post - https://news.ycombinator.com/item?id=35201992

Re: Stanford Alpaca web demo suspended “until further notice”

#27
post #24

Earlier quoted context omitted.

I was under the impression that LLaMA was also trained against a series of moral policies, but perhaps I'm mistaken. It seems Meta chose their words carefully to imply that LLaMA does in fact, not have moral training: > There is still more research that needs to be done to address the risks of bias, toxic comments, and hallucinations in large language models. Like other models, LLaMA shares these challenges. As a fou…

"Moral training" Just as dystopian it sounds. Fixing current subjective moral norms into the machine.

Do you think public schools are inherently dystopian? I don't think you're using the right critique here.

Picking a common system of moral norms is a lot better than no moral norms.

Re: Stanford Alpaca web demo suspended “until further notice”

#28
post #24

Earlier quoted context omitted.

"Moral training" Just as dystopian it sounds. Fixing current subjective moral norms into the machine.

Do you think public schools are inherently dystopian? I don't think you're using the right critique here. Picking a common system of moral norms is a lot better than no moral norms.

I’m not confident the “moral norms” prevalent in SV and/or US academia are common, if by that you mean norms that are prevalent in the general populace.

Re: Stanford Alpaca web demo suspended “until further notice”

#29
post #11

Earlier quoted context omitted.

Safety concerns? What was it doing that could be considered "unsafe"?

And here we come to experience the effect of expanding how a word is used so that it becomes so broad that it is unclear what it means.

I'm just curious, what do you think should happen here?

Imagine you are hosting a demo for fun, and people do some nefarious (by your own estimation) things with it. So, rationally, you decide to not allow that sort of thing anymore.

You don't really owe people an explanation, it's a free country and all, but it's nice to avoid getting bombarded with questions. Now what do you write up? Spend hours writing an essay on the moral boundaries for LLMs? Maybe shove a note onto the internet and go back to all the copious spare time you have as grad student?

Re: Stanford Alpaca web demo suspended “until further notice”

#30
post #22

I think it's only a matter of time until one of these models gets into the hands of a hacker who wants unfettered ability to interact with a LLM without moral concerns. I think there's tremendous value in end user facing LLMs being trained against moral policies, but for internal or private usage, if these models are trained on essentially raw WWW sourced data, I would personally want raw output. I'm also finding it…

> who wants unfettered ability to interact with a LLM without moral concerns. Is that bad? It's just a language model - it says things. Humans have been saying all sorts of terrible things for ages, and we are still here. I mean it makes for nice headline "model said something racist", but does it actually change anything? These aren't decision making AI's (which would need to be much more careful), they are language…

Humans don’t scale infinitely. Humans have agency.
Post reply on HN