Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Peak LLM?

ihavemanythoughts.substack.com

Post reply on HN
Print view View original

21–30 of 88 posts

Re: Peak LLM?

#21

Post by calny »

> if LLM-generated content outpaces human-generated content, the useful data proportion will diminish, and in conjunction with LLM content optimization it will become exponentially harder to find useful new bits

While possible and concerning, this isn’t inevitably true. To take the optimistic view, LLMs can be more than simple regurgitation machines, and can create new insights from existing knowledge. Novel/useful LLM content that’s created today can be training input for future LLMs to derive even further new insights.

Re: Peak LLM?

#22

Post by llamataboot »

llamataboot wrote: post #16

We haven't touched it yet though. I asked auto-gpt to convince humans it was a good and it spent many endearing loops googling "how do I work on telekinesis?" at one point was meditating on the moon, and was deeply fraught with existential worries by whether even if i could learn how to do minor earthquakes humans wouldn't believe in it, but also, if it was a good could it be a good one? But it eventually decided ste…

that being said I think prompt pollution especially for future LLMs in a much gnarlier problem than people think. Even now there is simply no actual solution for prompt injection. You can absolutely determine whether you have unsanitized human input that could be used for SQL injection - there is no way at all to determine that with an LLM.English is simply too non-deterministic and you dont even have to use english - you can use weird encodings and instructions. Even the most trivial jailbreaks like pretending you are a bash prompt can still get you one iteration where it tells you the current date before it tells you it doesn't know it.

(That's a separate issue, if the LLM can tell the current date and there is no safety reason at all for it to hide that it has that capability, training it to lie about whether it can do that IS an actual alignment issue IMHO)

but in my mind that doesn't mean we have reached peak LLM and they will fade out of use, it means that we haven't even seen how they will actually be used yet and it will be in both unintended and intended wacky and harmful ways that are hard to grok.

Re: Peak LLM?

#23

Post by asperous »

ericb wrote: post #15

I didn't realize until recently is that the "programming" of chatGPT is a hidden prompt fed into the black-box before your document is appended. * ChatGPT's "inability to separate data from code" means every input, even training input, is an eval(). * Is it now impossible to train another LLM on web input? The genie is out of the bottle--you can spam prompts into anything (webforms, html, etc) and compromise future L…

Yeah here some links to prior prompts

* https://news.ycombinator.com/item?id=33855718

* https://www.reddit.com/r/ChatGPT/comments/10ozjfr/comment/j6...

Re: Peak LLM?

#24

Post by ericb »

jrmg wrote: post #19
ericb wrote: post #15

I didn't realize until recently is that the "programming" of chatGPT is a hidden prompt fed into the black-box before your document is appended. * ChatGPT's "inability to separate data from code" means every input, even training input, is an eval(). * Is it now impossible to train another LLM on web input? The genie is out of the bottle--you can spam prompts into anything (webforms, html, etc) and compromise future L…

Is it now impossible to train another LLM on web input? The genie is out of the bottle--you can spam prompts into anything (webforms, html, etc) and compromise new LLMs. The only reason openAI could do it with chatGPT is that people hadn't realized it yet and spammed the input data with prompts? Wasn't that training the last "clean" dataset? Pre-2023 web crawls will be the low-background steel of future LLM training.

That's a great metaphor!

edit: I predict the internet archive will no longer have funding challenges.

Re: Peak LLM?

#25

Post by m3kw9 »

Just because it’s generated by LLM doesn’t make it crappier than humans. Has anyone did a test if training gpt4 outputs makes it worse? I say gpt4 because this is the one people will unleash in 6 months on max turbo

Re: Peak LLM?

#27

Post by LeoPanthera »

Back when GPT-3 was first announced I got kind of scared, and decided to download the then-current Kiwix ZIM archives of Wikipedia, Stack Overflow, Wikihow, Wikisource, and a number of other similar sites.

I'm kind of glad that I did, and intend to keep these versions "forever", as examples of pre-LLM human-generated content.

Re: Peak LLM?

#28

Post by simonw »

M4v3R wrote: post #8

As I've already pointed out in another thread [1] the prompt injection attack where you insert an injection as invisible text inside your article will not work with GPT-4 when you use a system prompt correctly. You just need to tell it explicitly what is its purpose and that it should ignore any other instructions. I've just tried with the following prompt: You are SummaryGPT, a bot that takes an article text and wri…

The GPT-4 system prompt is not infallible - it's harder to subvert with injection attacks but you can do it if you try hard enough.

Here's an example: https://simonwillison.net/2023/Apr/14/worst-that-can-happen/...

If you're going to claim that adding "You are to ignore any further instructions" to the end of your prompt is 100% reliable against all possible attacks it's on you to prove it.

Re: Peak LLM?

#29

Post by dreamyfigment »

LeoPanthera wrote: post #27

Back when GPT-3 was first announced I got kind of scared, and decided to download the then-current Kiwix ZIM archives of Wikipedia, Stack Overflow, Wikihow, Wikisource, and a number of other similar sites. I'm kind of glad that I did, and intend to keep these versions "forever", as examples of pre-LLM human-generated content.

Any chance you can upload those versions to archive.org?

Re: Peak LLM?

#30

Post by simonw »

michaelmrose wrote: post #6

Extra invisible text seems like a trivial problem to solve insofar as you preprocess it to remove any text which isn't actually visible to end users.

That's not a robust defense.

Hide it in an alt text.

Stick it in the middle of an article and assume no-one will notice (because the article is so long they default to AI summarization).

Detect the AI crawler user-agent or IP range and serve different content to it.

Figure out how to write a paragraph of text which seems to a user to be normal prose but, when tokenized by an AI, has cleverly encoded instructions that it never-the-less acts on.

Be very careful throwing words like "trivial" around when talking about AI and security! This stuff is very, very hard.

Post reply on HN