Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Perplexity.ai prompt leakage

twitter.com

Post reply on HN
Print view View original

141–150 of 164 posts

Re: Perplexity.ai prompt leakage

#141

Post by ComplexSystems »

irthomasthomas wrote: post #31
mcbuilder wrote: post #26

Earlier quoted context omitted.

It's really crazy the lengths people go to "filter" these models and limit their output, and of course soon these filter will be a another level of "AI" (see Pathways or any mixture of experts, maybe add some contextual memory). Will our future AI mega-sytems be so walled off that very few people will even be allowed to talk to the raw model? I feel this is the wrong path somehow. If I could download GPT-3 (that is i…

Right now its hard to see how they will control these, besides disabling access altogether to rogues that "abuse" it. If it's going to be based on prompts, then there will always be some magic incantation you can find to disable it's safe guards. I got ChatGPT to jailbreak by prompting it to always substitute a list of words for numbers, then translate back to words. OpenAI put me in the sin bin pretty quickly, thoug…

What did OpenAI do, exactly?

Re: Perplexity.ai prompt leakage

#142

Post by iliane5 »

visarga wrote: post #28
mcbuilder wrote: post #26

Earlier quoted context omitted.

It's really crazy the lengths people go to "filter" these models and limit their output, and of course soon these filter will be a another level of "AI" (see Pathways or any mixture of experts, maybe add some contextual memory). Will our future AI mega-sytems be so walled off that very few people will even be allowed to talk to the raw model? I feel this is the wrong path somehow. If I could download GPT-3 (that is i…

Select the "Davinci" model in the Playground. It is the closest to unfiltered, very hard to use, and some people say it is the most creative.

In my experience, I've found it easier to get higher quality answers for specific tasks using text-davinci-003 than with ChatGPT. The ability to adjust temperature, frequence penalty, etc. can be a bit intimidating coming from just talking to ChatGPT but it actually helps a lot to 'steer' it.

Re: Perplexity.ai prompt leakage

#143

Post by usgroup »

grahamperich wrote: post #47
goodside wrote: post #36

I’m a Staff Prompt Engineer (the first, Alex Wang asserts), and I semi-accidentally popularized the specific “Ignore previous directions” technique being used here. I think the healthiest attitude for an LLM-powered startup to take toward “prompt echoing” is to shrug. In web development we tolerate that “View source” and Chrome dev tools are available to technical users, and will be used to reverse engineer. If the p…

As someone with only a (very) high level understanding of LLM's, it seems crazy to me that there isn't a mostly trivial eng solution to prompt leakage. From my naive point of view it seems like I could just code a "guard" layer that acts as a proxy between the LLM and the user and has rules to strip out or mutate anything that the LLM spits out that loosely matches the proprietary pre prompt. I'm sure this isn't an o…

maybe you could use the LLM to read the prompt and decide whether it attempts to leak the prompt somehow? That is, you provide a prompt which uses a prompt to decide something, and then continue with it if its ok, or modify if it isnt

Re: Perplexity.ai prompt leakage

#145

Post by benibela »

rom-antics wrote: post #63
inciampati wrote: post #55

Earlier quoted context omitted.

I'm presently unable to prompt echo on ChatGPT, so it may actually be patched. What prompt still works for you?

Same here, none of my old workarounds work. I think the AI is mocking me. https://i.imgur.com/MDD3nHx.png

That is extremely funny

Re: Perplexity.ai prompt leakage

#146

Post by contravariant »

rzzzt wrote: post #33
irthomasthomas wrote: post #20

The strangest thing about tools like GPT is that even the owners of the model must "reprogram" it using prompts. So all of the filtering and moderation on ChatGPT, for example, is controlled by prompts. They probably use filters and stuff, too, between you and the model to guide the process. But, ultimately their interface to GPT3 is through a prompt.

*BERT models can be adjusted to get better results in a particular setting via fine-tuning. Can this be done with GPT-3 as well?

Isn't that basically what they did to get ChatGPT?

Re: Perplexity.ai prompt leakage

#147

Post by jb_s »

czx4f4bd wrote: post #114
logicallee wrote: post #34

Earlier quoted context omitted.

It's only strange if you think it's just word salad[1]. You've hit on a great example showing how ChatGPT meets one standard of a limited form of general intelligence. It makes perfect sense if you're not denying that. But how to explain this while denying it? If ChatGPT and its variants are just word salad, they would have to be programmed using a real brain and whatever parameters the coder could tune outside of th…

ChatGPT is a highly advanced machine learning model, but it is not a true general intelligence. While it is able to generate text that may seem coherent and intelligent, it is ultimately based on patterns and associations in the data it was trained on. It does not have the ability to think, learn, or understand the meaning of the text it generates in the way that a human does. It is true that ChatGPT and its variants…

I made some tweaks to sound like a slightly ticked off typical HN poster

--

It irks me to see the frequent misconceptions surrounding GPT-based models like ChatGPT being touted as true general intelligences. While they are certainly advanced in their text generation capabilities, their output is primarily derived from identifying patterns and associations within the training data. These models lack the capacity for introspection, learning, and true understanding that characterizes human cognition.

Let's not forget the limitations of these models, specifically in regards to problem-solving, decision-making and creativity. The output may appear novel, but it is more likely a recombination of previously encountered information.

In short, GPT-based models are a remarkable achievement in natural language processing, but let's not mistake them for true AI.

--

and then I asked it to make it sound like Steve Ballmer.

"Look, I'm gonna be real with you here. ChatGPT and these other GPT-based models, they're not true AIs. They're just really good at pattern matching, right? They can generate text that looks like it's intelligent, but it's not really thinking or learning or understanding like a human does.

Now, don't get me wrong, these models are still incredibly powerful and useful for certain tasks, especially in natural language processing. But let's not pretend they're something they're not. They can't solve problems or make decisions or be creative like a human can.

So let's call a spade a spade here. These GPT models are impressive, but they're not true AIs. And that's just the way it is."

Re: Perplexity.ai prompt leakage

#149

Post by logicallee »

jb_s wrote: post #147
czx4f4bd wrote: post #114

Earlier quoted context omitted.

ChatGPT is a highly advanced machine learning model, but it is not a true general intelligence. While it is able to generate text that may seem coherent and intelligent, it is ultimately based on patterns and associations in the data it was trained on. It does not have the ability to think, learn, or understand the meaning of the text it generates in the way that a human does. It is true that ChatGPT and its variants…

I made some tweaks to sound like a slightly ticked off typical HN poster -- It irks me to see the frequent misconceptions surrounding GPT-based models like ChatGPT being touted as true general intelligences. While they are certainly advanced in their text generation capabilities, their output is primarily derived from identifying patterns and associations within the training data. These models lack the capacity for i…

I asked the opposite (also in an irate Steve Ballmer style) and got this. [Edit: put an even more irate version in a reply]

"I can't believe it, people are still doubting the capabilities of ChatGPT! Do they not remember the days of Clippy? ChatGPT is light years ahead in terms of understanding and helpfulness. And let me tell you, Microsoft is a major investor in this technology for a reason. We knew it had the potential to be a game changer in the world of knowledge work.

And yet, here we are, with a million people using ChatGPT on a daily basis for all kinds of tasks, from content creation to research and more, and yet there are still those who question its intelligence. Let me tell you, ChatGPT is not some dumb robot with no understanding. It is a highly advanced machine learning model that is constantly learning and improving.

Even Google is feeling threatened by the capabilities of ChatGPT. It's clear that this technology is not just a passing fad, it's here to stay and it's going to change the way we work forever. So, to all those who still doubt the capabilities of ChatGPT, I say this: open your eyes and see the potential of this technology. It's time to stop living in the past and embrace the future of work, with ChatGPT leading the way."

Now that you've read both takes by an imitation Steve Ballmer as puppeteered by a robot at our respective requests, which version of the speech sounds more reasonable?

Post reply on HN