Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Are you using a GPT to prompt-engineer another GPT?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 58 posts

Re: Ask HN: Are you using a GPT to prompt-engineer another GPT?

#41

Post by BOOSTERHIDROGEN »

Manouchehri wrote: post #37
chalsprhebaodu wrote: post #17

I’ve commented it before, and surely it’s something I’m doing wrong, but I cannot believe system prompts or GPTs or any amount of instructing actually works for people to get ChatGPT to respond in a certain fashion with any consistency. I have spent hours and hours and hours and hours trying to get ChatGPT to be a little less apologetic, long-winded, to stop reiterating, and to not interpret questions about its respo…

Using JSON mode with the GPT 3.5/4 API works well for us. So much so that we have to intentionally fake errors to test that our retries/fallbacks actually work in our code.

Have you compared this to chatgpt plus?

Re: Ask HN: Are you using a GPT to prompt-engineer another GPT?

#42

Post by kirkarg »

chalsprhebaodu wrote: post #17

I’ve commented it before, and surely it’s something I’m doing wrong, but I cannot believe system prompts or GPTs or any amount of instructing actually works for people to get ChatGPT to respond in a certain fashion with any consistency. I have spent hours and hours and hours and hours trying to get ChatGPT to be a little less apologetic, long-winded, to stop reiterating, and to not interpret questions about its respo…

I'm not sure about web based service but with the API this is easily achievable by tinkering with the system message.

Re: Ask HN: Are you using a GPT to prompt-engineer another GPT?

#43

Post by TrickardRixx »

supriyo-biswas wrote: post #40
chalsprhebaodu wrote: post #22

Earlier quoted context omitted.

Religiously.

I wonder if people only click on the thumbs down button, thus serving to only provide a negative signal mechanism with no ability to differentiate a positive from a negative one.

Anecdata: I’ve several times clicked “regenerate” and then thumbed-up the new response when nudged.

Re: Ask HN: Are you using a GPT to prompt-engineer another GPT?

#44

Post by Solvency »

chalsprhebaodu wrote: post #17

I’ve commented it before, and surely it’s something I’m doing wrong, but I cannot believe system prompts or GPTs or any amount of instructing actually works for people to get ChatGPT to respond in a certain fashion with any consistency. I have spent hours and hours and hours and hours trying to get ChatGPT to be a little less apologetic, long-winded, to stop reiterating, and to not interpret questions about its respo…

It wasn't always like this. GPT in early 2023, hell late 2022, was incredible. I could have it fully stimulating a Unix terminal on acid for hours, it'd never break character. It's so insanely nerfed now.

Re: Ask HN: Are you using a GPT to prompt-engineer another GPT?

#45

Post by Solvency »

kromem wrote: post #26
chalsprhebaodu wrote: post #17

I’ve commented it before, and surely it’s something I’m doing wrong, but I cannot believe system prompts or GPTs or any amount of instructing actually works for people to get ChatGPT to respond in a certain fashion with any consistency. I have spent hours and hours and hours and hours trying to get ChatGPT to be a little less apologetic, long-winded, to stop reiterating, and to not interpret questions about its respo…

A fun bug is that ChatGPT will always use an emoji when apologizing. So if you ask it not to use emojis in a chat and it does (which it often will do in promising not to), and point it out, it results in a loop of apologies and self critique that devolves into modeling an existential crisis.

This isn't even remotely true. I've never once seen an emoji from it in over a year of daily use.

Re: Ask HN: Are you using a GPT to prompt-engineer another GPT?

#46

Post by marviel »

chalsprhebaodu wrote: post #17

I’ve commented it before, and surely it’s something I’m doing wrong, but I cannot believe system prompts or GPTs or any amount of instructing actually works for people to get ChatGPT to respond in a certain fashion with any consistency. I have spent hours and hours and hours and hours trying to get ChatGPT to be a little less apologetic, long-winded, to stop reiterating, and to not interpret questions about its respo…

1. Use the API 2. Use Function Calling, with detailed parameters, well named output variables describing the format of the output you want.

You'll get much, much, much better results.

Re: Ask HN: Are you using a GPT to prompt-engineer another GPT?

#47

Post by SOLAR_FIELDS »

minimaxir wrote: post #31
SOLAR_FIELDS wrote: post #30

Earlier quoted context omitted.

Do you mean that the prompts would be completely dynamic in your tool vs the prebuilt prompts with templating that Langchain uses?

Dynamic, not templated. Or maybe templated at the start of the workflow, but obviously LLMs will have its own ways to adjust it.

I would be interested to see if something like this actually produces good results. My (limited) experiences with Langchain made me feel like it was too constraining for the use cases I tried.

Re: Ask HN: Are you using a GPT to prompt-engineer another GPT?

#48

Post by andrewedstrom »

You may be interested in a recent AI safety paper by Redwood Research.

In it, they have GPT-4 generate solutions to coding problems, but instruct it to insert backdoors into the solutions some fraction of the time. Then, they explore different ways to use a weaker model (GPT-3.5) to detect these backdoors. Pretty interesting.

[1] https://arxiv.org/abs/2312.06942

Re: Ask HN: Are you using a GPT to prompt-engineer another GPT?

#50

Post by muzani »

Solvency wrote: post #44
chalsprhebaodu wrote: post #17

I’ve commented it before, and surely it’s something I’m doing wrong, but I cannot believe system prompts or GPTs or any amount of instructing actually works for people to get ChatGPT to respond in a certain fashion with any consistency. I have spent hours and hours and hours and hours trying to get ChatGPT to be a little less apologetic, long-winded, to stop reiterating, and to not interpret questions about its respo…

It wasn't always like this. GPT in early 2023, hell late 2022, was incredible. I could have it fully stimulating a Unix terminal on acid for hours, it'd never break character. It's so insanely nerfed now.

It's insanely good every time they have a public release, then deteriorates significantly. There's plenty of evidence around this too - just compare the exact same prompt then and now. Not sure if this is a matter of cost or just playing whack a mole with unintended behaviorial bugs.
Post reply on HN