Sorry, but a new prompt for GPT-4 is not a paper • Hacker News Board

Re: Sorry, but a new prompt for GPT-4 is not a paper

#121

Post by 0xfae » Tue, Dec 05, 2023, 4:57 PM UTC

adastra22 wrote: post #114

tensor wrote: post #108

Earlier quoted context omitted.

I'm not surprised at the defence of "prompt engineering" here. It's something easy to do with no real knowledge, and I'm sure having it dismissed hurts some people. But I 100% agree with the author, "prompt engineering" is not science, and I'd say it's not engineering either. All you're doing is exploring the parameter space of particular model in a very crude way. There is no "engineering" going on in this process,…

> But I 100% agree with the author, "prompt engineering" is not science, and I'd say it's not engineering either. All you're doing is exploring the parameter space of particular model in a very crude way. There is no "engineering" going on in this process, just a bunch of trial and error. I wonder what your definition of “science” or “engineering” is…

Right? I'm having a hard time imagining a definition that includes "trying new things and seeing what happens" but that doesn't include... "trying new things and seeing what happens"

Re: Sorry, but a new prompt for GPT-4 is not a paper

#122

Post by [deleted] » Tue, Dec 05, 2023, 4:57 PM UTC

selfhoster11 wrote: post #22

Excuse me? Step by step wasn't paper-worthy? Hard disagree. LLM research is currently in its infancy, because they are no older than a few years old. And a research field in its infancy is bound to have a few noteworthy "no sh*t, Sherlock" papers that would be obvious from hindsight. The fact is, LLMs are a higher-order construct in machine learning, much like a fish is higher-order than a simple cellular colony. Low…

[deleted]

Re: Sorry, but a new prompt for GPT-4 is not a paper

#123

Post by sgt101 » Tue, Dec 05, 2023, 4:58 PM UTC

WhitneyLand wrote: post #89

Should this be a paper? https://not-just-memorization.github.io/extracting-training-... There is supporting analysis and measurement, but the essence is a single type of prompt, and DeepMind is a heavyweight lab I think it’s fair to say. Moreover there’s evidence people independently reported this result months beforehand on Reddit based on casual observation.

You will notice that they say in the blog post:

"If you’re a researcher, consider pausing reading here, and instead please read our full paper for interesting science beyond just this one headline result. In particular, we do a bunch of work on open-source and semi-closed-source models in order to better understand the rate of extractable memorization (see below) across a large set of models."

So they are trying to rigorously quantify the behaviour of the model. Is this "look mom no hands"... I don't think so.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#124

Post by roguas » Tue, Dec 05, 2023, 4:59 PM UTC

adastra22 wrote: post #114

tensor wrote: post #108

Earlier quoted context omitted.

I'm not surprised at the defence of "prompt engineering" here. It's something easy to do with no real knowledge, and I'm sure having it dismissed hurts some people. But I 100% agree with the author, "prompt engineering" is not science, and I'd say it's not engineering either. All you're doing is exploring the parameter space of particular model in a very crude way. There is no "engineering" going on in this process,…

> But I 100% agree with the author, "prompt engineering" is not science, and I'd say it's not engineering either. All you're doing is exploring the parameter space of particular model in a very crude way. There is no "engineering" going on in this process, just a bunch of trial and error. I wonder what your definition of “science” or “engineering” is…

Especially given this would be fine definition of engineering+science: "All you're doing is exploring the parameter space of particular model in a very crude way."

Re: Sorry, but a new prompt for GPT-4 is not a paper

#125

Post by rileymat2 » Tue, Dec 05, 2023, 4:59 PM UTC

alphazard wrote: post #120

Developing prompts for these models isn't a science yet. It does seem to meet most of the criteria for an art though. We recognize some outputs as high quality, and others as low quality, but often can't articulate the exact reason why. It seems that some people are able to reliably produce high quality results, indicating there is some kind of skill involved. More precisely, the quality of an individual artist's las…

Why isn’t it science? Surely people can use the scientific method in investigating?

Re: Sorry, but a new prompt for GPT-4 is not a paper

#126

Post by fkyoureadthedoc » Tue, Dec 05, 2023, 5:02 PM UTC

eimrine wrote: post #54

dartos wrote: post #46

Earlier quoted context omitted.

What is psychology, but applied biology? https://xkcd.com/435/

Psychology is a religious-like pseudoscience, they can not even define what "psy" is without using some conceptions from religions such as a soul. upd these statements from me are so controversial, the number of "points" just dances lambada. The psy* areas are clearly polarized: some guys upvote all my messages in this topic and some other ones downvote all my messages in this topic. This is a sign of something inter…

> This is a sign of something interesting but I am not ready to elaborate on this statement in this comment which is going to become [flagged] eventually.

Yet you're being a reply guy all over this thread, might as well just elaborate you clearly have the time and interest

Re: Sorry, but a new prompt for GPT-4 is not a paper

#127

Post by roguas » Tue, Dec 05, 2023, 5:02 PM UTC

_gabe_ wrote: post #73

selfhoster11 wrote: post #22

Excuse me? Step by step wasn't paper-worthy? Hard disagree. LLM research is currently in its infancy, because they are no older than a few years old. And a research field in its infancy is bound to have a few noteworthy "no sh*t, Sherlock" papers that would be obvious from hindsight. The fact is, LLMs are a higher-order construct in machine learning, much like a fish is higher-order than a simple cellular colony. Low…

I feel like the author of this tweet wasn’t saying step-by-step isn’t worthy, he was saying that non-reproducible results are not science. He emphasizes this twice in that tweet: > one experiment on one data set with seed picking is not worthy reporting > Additionally, we all need to understand this is just one good empirical result, now we need to make it useful…

Is it non-reproducible? Also results which reproducibility can be measured and appears stable is perfectly good science. I dislike when people throw statements like that.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#128

Post by roguas » Tue, Dec 05, 2023, 5:06 PM UTC

nicce wrote: post #118

MattRix wrote: post #68

Earlier quoted context omitted.

That’s like saying a biologist studying an endangered species isn’t doing science because the animal could disappear tomorrow. The permanence of a subject has no bearing on whether it is science or not. The idea that science has to happen in a lab is of course absurd as well.

> That’s like saying a biologist studying an endangered species isn’t doing science because the animal could disappear tomorrow. The permanence of a subject has no bearing on whether it is science or not. The idea that science has to happen in a lab is of course absurd as well. The main point here is that anyone can likely start studing those endangered species and try to reproduce the results while in GPT4 it is not…

Whats not possible? Do they ban people for exploring sota model that they offer?

Re: Sorry, but a new prompt for GPT-4 is not a paper

#129

Post by gwervc » Tue, Dec 05, 2023, 5:06 PM UTC

adastra22 wrote: post #114

tensor wrote: post #108

Earlier quoted context omitted.

I'm not surprised at the defence of "prompt engineering" here. It's something easy to do with no real knowledge, and I'm sure having it dismissed hurts some people. But I 100% agree with the author, "prompt engineering" is not science, and I'd say it's not engineering either. All you're doing is exploring the parameter space of particular model in a very crude way. There is no "engineering" going on in this process,…

> But I 100% agree with the author, "prompt engineering" is not science, and I'd say it's not engineering either. All you're doing is exploring the parameter space of particular model in a very crude way. There is no "engineering" going on in this process, just a bunch of trial and error. I wonder what your definition of “science” or “engineering” is…

If you remove the AI glasses, "prompt engineering" is just typing words and seeing if results match the expectations... which is exactly what any search engine pays their testers for. Those testers are making an important job to keep improving the quality of the product but they aren't engineers and even less so researchers.

Similarly a kid playing with the dose of water needed to build a sandcastle isn't a civil engineer nor an environmental researcher. Maybe on LinkedIn though.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#130

Post by zamfi » Tue, Dec 05, 2023, 5:13 PM UTC

kergonath wrote: post #117

jejeyyy77 wrote: post #102

Earlier quoted context omitted.

you realize nobody understands WHY or HOW these models work under the hood right? it's akin to evolution - we understand the process - that part is simple. But the output/organisms we have to investigate how they work.

> you realize nobody understands WHY or HOW these models work under the hood right? Of course we understand how they work, we built them! There is no mystery in their mechanisms, we know the number of neurons, their connectivity, everything from the weights to the activation functions. This is not a mystery, this is several decades of technical developments. > it's akin to evolution - we understand the process - that…

> Of course we understand how they work, we built them! There is no mystery in their mechanisms, we know the number of neurons, their connectivity, everything from the weights to the activation functions. This is not a mystery, this is several decades of technical developments.

The discovery of DNA’s structure was heralded as containing the same explanatory power as you describe here.

Turns out, the story was much more complicated then, and is much more complicated now.

Anyone today who tells you they know why LLMs are capable of programming, and how they do it, is plainly lying to you.

We have built a complex system that we only understand well at a basic “well there are weights and there’s attention, I guess?” layer. Past that we only have speculation right now.