Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sorry, but a new prompt for GPT-4 is not a paper

twitter.com

Post reply on HN
Print view View original

171–180 of 193 posts

Re: Sorry, but a new prompt for GPT-4 is not a paper

#171

Post by daveguy »

elif wrote: post #166
daveguy wrote: post #151

Earlier quoted context omitted.

No prompt will cause an LLM to rapidly improve itself, much less into an AGI. Prompts don't cause permanent change in the LLM, only differences in output.

You're talking about how GPT functions in 2023. I am discussing such a point where when LLM outputs become valuable LLM modifications. AI recursing on itself progressing toward an AGI.

No matter how many times you feed the output of an LLM back to itself, the underlying model does not change. Online training (of actual model weights not just fine-tuning) would be hugely resource intensive and not guaranteed to do any better than the initial training. Interference will happen whether catastrophic or simply drift. We can fantasize about future architectures all day long, but that doesn't make them capable of AGI or even give us a path forward.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#172

Post by starbugs »

IanCal wrote: post #152
starbugs wrote: post #66

Earlier quoted context omitted.

> You can easily see this because it can be flipped around easily - you made a claim that they are being changed, even every few weeks! Should it really be on me to show that your very specific claim is false? Wait a minute? The author of such a paper makes a claim about some observation that's based on the assumption that the studied model is defined in a way. I am disputing that claim since no evidence has been sho…

> I am disputing that claim You are entirely within your rights to say that the authors have assumed that openai is not lying about their models. They've probably also assumed that other paper authors are not lying in their papers. You then say however: > GPT is a closed source/weights, proprietary product that changes every couple of weeks or so. And when I ask for evidence of this very specific claim, you turn arou…

Look, the burden of proof in a scientific paper is on the authors. Not on me.

A company with a proprietary product that says something is not acceptable evidence in a scientific context. No need to allege that anyone is lying. Lying is irrelevant. What's relevant is that the research is falsifiable. It cannot be falsifiable if you don't know what the actual model is at a given point in time.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#173

Post by zamfi »

kergonath wrote: post #144
zamfi wrote: post #130

Earlier quoted context omitted.

> Of course we understand how they work, we built them! There is no mystery in their mechanisms, we know the number of neurons, their connectivity, everything from the weights to the activation functions. This is not a mystery, this is several decades of technical developments. The discovery of DNA’s structure was heralded as containing the same explanatory power as you describe here. Turns out, the story was much mo…

> The discovery of DNA’s structure was heralded as containing the same explanatory power as you describe here. Not at all. It's like saying that since we can read hieroglyphics we know all about ancient Egypt. Deciphering DNA is tool to understand biology, it is not that understanding in itself. > Turns out, the story was much more complicated then, and is much more complicated now. We are reverse engineering biology…

> How so? They can do it because we taught them, there is no magic.

Yeah, no. I mean, we can’t introspect the system to see how it actually does programming at any useful level of abstraction. “Because we taught them” is about as useful a statement as “because its genetic parents were that way”.

No, of course it’s not magic. But that doesn’t mean we understand it at a useful level.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#174

Post by starbugs »

IanCal wrote: post #155
dartos wrote: post #50

Earlier quoted context omitted.

Apples and oranges comparison. You couldn’t get the same participants, but you could get the same drugs. If you could get identical participants, that wouldn’t be very helpful since humans are so varied. But for GPT based papers, what you’re actually testing could change without you knowing. There’s no way to know if a paper is reproducible at all. If you can’t reproduce results, is it really research, or just show a…

> If you can’t reproduce results, is it really research, or just show and tell? You can't start with a statement about clinical trials not being perfectly reproducible and that's fine, then say this. > what you’re actually testing could change without you knowing If people are lying about an extremely important part of their product, which they have little reason to. But then this applies to pretty much everything. S…

> You can't start with a statement about clinical trials not being perfectly reproducible and that's fine, then say this.

Nobody said that to begin with. Re-read their comment.

> If people are lying about an extremely important part of their product [...]

Nobody is alleging that anyone is lying. It's just that we cannot be sure what the research actually refers to, because of the nature of a proprietary/closed model.

> This is a little silly because these models are available extremely easily and at a pay-as-you-go pricing.

What does this have to do with the parent comment? I don't think it's appropriate to call anyone here silly, just because you don't like their comment and don't have good counter arguments.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#175

Post by starbugs »

cqqxo4zV46cp wrote: post #170
starbugs wrote: post #34

Earlier quoted context omitted.

> You can select a static snapshot that presumably does not change, if you use the API Sorry, I won't blindly believe a company who are cynical enough to call themselves "OpenAI", then publish a commercial closed source/weights model for profit. Evidence that they do not change without notice or it didn't happen. Better even, provide the source and weights for research purposes. These models could be pulled at every…

Yeah, here it comes. In these conversations you don’t need to ask very many “why”s before it just turns out that the antagonist (you) has an axe to grind about OpenAI, and has added that the their misplaced sense of expertise with regard to the typical standards of proof in academic publications.

> Yeah, here it comes. In these conversations you don’t need to ask very many “why”s before it just turns out that the antagonist (you) has an axe to grind about OpenAI, and has added that the their misplaced sense of expertise with regard to the typical standards of proof in academic publications.

Seems to have hit hard?

I would find it borderline acceptable being offended by a user whose name has obviously been generated using a password generator if you could at least provide some substance to the discussion. Just labeling someone and questioning their competence based on your hurt feelings is a bit low. Please improve.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#176

Post by j2kun »

selfhoster11 wrote: post #149
j2kun wrote: post #140

Earlier quoted context omitted.

Except there's also, you know, building coherent theories and using those theories to predict the system behavior.

All right, here is a theory: LLMs contain "latent knowledge" that is sometimes used by the model during inference, and sometimes it isn't. One way to "engage" these internal representations is to include keywords or patterns of text that make that latent knowledge more likely to "activate". Say, if you want to ask about palm trees, include a paragraph talking about a species of a palm tree (no matter whether it conta…

> It might not be the actual truth or what's going on inside the model.

This sums up pretty nicely why prompt hacking is not science. A scientific theory is related in a concrete way to the mechanism by which the phenomenon being studied works.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#177

Post by DonHopkins »

alphazard wrote: post #120

Developing prompts for these models isn't a science yet. It does seem to meet most of the criteria for an art though. We recognize some outputs as high quality, and others as low quality, but often can't articulate the exact reason why. It seems that some people are able to reliably produce high quality results, indicating there is some kind of skill involved. More precisely, the quality of an individual artist's las…

In the same sense that "Bullshit artist" is a better term than "Bullshit engineer".

Re: Sorry, but a new prompt for GPT-4 is not a paper

#178

Post by lmeyerov »

harlanlewis wrote: post #160
lmeyerov wrote: post #135

To bring some data to a sour grapes fight: https://paperswithcode.com/sota/code-generation-on-humaneval For code generation, GPT4 is getting beat by the small prompt library LATS wrapped around GPT3.5. Given the recent release of MagicCoder / Instruct-OSS, that means a small prompt library + a small 7B model you can self-host beats the much fancier GPT4. Similar to when simple NNs destroyed a decade of Bayesian model…

> GPT4 is getting beat by the small prompt library LATS wrapped around GPT3.5 The link you shared doesn’t quite reflect this. Omitting other models… LATS (gpt-4): 94.4 Reflexion (gpt-4): 91.0 gpt-4: 86.6 … LATS (gpt-3.5): 83.8 … zero-shot (gpt-4): 67.0 zero-shot (gpt-3.5): 48.1 I’m not quite sure how to translate leaderboards like these into actual utility, but it certainly feels like “good enough” is only going to g…

Ah you're right, LATS GPT3.5 is 84 while standalone GPT4 is 87

Given standalone GPT3.5 is "just" 48.. it's less about beating and more about meeting

RE:Good Enough & Feel... very much agreed. I find it very task dependent!

For example, GPT4 is 'good enough' that developers are comfortable copy-pasting & trying, even vs stack overflow results. We haven't seen LATS+MagicCoder yet, but as MagicCoder 7b already meets+exceeds GPT3.5 for HumanEval, there's a plausible hope for agent-aided GPT4-grade tools being always-on for all coding tasks, and sooner vs later. We made that bet for Louie.AI's interactive analyst interface, and as each month passes, evidence mounts. We can go surprisingly far with GPT3.5 before wanting to switch to GPT4 for this kind of interaction scenario.

Conversely... I've yet to see a true long-running autonomous coding autoGPT where the error rate doesn't kill it. We're experimenting with design partners on directions here -- think autonomous investigations etc -- but there's more on the advanced fringe and with special use cases, guard rails, etc. For most of our users and use cases... we're able to more reliably deliver -- today -- on the interactive scenarios with smaller snippets.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#179

Post by kjkjadksj »

sebzim4500 wrote: post #25
starbugs wrote: post #15

Earlier quoted context omitted.

> Compare with drug trials: Adderall only differs from regular amphetamine in the relative concentration of enantiomers, and the entire value of the drug is in the measurements. Drug trials may be expected to be somewhat reproducible. What I don't get is how it can even be called research if it cannot be expected to be reproducible at all! GPT is a closed source/weights, proprietary product that changes every couple…

If I book telescope time and capture a supernova then no one will ever be able to reproduce my raw results because it has already happened. I don't see why OpenAI pulling old model snapshots is any different.

But you know the parameters of your telescope at least. If openai wants to update all the time, fine, but then they should be like how every other piece of research software works, where you can list what exact version of software you used and pull that version yourself if need be.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#180

Post by [deleted] »

wongarsu wrote: post #31

I feel this has nothing at all to do with LLMs and more to do with academic incentives in general. Focusing on quality over quantity won't advance your career. Publishing lots of new papers will, as long as they meet the minimum threshold to be accepted into whatever journal or conference you are aiming for. Having one good paper won't increase your h-score, three mediocre papers might. Doubly so when there's a new b…

[deleted]
Post reply on HN