Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sorry, but a new prompt for GPT-4 is not a paper

twitter.com

Post reply on HN
Print view View original

31–40 of 193 posts

Re: Sorry, but a new prompt for GPT-4 is not a paper

#31

Post by wongarsu »

I feel this has nothing at all to do with LLMs and more to do with academic incentives in general. Focusing on quality over quantity won't advance your career. Publishing lots of new papers will, as long as they meet the minimum threshold to be accepted into whatever journal or conference you are aiming for. Having one good paper won't increase your h-score, three mediocre papers might.

Doubly so when there's a new breakthrough, where one of your low-effort papers might end up being the first saying something obvious that ends up being really important. Because then everyone will end up quoting your paper in perpetuity.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#32

Post by mensetmanusman »

It is a _paper_, but it's not science, since GTP-4 is closed source and thus not reproducible in a lab.

If OpenAI disappears tomorrow, papers are GTP-4 will likely be of little to no value, which is another tell of a non-scientific exploration.

(note: not all explorations are scientific, and that is great! Science is just one of many tools for exploring lived reality.)

Re: Sorry, but a new prompt for GPT-4 is not a paper

#33

Post by snet0 »

It's hard to draw these lines, because you will certainly filter out a lot of bad (i.e. useless, low contribution to any field) papers, but you might also filter out some really important papers. Research being basic or something anyone could've done doesn't count againt its potential importance, just the expected value of importance I guess.

I'd rather we had a few too many bad papers than a few too few great papers.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#34

Post by starbugs »

awestroke wrote: post #19
starbugs wrote: post #15

Earlier quoted context omitted.

> Compare with drug trials: Adderall only differs from regular amphetamine in the relative concentration of enantiomers, and the entire value of the drug is in the measurements. Drug trials may be expected to be somewhat reproducible. What I don't get is how it can even be called research if it cannot be expected to be reproducible at all! GPT is a closed source/weights, proprietary product that changes every couple…

You can select a static snapshot that presumably does not change, if you use the API

> You can select a static snapshot that presumably does not change, if you use the API

Sorry, I won't blindly believe a company who are cynical enough to call themselves "OpenAI", then publish a commercial closed source/weights model for profit.

Evidence that they do not change without notice or it didn't happen. Better even, provide the source and weights for research purposes. These models could be pulled at every instant if the company sees fit or ceases to exist.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#35

Post by forgotpwd16 »

jatins wrote: post #29

Can a person just go upload anything on arxiv or is there a review process around these things? What I am really asking is "what makes something a paper and not a blogpost"?

ArXiv is kinda reputation-based. That is to submit something you need to be endorsed, either done automatically, based on institution, or asking established authors. After being endorsed to a subject area, you can submit freely to it, keeping in mind:

>Submissions to arXiv are subject to a moderation process that classifies material as topical to the subject area and checks for scholarly value. Material is not peer-reviewed by arXiv - the contents of arXiv submissions are wholly the responsibility of the submitter and are presented “as is” without any warranty or guarantee.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#36

Post by Retr0id »

Der_Einzige wrote: post #26
selfhoster11 wrote: post #22

Excuse me? Step by step wasn't paper-worthy? Hard disagree. LLM research is currently in its infancy, because they are no older than a few years old. And a research field in its infancy is bound to have a few noteworthy "no sh*t, Sherlock" papers that would be obvious from hindsight. The fact is, LLMs are a higher-order construct in machine learning, much like a fish is higher-order than a simple cellular colony. Low…

NLP/Computational Lingustics is NOT a new field. The ACL was founded in 1962 LM research is also old, papers using very shitty LMs (I.e Markov chains) and discussing them have existed since the early 2000s and likely before. Check yourself before you try to check others.

Try asking a markov chain to think step by step

Re: Sorry, but a new prompt for GPT-4 is not a paper

#38

Post by phkahler »

selfhoster11 wrote: post #22

Excuse me? Step by step wasn't paper-worthy? Hard disagree. LLM research is currently in its infancy, because they are no older than a few years old. And a research field in its infancy is bound to have a few noteworthy "no sh*t, Sherlock" papers that would be obvious from hindsight. The fact is, LLMs are a higher-order construct in machine learning, much like a fish is higher-order than a simple cellular colony. Low…

>> LLM research is currently in its infancy

Everything that went in to creating GPT4 is AI/science or whatever. Probing GPT4 and trying to understand and characterize it is also a very worthy thing to do - else how can it be improved upon? But if making GPT is science, I'd say this stuff is more akin to psychology ;-)

Re: Sorry, but a new prompt for GPT-4 is not a paper

#39

Post by MzxgckZtNqX5i »

jatins wrote: post #29

Can a person just go upload anything on arxiv or is there a review process around these things? What I am really asking is "what makes something a paper and not a blogpost"?

"Articles" on arXiv are not peer-reviewed, they just check whether it looks like it belongs to one of the categories they hosts:

"Registered users may submit articles to be announced by arXiv. There are no fees or costs for article submission. Submissions to arXiv are subject to a moderation process that classifies material as topical to the subject area and checks for scholarly value. Material is not peer-reviewed by arXiv - the contents of arXiv submissions are wholly the responsibility of the submitter and are presented “as is” without any warranty or guarantee." [0]

They are commonly known as pre-prints, in a similar fashion to IACR ePrint [1] for cryptography.

[0]: https://info.arxiv.org/about/index.html

[1]: https://eprint.iacr.org/

Re: Sorry, but a new prompt for GPT-4 is not a paper

#40

Post by whywhywhywhy »

Academia needs to get over itself, can't wait to see how amazing this tech is going to get when the next generation who decide never to bother with those stuffy and navel gazing institutions becomes the driving force behind it.

Looking forward to "I made this cool thing, here's the code/library you can use" rather than the papers/gatekeeping/ego stroking/"muh PhD".

Think if Google had built an AI team around the former rather than the latter, they wouldn't have risked the future of their entire company and squandered their decade head start.

Post reply on HN