Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sorry, but a new prompt for GPT-4 is not a paper

twitter.com

Post reply on HN
Print view View original

61–70 of 193 posts

Re: Sorry, but a new prompt for GPT-4 is not a paper

#61

Post by eimrine »

Woshiwuja wrote: post #56
eimrine wrote: post #54

Earlier quoted context omitted.

Psychology is a religious-like pseudoscience, they can not even define what "psy" is without using some conceptions from religions such as a soul. upd these statements from me are so controversial, the number of "points" just dances lambada. The psy* areas are clearly polarized: some guys upvote all my messages in this topic and some other ones downvote all my messages in this topic. This is a sign of something inter…

What? Are you literally referring to ancient greek psy? jesus...

What is incorrect in this reference? You have not proposed any counterarguments. Also if you need just more fresh data - how do you propose to interpret the result of the Rosenhan's experiment?

Re: Sorry, but a new prompt for GPT-4 is not a paper

#64

Post by glitchc »

The current scientific research apparatus is more about being first than about being correct or thorough. A paper that gets out early means more citations, and many of the faculty sit on the editorial boards, and are able to suggest/enforce specific citations during the review process. Academics aren't fully to blame for this, it's just how the incentives are set up in the system. Tenure and promotions are increasingly based on h-index; a measure of impact based largely on the number of citations.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#65

Post by etewiah »

Behind all this is a valid question. How does one evaluate prompts and LLMs? As gipeties (custom gpts) become more popular millions of hours will be wasted by ones that have been built badly. Without some sort of automated quality control, gipeties will become a victim of their own success.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#66

Post by starbugs »

IanCal wrote: post #43
starbugs wrote: post #20

Earlier quoted context omitted.

> Do you have any evidence that the weights for versioned models are being changed without notifications? I think in a real scientific process, it's upon those who claim that they are not to provide the evidence.

I'm sorry, but that's entirely ridiculous. You're mangling up a concept of burden of proof here. You can easily see this because it can be flipped around easily - you made a claim that they are being changed, even every few weeks! Should it really be on me to show that your very specific claim is false? Aside - but even if the model weights did change, that wouldn't stop research being possible. Otherwise no drug tri…

> You can easily see this because it can be flipped around easily - you made a claim that they are being changed, even every few weeks! Should it really be on me to show that your very specific claim is false?

Wait a minute? The author of such a paper makes a claim about some observation that's based on the assumption that the studied model is defined in a way. I am disputing that claim since no evidence has been shown that it is defined because no definition has been given.

If your twist on this issue would be true, then I would, by definition, have to accept everything that they claim as true without any evidence. That's not called science. That's called authority.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#67

Post by yieldcrv »

Arxiv is like the MENSA of the tech world

The similarity being that it’s ego masquerading as academic.

Most things shared from there should have just been a blog post.

The last year has showed that AI/ML research and use did not need academic gatekeeping by PhDs and yet many in that scene keep trying self infatuating things with the lowest utility.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#68

Post by MattRix »

mensetmanusman wrote: post #32

It is a _paper_, but it's not science, since GTP-4 is closed source and thus not reproducible in a lab. If OpenAI disappears tomorrow, papers are GTP-4 will likely be of little to no value, which is another tell of a non-scientific exploration. (note: not all explorations are scientific, and that is great! Science is just one of many tools for exploring lived reality.)

That’s like saying a biologist studying an endangered species isn’t doing science because the animal could disappear tomorrow. The permanence of a subject has no bearing on whether it is science or not.

The idea that science has to happen in a lab is of course absurd as well.

Re: Sorry, but a new prompt for GPT-4 is not a paper

#69

Post by rollcat »

glitchc wrote: post #64

The current scientific research apparatus is more about being first than about being correct or thorough. A paper that gets out early means more citations, and many of the faculty sit on the editorial boards, and are able to suggest/enforce specific citations during the review process. Academics aren't fully to blame for this, it's just how the incentives are set up in the system. Tenure and promotions are increasing…

It's hard to estimate the impact of an idea in the same way that you can estimate the impact of an investment (stock is a number that goes up or down). You're right that the current incentive system might be to blame, but a simplistic metric will be gamed just as easily - what would you propose?

Re: Sorry, but a new prompt for GPT-4 is not a paper

#70

Post by mo_42 »

Why not? A paper is not necessarily scientific nor a breakthrough. In my view, a paper is written and documented communication that's usually approved by peers in the field. Also a blunt observation in nature can be noteworthy. However, we don't see such papers anymore as these fields have matured. Just go back in the history of your field and you will find trivial papers.
Post reply on HN