Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The danger of relying on OpenAI's Deep Research

economist.com

Post reply on HN
Print view View original

51–60 of 63 posts

Re: The danger of relying on OpenAI's Deep Research

#51

Post by bruce511 »

aeternum wrote: post #50
bruce511 wrote: post #30

Earlier quoted context omitted.

Right now, today, nothing stops academia from publishing every paper they write to be open to all. The internet is there and publishing is free. The fact that they don't implies that they perceive some value is provided by publishing in the traditional way. Assuming that journals die as a result of your law (why would people pay for a journal anymore?) is it worth considering that value (perceived or real) before adv…

Many of the prestigious journals do require an exclusive license: https://www.springernature.com/gp/open-science/policies/jour... Under the Springer Nature Subscription licence agreement Share the final published work with peers: Limited sharing for research and career advancement allowed And it's typically quite expensive to publish under the OA license. I still see no problem with the proposed law. One of two thing…

>> Many of the prestigious journals do require an exclusive license

Of course they do. But nothing forces the researcher to use those journals. So the question becomes why do they? Perhaps there's prestige involved?

>> 1) Federally funded researchers publish elsewhere

That's my point. Since researchers already have this option why are they not exercising it? Why are researchers happy, indeed prefer, publishing with Springer? Only by understanding why they currently choose to use Springer et al, can you understand what is lost by requiring free publishing.

Re: The danger of relying on OpenAI's Deep Research

#52

Post by bruce511 »

visarga wrote: post #32
bruce511 wrote: post #30

Earlier quoted context omitted.

Right now, today, nothing stops academia from publishing every paper they write to be open to all. The internet is there and publishing is free. The fact that they don't implies that they perceive some value is provided by publishing in the traditional way. Assuming that journals die as a result of your law (why would people pay for a journal anymore?) is it worth considering that value (perceived or real) before adv…

> they perceive some value is provided by publishing in the traditional way Yes, but it provides value only for scientist careers, not for science.

>> Yes, but it provides value only for scientist careers, not for science.

That's a very definitive statement. But perhaps it is untrue?

It would seem yes, that it's good for scientist careers.Are "good" careers versus "bad" careers good or bad for science? In other words, is it useful to rank scientists when allocating grant money? Do good scientists make better use of the money than bad ones? Are there other ways you could suggest to rank scientists?

Perhaps there are other groups that benefit as well? What about people issuing grants? Is it useful to them that there is industry recognised feedback regarding the scientist and her work? Is this useful when allocating limited funds to an unlimited demand?

What about people following the science? Let's take industry. Say I want to make a commercial product. Should I start by paying attention to the field, understanding the science? Or is it ok to just read any unvetted thing?

What about media, and by extension the population? Can a media outlet run a story based on "something someone wrote on the internet"? Or should they prefer credible sources? Should the public have some interest in the truthiness of something? Should the public (via the media) understand the difference between a result published in the New England Journal of Medicine, or what my homeopath down the road published on their blog?

What about say doctors keeping up with "current ideas"? Should they believe everything, every "study" posted on a blog? Or should there be a system of gate-keeping, sifting the valuable studies from the chaff? Presumably knowing that a study is well formed, and not paid for by say big pharma, might lend it more weight?

Of course publishers benefit (financially) from this system as well. But they don't matter right? So we'll ignore that. But even if you remove them from the equation, I'd suggest that "science" does indeed get value from the system, beyond just scientist careers.

Now, could all these benefits be gained in an alternative way to expensive journals? Almost certainly so. But in order to build such a system it's important to understand the strengths of the current one. A "simple" law might solve one part of the puzzle, but at the same time have very foreseeable consequences in other parts of the picture.

Re: The danger of relying on OpenAI's Deep Research

#53

Post by bruce511 »

oefrha wrote: post #41
bruce511 wrote: post #30

Earlier quoted context omitted.

Right now, today, nothing stops academia from publishing every paper they write to be open to all. The internet is there and publishing is free. The fact that they don't implies that they perceive some value is provided by publishing in the traditional way. Assuming that journals die as a result of your law (why would people pay for a journal anymore?) is it worth considering that value (perceived or real) before adv…

There are very cheap arXiv overlay journals where the processing fee is about $10 per submission.[1] And that’s with a for profit entity providing the platform. Reviewing is already unpaid, editing is often unpaid even with traditional journals. Your “journals die as a result” assumption is faulty. Not to mention for quite a few fields you can already find every remotely worthwhile paper on arXiv. Those fields didn’t…

so, given that all scientists and all fields have not immediately adopted this journal, can you elaborate on why some have chosen to not take this approach?

By understanding their reasoning it's perhaps possible to understand what benefits are missing from this model?

If this cheap publishing approach is better for grant providers, why do they in turn not place more weight on previous papers published this way. Presumably if they "prioritised researchers who do this" they would driver behaviour - no law required? Is big-journal leaning on them?

Re: The danger of relying on OpenAI's Deep Research

#54

Post by oefrha »

bruce511 wrote: post #53
oefrha wrote: post #41

Earlier quoted context omitted.

There are very cheap arXiv overlay journals where the processing fee is about $10 per submission.[1] And that’s with a for profit entity providing the platform. Reviewing is already unpaid, editing is often unpaid even with traditional journals. Your “journals die as a result” assumption is faulty. Not to mention for quite a few fields you can already find every remotely worthwhile paper on arXiv. Those fields didn’t…

so, given that all scientists and all fields have not immediately adopted this journal, can you elaborate on why some have chosen to not take this approach? By understanding their reasoning it's perhaps possible to understand what benefits are missing from this model? If this cheap publishing approach is better for grant providers, why do they in turn not place more weight on previous papers published this way. Presu…

Inertia and prisoner’s dilemma. If an entire field decided unanimously that all existing editorial boards of for-profit journals will resign and form a corresponding Free overnight and somehow with an inherited impact factor, then everyone except the leeches in for-profit publishing wins. But that’s an impossible ask. Right now, even when an entire editorial board resigns and starts over, the new journal needs to build reputation from scratch, and anyone submitting work (that qualifies for leading journals) to the blank slate instead of leading journals is committing career self-sabotage in a cutthroat environment. (And the publisher can hire a new board to keep the name going, the number of people willing to sit on the board for prestige is proportional to the journal’s prestige.) That leaves tenured folks who don’t need to worry about climbing the ladder, but they tend to have younger coauthors too. That really leaves tenured profs flying solo or collaborating with each other, and who’s fed up enough to do something about it even though they probably won’t have the network effect for a long time, if ever. It’s no coincidence that I linked to a Fields medalist (mathematician).

The next best thing without legislation is publishing preprints to open platforms while leaving the money sucking journals intact. Which has indeed happened in every field I’m familiar with, no idea what’s holding up the others, it’s not like any academic is living on royalties from papers. Inertia perhaps, and negotiation power?

And legislation may actually make a difference here, but hey, politicians don’t typically mess with big money.

Re: The danger of relying on OpenAI's Deep Research

#55

Post by yunwal »

marcus_holmes wrote: post #47
yunwal wrote: post #19

Earlier quoted context omitted.

Great, go get yourself 4 phds and a 600k/year job if ai can do everything for you! Meanwhile the rest of us in the real world will continue understanding the limits of the tools we use.

Both those things (PhD and high salary) require human interaction to achieve, not knowledge or tool use. There are many examples of people achieving both/either whilst being ignorant and/or unintelligent. The comment was about "you're not going to know anything because the LLM is doing it for you" which is easily obviously true. This won't stop anyone from getting a PhD or a high salary. It will just stop them from k…

> you're not going to know anything because the LLM is doing it for you" which is easily obviously true

Not obviously true to me. LLMs don’t know everything, so they can’t solve all problems. You can try and give it the proper context but you still need to understand the context is necessary, which requires knowledge.

Kids who grow up today will understand and intuit that synthesizing 3 facts from an article doesn’t constitute “work” any more than multiplying 4-digit numbers. Adults will eventually pick up on this as well.

Re: The danger of relying on OpenAI's Deep Research

#56

Post by cudgy »

snoman wrote: post #43
cudgy wrote: post #7

Earlier quoted context omitted.

And they’re usually right.

The data (re: the Flynn effect) suggests otherwise.

There are explanations to the Flynn effect that go beyond isolated intelligence. How does one isolate intelligence test when society, nutrition, religions, educational systems, and the very IQ standard itself has changed?

Re: The danger of relying on OpenAI's Deep Research

#57

Post by aeternum »

bruce511 wrote: post #51
aeternum wrote: post #50

Earlier quoted context omitted.

Many of the prestigious journals do require an exclusive license: https://www.springernature.com/gp/open-science/policies/jour... Under the Springer Nature Subscription licence agreement Share the final published work with peers: Limited sharing for research and career advancement allowed And it's typically quite expensive to publish under the OA license. I still see no problem with the proposed law. One of two thing…

>> Many of the prestigious journals do require an exclusive license Of course they do. But nothing forces the researcher to use those journals. So the question becomes why do they? Perhaps there's prestige involved? >> 1) Federally funded researchers publish elsewhere That's my point. Since researchers already have this option why are they not exercising it? Why are researchers happy, indeed prefer, publishing with S…

>Only by understanding why they currently choose to use Springer et al, can you understand what is lost by requiring free publishing.

Because we're stuck in a local sub-optima. Those journals are currently prestigious because top researchers publish in them and top researchers publish in them because they are prestigious.

A high acceptance bar may play a role but that is easily achievable and generally does not work anyway based on the number of fraudulent papers that have been accepted. Reviewer comments should be published alongside as should the data.

Re: The danger of relying on OpenAI's Deep Research

#58

Post by cratermoon »

casey2 wrote: post #45
cratermoon wrote: post #21

Earlier quoted context omitted.

Allow me to introduce The First Step Fallacy. https://thebullshitmachines.com/lesson-16-the-first-step-fal...

It's an inductive claim yes, nevertheless it's based on mountains of evidence. Nobody is claiming to have logically deduced that AI progress is inevitable, we might all drop dead tomorrow or stop working on it, but the trends are clear

> mountains of evidence

Yes? For very small values of 'mountain'.

Re: The danger of relying on OpenAI's Deep Research

#59

Post by azinman2 »

qingcharles wrote: post #48
azinman2 wrote: post #3

The world is going to become stupider because it’s easier. This will be most dramatic for the youth in both school learning as well as in the field. Unfortunately I don’t realistically see another path in any reasonable time unless the world takes a radical turn away from AI (extremely unlikely).

> "SHE PROCEEDS TO ASK CHATGPT TO ANSWER EVERY SINGLE QUESTION IN HER 2 PAGE MATH HOMEWORK and these include VERY SIMPLE QUESTIONS like how many hours are there in 1 day and 7 hours" https://www.reddit.com/r/ChatGPT/comments/1hun3e4/my_little_...

Exactly

Re: The danger of relying on OpenAI's Deep Research

#60

Post by marcus_holmes »

yunwal wrote: post #55
marcus_holmes wrote: post #47

Earlier quoted context omitted.

Both those things (PhD and high salary) require human interaction to achieve, not knowledge or tool use. There are many examples of people achieving both/either whilst being ignorant and/or unintelligent. The comment was about "you're not going to know anything because the LLM is doing it for you" which is easily obviously true. This won't stop anyone from getting a PhD or a high salary. It will just stop them from k…

> you're not going to know anything because the LLM is doing it for you" which is easily obviously true Not obviously true to me. LLMs don’t know everything, so they can’t solve all problems. You can try and give it the proper context but you still need to understand the context is necessary, which requires knowledge. Kids who grow up today will understand and intuit that synthesizing 3 facts from an article doesn’t…

yeah, one of the problems we're seeing with using LLMs at the moment is that they don't really work for beginners, because you need to have a certain level of knowledge in order to be able to spot the hallucinations and understand the context.

I've seen this in coding, where as an experienced coder I can spot where the LLM is doing bad things, but if I try using it in a language or environment I'm not familiar with, I get lots of errors that I don't understand how to fix and all I can do is feed back the errors to the LLM and hope it does better next time.

So I guess my point is that if we end up using LLMs for everything then we have a chicken & egg problem - LLMs don't work for beginners, but beginners don't learn anything while they're using LLMs because they can't get past basic errors.

Your point about multiplying 4-digit numbers would be valid if calculators often made basic maths mistakes, and you can only really use them if you already know the approximate answer so you can detect when they've made a mistake.

Post reply on HN