Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

simonwillison.net

Post reply on HN
Print view View original

131–140 of 268 posts

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#131

Post by j_bum »

simonw wrote: post #98
j_bum wrote: post #71

Is this the “Web Search”, “Deep Research”, or “Agent Mode” feature of ChatGPT? Navigating their feature set is… fun.

It's not the Deep Search or Agent Mode. I select "GPT-5 Thinking" from the model picker and make sure its regular search tool is enabled.

Good to know, I’ll try to just use this a bit more then. I always opt for one of the above modes, with varying degrees of success.

Not sure if you tend to edit your posts, but it could be worth clarifying.

Btw — my colleagues and I all love your posts. I’ll quit fanboying now lol.

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#132

Post by larsiusprime »

typpilol wrote: post #128
ants_everywhere wrote: post #127

Earlier quoted context omitted.

This isn't really how you described. You have an opinion that conflicts with the research literature. You published a blog about that opinion, and you want ChatGPT to say you're to accept your view. Your view is grinding a political axe and I don't think you're in a position to objectively assess whether ChatGPT failed in this case.

Yea this isn't really a chat gpt problem as a source credibility problem no?

It’s mostly that it was not citing verifiable - and available online - primary source documents, the way I would expect an actual researcher investigating this question would. This is relevant when it is billed as "Research Grade" or "PhD" level intelligence. I expect a PhD level researcher to find the German-language primary sources.

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#133

Post by yeasku »

the_pwner224 wrote: post #107
Pepe1vo wrote: post #85

Earlier quoted context omitted.

A counter example to this is that I asked it about NovaMin® 5 minutes ago and it essentially told me to not bother and buy whatever toothpaste has >1450 ppm fluoride.

A year ago I asked it to do deep research on Biomin F + a comparison to NovaMin & fluoride. It gave a comprehensive answer detailing the benefits of BioMin & NovaMin over regular fluroide.

A year ago I asked the change of dolar euro and it made up the number.

How do you know it did not made it up. Are you an expert in the field?

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#134

Post by tagawa »

edverma2 wrote: post #69

From GPT-5-Pro with Deep Research selected: > FWIW Deep Research doesn’t run on whatever you pick in the model selector. It’s a separate agent that uses dedicated o‑series research models: full mode runs on o3; after you hit the full‑mode cap it auto‑switches to a lightweight o4‑mini version. The picker governs normal chat (and the pre‑research clarifying Qs), not the research engine itself.

From the OP's comment above:

"It's not the Deep [Re]Search or Agent Mode. I select 'GPT-5 Thinking' from the model picker and make sure its regular search tool is enabled."

Source: https://news.ycombinator.com/item?id=45162802

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#135

Post by mastercheif »

psadri wrote: post #79

I do miss the earlier "heavy" models that had encyclopedic knowledge vs the new "lighter" models that rely on web search. Relying on web search surfaces a shallow layer of knowledge (thanks to SEO and all the other challenges of ranking web results) vs having ingested / memorized basically the entirety of human written knowledge beyond what's typically reachable within the first 10 results of a web search (eg: digiti…

I kept search off for a long time due to it tanking the quality of the responses from ChatGPT.

I recently added the following to my custom instructions to get the best of both worlds:

# Modes

When the user enters the following strings you should follow the following mode instructions:

1. "xz": Use the web tool as needed when developing your answer.

2. "xx": Exclusively use your own knowledge instead of searching the internet.

By default use mode "xz". The user can switch between modes during a chat session. Stay with the current mode until the user explicitly switches modes.

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#136

Post by typpilol »

the_pwner224 wrote: post #107
Pepe1vo wrote: post #85

Earlier quoted context omitted.

A counter example to this is that I asked it about NovaMin® 5 minutes ago and it essentially told me to not bother and buy whatever toothpaste has >1450 ppm fluoride.

A year ago I asked it to do deep research on Biomin F + a comparison to NovaMin & fluoride. It gave a comprehensive answer detailing the benefits of BioMin & NovaMin over regular fluroide.

I'd be curious if you have the same prompt and repeat it what you get.

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#137

Post by ants_everywhere »

larsiusprime wrote: post #129
ants_everywhere wrote: post #127

Earlier quoted context omitted.

This isn't really how you described. You have an opinion that conflicts with the research literature. You published a blog about that opinion, and you want ChatGPT to say you're to accept your view. Your view is grinding a political axe and I don't think you're in a position to objectively assess whether ChatGPT failed in this case.

What are you talking about? There are verifiable primary sources that ChatGPT was not citing. There are direct primary historical sources that lay out the full budget of the historical German colony in extreme detail, that directly contradict assertions made in the Silagi paper, that’s not a matter of opinion that’s a matter of verifiable fact. Also what “axe” am I grinding? The findings are specifically inconvenient…

From your blog you appear to be a Georgist or inspired by Georgist socialism. And given that you appear to have a business and blog related to these subjects, you give the impression that you're a sort of activist for Georgism. I.e. not just researching it by trying to advance it.

So just zooming out, that's not the right sort of setup for being an impartial researcher. And in your blog post your disagreements come off to me as wanting a sort of purity with respect to Georgism that I wouldn't be expected to be reflected in the literature.

I like Kant, but it would be a bit like me saying ChatGPT was fundamentally wrong because it considered John Rawls a Kantian because I can point to this or that paper where he diverges from Kant. I could even write a blog post describing this and pointing to primary sources. But Rawls is considered a Kantian and for good reason, and it would (in my opinion) be misleading for me to say that ChatGPT made a big failure mode because it didn't take my view on my pet subject as seriously as I wanted.

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#138

Post by softwaredoug »

I agree with Simon’s article but I usually think about “research” to mean comparing different kinds of evidence (not just the search part). Like evidence for the effectiveness of Obamacare. Or how some legal case may play out in the courts. Or how much The Critic influenced The Family Guy. Or even what the best way to use X feature of Y library.

I’ve found ChatGPT and other LLMS can struggle to evaluate evidence - to understand the biases behind sources - ie taking data from a sketchy think tank as gospel. I also have found in my work the more reasoning, the more hallucination. Especially when gathering many statistics.

That plus the usual sycophancy can cause the model to really want to find evidence to support your position. Even if you don’t think you’re asking a leading question, it can really want to answer your question in the affirmative.

I always ask ChatGPT do directly cite and evaluate sources. And try to get it in the mindset of comparing and contrasting arguments for and against. And I find I must argue against its points to see how it reacts.

More here https://softwaredoug.com/blog/2025/08/19/researching-with-ag...

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#139

Post by larsiusprime »

ants_everywhere wrote: post #137
larsiusprime wrote: post #129

Earlier quoted context omitted.

What are you talking about? There are verifiable primary sources that ChatGPT was not citing. There are direct primary historical sources that lay out the full budget of the historical German colony in extreme detail, that directly contradict assertions made in the Silagi paper, that’s not a matter of opinion that’s a matter of verifiable fact. Also what “axe” am I grinding? The findings are specifically inconvenient…

From your blog you appear to be a Georgist or inspired by Georgist socialism. And given that you appear to have a business and blog related to these subjects, you give the impression that you're a sort of activist for Georgism. I.e. not just researching it by trying to advance it. So just zooming out, that's not the right sort of setup for being an impartial researcher. And in your blog post your disagreements come o…

You misunderstand. I’m indeed a Georgist, and I discovered that a popular Georgist narrative was exaggerated! The findings of the historically verifiable primary source documents contradicted a prevailing narrative based on the Silagi paper. The Silagi paper is pro Georgist! But it’s exaggerated!

The literature — the primary source documents — do not in fact support a maximalist Georgist case! This is what I have been trying to say!!!

You are accusing me of the exact opposite thing I’m arguing for!!! The historical case the primary sources show is inconvenient for my political movement!

The failure of chat gpt is not that it disagrees with any opinion of mine, but that it does not surface primary source documents. That’s the issue.

Its baffling to be accused of confirmation bias when I point out research findings that goes against what would be maximally convenient for my own cause.

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#140

Post by tasercake »

croemer wrote: post #83

Yes, "GPT-5 with thinking" is great at search, but it's horrible that it shows "Network connection lost. Attempting to reconnect..." after you switch away from the app for even just a few seconds before coming back. It's going to take a minute, so why do I need to keep looking at it and can't go read some more Wikipedia in the mean time? This is insanely user hostile. Is it just me who encounters this? I'm on Plus pl…

I’ve had this happen on iOS too, usually when I switch away from the thread or the app before it progresses past the initial “Thinking…”.

But I’ve found that no matter the error - even if I disconnect from the internet entirely - I eventually get a push notification and opening up the thread a while later shows me the full response. (disclaimer: N=1)

Post reply on HN