Live data from Hacker News

GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

simonwillison.net

151–160 of 268 posts

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#151
post #98
post #71

Is this the “Web Search”, “Deep Research”, or “Agent Mode” feature of ChatGPT? Navigating their feature set is… fun.

It's not the Deep Search or Agent Mode. I select "GPT-5 Thinking" from the model picker and make sure its regular search tool is enabled.

> This is excellent for satisfying curiosity, and occasionally useful for more important endeavors as well.

Small nit, Simon: satisfying curiosity is the important endeavor.

<3

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#152

Earlier quoted context omitted.

Don’t sleep on Gemini Deep Research feature either. I use it for my car work and it beats ChatGPT’s offering at that price point every time.

I dunno, I use Deep Research from Claude, ChatGPT, and Gemini, and Gemini is the only one that ignores my requests and always produces the most inane high school student wannabe management consultant "report" with introduction and restatement of the problem and background and all that. Its "voice" (the prose, I mean, not text to speech) is so irritating I've stopped using it. The other ones will do the thing I want:…

Suppose it depends. I think of it like this article suggests. It is very good at searching and scraping a lot of websites fast. And then summarizing that some.

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#153

Earlier quoted context omitted.

Don’t sleep on Gemini Deep Research feature either. I use it for my car work and it beats ChatGPT’s offering at that price point every time.

I dunno, I use Deep Research from Claude, ChatGPT, and Gemini, and Gemini is the only one that ignores my requests and always produces the most inane high school student wannabe management consultant "report" with introduction and restatement of the problem and background and all that. Its "voice" (the prose, I mean, not text to speech) is so irritating I've stopped using it. The other ones will do the thing I want:…

I like Gemini Deep Research because ChatGPT's has very low limits, but it is extremely on rails. Yesterday as an experiment I asked it to do a bunch of math rather than write a report, and it did the math but then wrote a report scolding me for not appreciating the beauty of the humanities.

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#154
post #83

Yes, "GPT-5 with thinking" is great at search, but it's horrible that it shows "Network connection lost. Attempting to reconnect..." after you switch away from the app for even just a few seconds before coming back. It's going to take a minute, so why do I need to keep looking at it and can't go read some more Wikipedia in the mean time? This is insanely user hostile. Is it just me who encounters this? I'm on Plus pl…

This is fine because it does reconnect, but some actions cause it to just stop thinking and not write a response, and I can't tell what they are. Wastes a few minutes because you have to re-run it.

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#155

Earlier quoted context omitted.

Don’t sleep on Gemini Deep Research feature either. I use it for my car work and it beats ChatGPT’s offering at that price point every time.

I dunno, I use Deep Research from Claude, ChatGPT, and Gemini, and Gemini is the only one that ignores my requests and always produces the most inane high school student wannabe management consultant "report" with introduction and restatement of the problem and background and all that. Its "voice" (the prose, I mean, not text to speech) is so irritating I've stopped using it. The other ones will do the thing I want:…

[deleted]

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#156

It really is great. When I was still on Reddit, I made regular use of the "Tip of My Tongue" sub to track down obscure stuff I half-remembered from years ago. It mostly worked, but there were a few stubborn cases that went unsolved, even after pouring every ounce of my Google Fu into the endeavor. I recently took the text of these unsolved posts and submitted them to Deep Research -- and within an hour, it had cracke…

When it comes to Information Retrieval, you can get anything between links to existing documents or generated content based on those processed information. I agree that the second one is really powerfuly and just amazing and seemilngly useful. But the fact that it can also be wrong in more cases and I won't know keep being reminded to my using it for things I'm not good at and they just don't work s they should.

I just wish the business models could justify a confidence level being attached to the response.

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#157
I guess the part where I'm still skeptical are: Google is also still pretty good at search (especially if I avoid the AI summary with udm=14).

I'll take one of your examples: Britannica to seed Wikipedia. I searched for "wikipedia encyclopedia brtannica". In less than 1 second, I got search results back.

I spend maybe 30 seconds scanning the page; past the Wikipedia article on Encyclopedia Britannica, past the Encyclopedia article about Wikipedia, past a Reddit thread comparing them, past the Simple English Wikipedia article on Britannica, and past the Britannica article on Wiki. OK, there it is, the link to "Wikipedia:WikiProject Encyclopaedia Britannica", that answers your question.

Then to answer your follow up, I spend a couple more seconds to search Wikipedia for Wikipedia, and find in the first paragraph that it was founded in 2001.

So, let's say a grand total of 60 seconds of me searching, skimming, and reading the results. The actual searching was maybe 2 or 3 seconds of time total, once on Google, and once on Wikipedia.

Compared to nearly 3 minutes for ChatGPT to grind through all of that, plus the time for you to read it, and hopefully verify by checking its references because it can still hallucinate.

And what did you pay for the privilege of doing that? How much extra energy did you burn for this less efficient response? I wish that when linking to chat transcripts like you do, ChatGPT would show you the token cost of that particular chat

So yeah, it's possible to do search with ChatGPT. But it seems like it's slower and less efficient than searching and skimming yourself, at least for this query.

That's generally been my impression of LLMs; it's impressive that they can do X. But when you add up all the overhead of asking them to do X, having them reason about it, checking their results, following up, and dealing with the consequences of any mistakes, the alternative of just relying on plain old search and your own skimming seems much more efficient.

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#159

Earlier quoted context omitted.

I feel the opposite. Before I can use information from a model's "internal" knowledge I have to engage in independent research to verify that it's not a hallucination. Having an LLM generate search strings and then summarize the results does that research up front and automatically, I need only click the sources to verify. Kagi Assistant does this really well.

So does anyone have any good examples of it effectively avoiding the blogspam and SEO? Or being fooled by it? How often either way?

I find one thing it doesn't do very well is avoiding marketing articles pushed by a brand itself. e.g. if I search is X better than Y, very likely landing on articles by makers of brand X and Y and not a 3rd party reviewer. When I manually search on Google I can spot marketing articles just by the URL.

Re: GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is good at search

#160
> Starbucks in the UK don’t sell cake pops! Do a deep investigative dive

...

I used to play games on my computer a lot. Not so much anymore, don't really want to lock myself in a room alone and play games. I have kids and a wife, and it feels isolative.

But those days I would, and often the hardware I had was underpowered to be able to experience the game in its full glory. I would often spend hours and hours just honing settings and config and environment to get the game to run at peak capability on my machine.

At some point, I would reach a zenith. Some perfect arrangement of settings and environment that gave me a game running at top quality on my machine (or as close to top as I could get). The experience for me is joyous. So enjoyable that I often didn't even play the game except maybe to test the boundaries of its performance at that level.

Reading this article made me sad for people who don't put in work for some sort of accomplishment that amounts to nothing. And it made me think of my own experience with it. Accomplishment for its own sake is still accomplishment. And it's still self realization, which is important to existing.

Post reply on HN