Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI assisted search-based research works now

simonwillison.net

Post reply on HN
Print view View original

131–140 of 156 posts

Re: AI assisted search-based research works now

#131

Post by simonw »

southernplaces7 wrote: post #98
danielmarkbruce wrote: post #59

Earlier quoted context omitted.

This is just a bad match to the capabilities. What you are actually looking for is analysis, similar in nature to what a data scientist may do. The deep research capabilities are much better suited to more qualitative research / aggregation.

Your logic is.... strange... Because it failed miserably at a very simple task of looking through some scattered charts, the human asking should blame themselves for this basic failure and trust it to do better with much harder and more specialized tasks?

"Don’t fall into the trap of anthropomorphizing LLMs and assuming that failures which would discredit a human should discredit the machine in the same way." - https://simonwillison.net/2025/Mar/11/using-llms-for-code/#s...

Re: AI assisted search-based research works now

#133

Post by danielmarkbruce »

southernplaces7 wrote: post #98
danielmarkbruce wrote: post #59

Earlier quoted context omitted.

This is just a bad match to the capabilities. What you are actually looking for is analysis, similar in nature to what a data scientist may do. The deep research capabilities are much better suited to more qualitative research / aggregation.

Your logic is.... strange... Because it failed miserably at a very simple task of looking through some scattered charts, the human asking should blame themselves for this basic failure and trust it to do better with much harder and more specialized tasks?

No logic needed. Just use them, build them, play with them. You'll figure out what they are good at and what they aren't good at.

Re: AI assisted search-based research works now

#134

Post by energy123 »

loufe wrote: post #22
energy123 wrote: post #19

> The user-facing Google Gemini app can search too, but it doesn’t show me what it’s searching for. Gemini 2.5 Pro is also capable of search as part of its chain of thought but it needs light prodding to show URLs, but it'll do so and is good at it. Unrelated point, but I'm going to keep saying this anywhere Google engineers may be reading, the main problem with Gemini is their horrendous web app riddled with 5 annoy…

There must be some form of memory leak in AI Studio as I'll have to close and open a new tab after about 2 hours as it slowly grinds my slower computers to a halt. Its ability to create a markdown file without escaping the markdown itself (included code snippets) is definitely my first suggestion for them to fix. It's a great tool, but sometimes frustrating.

It's a great model, but the web developers that built the web app either don't care or are incompetent.

Re: AI assisted search-based research works now

#135

Post by riku_iki »

ljsprague wrote: post #101
riku_iki wrote: post #84

Earlier quoted context omitted.

https://chatgpt.com/share/6807278c-a0d0-8006-80f3-f62ae9f8ff...

Did you run the code?

No, but even if it is buggy, it is usual LLM cycle:

- you: there is no such function

- LLM: you are absolutely right, here is fixed code

Re: AI assisted search-based research works now

#136

Post by [deleted] »

gilbetron wrote: post #122
CSMastermind wrote: post #57

The various deep research products don't work well for me. For example I asked these tools yesterday, "How many unique NFL players were on the roster for at least one regular season game during the 2024 season? I'd like the specific number not a general estimate." I as a human know how to find this information. The game day rosters for many NFL teams are available on many sites. It would be tedious but possible for m…

To avoid "result corruption" I asked a similar question, but for NBA players, and used O4-mini, and got a specific answer: "For the 2023‑24 NBA regular season (which ran from October 24, 2023 to April 14, 2024), a total of 561 distinct players logged at least one game appearance, as indexed by their “Rk” on the Basketball‑Reference “Player Stats: Totals” page (the final rank shown is 561)" Doing a quick search on my…

[deleted]

Re: AI assisted search-based research works now

#137

Post by sanderjd »

sitkack wrote: post #126
navinsylvester wrote: post #55

Earlier quoted context omitted.

Ironically, https://context7.com/

This is devdocs to be consumed by LLMs, https://github.com/upstash/context7 Brilliant (and one less think I don't have to build)!

I'm glad this exists, but can you describe to me why it needs to? Why can't agents just read the docs directly?

Re: AI assisted search-based research works now

#138

Post by BambooBandit »

I feel like the bigger problem isn't whether these deep research products work, but rather the raw material, so to speak, that they're working with.

For example, a lot of the "sources" cited in Google's AI Overview (notably not a deep research product) are not official, just sites that probably rank high in SEO. I want the original source, or a reliable source, not joeswebsite dot com (no offense to this website if it indeed exists).

Re: AI assisted search-based research works now

#139

Post by sitkack »

sanderjd wrote: post #137
sitkack wrote: post #126

Earlier quoted context omitted.

This is devdocs to be consumed by LLMs, https://github.com/upstash/context7 Brilliant (and one less think I don't have to build)!

I'm glad this exists, but can you describe to me why it needs to? Why can't agents just read the docs directly?

By returning detailed docs for exactly what the AI is coding at the time, it greatly reduces the the likelihood it will make a mistake. It moves from a recall from the training data problem, to a transcription problem.

This is RAG but for API docs.

Re: AI assisted search-based research works now

#140

Post by daveguy »

simonw wrote: post #131
southernplaces7 wrote: post #98

Earlier quoted context omitted.

Your logic is.... strange... Because it failed miserably at a very simple task of looking through some scattered charts, the human asking should blame themselves for this basic failure and trust it to do better with much harder and more specialized tasks?

"Don’t fall into the trap of anthropomorphizing LLMs and assuming that failures which would discredit a human should discredit the machine in the same way." - https://simonwillison.net/2025/Mar/11/using-llms-for-code/#s...

What I got out of that essay is that you should discredit most responses of LLMs unless you want to do just as much or more work yourself confirming the accuracy of an unreliable and deeply flawed partner. Whereas if a human "hallucinated a non-existent library or method you would instantly lose trust in them." But, for reasons, we should either give the machine the benefit of the doubt or manually confirm everything.
Post reply on HN