Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI assisted search-based research works now

simonwillison.net

Post reply on HN
Print view View original

31–40 of 156 posts

Re: AI assisted search-based research works now

#31

Post by simonw »

bluefirebrand wrote: post #29
simonw wrote: post #26

Earlier quoted context omitted.

Using AI for coding effectively involves getting very good at testing (both manual and automated) and code review: https://simonwillison.net/2025/Mar/2/hallucinations-in-code/...

Manual testing, automated testing, and code review All three of those things are things that software engineers rather reliably are bad at and cut corners on, because they are the least engaging and least interesting part of the job of building software

Yep. Engineers who aren't willing to invest in those skills will have limited success with AI-assisted development.

I've seen a few people state that they don't like using LLMs because it takes away the fun part (writing the code) and leaves them with the bits they don't enjoy.

Re: AI assisted search-based research works now

#32

Post by sshine »

simonw wrote: post #7
jsemrau wrote: post #3

My main observation here is 1. Technically it might be possible to search the Internet, but it might not surface correct and/or useful information. 2. High-value information that would make a research report valuable is rarely public nor free. This holds especially true in capital-intensive or regulated industries.

I fully expect one of the AI-related business models going forward to be charging subscriptions for LLM search tool access to those kinds of archives. ChatGPT plus an extra $30/month for search access to a specific archive would make sense to me.

Kagi is $10/mo. for search and +$15/mo. for premium LLMs with agentic access to search.

Re: AI assisted search-based research works now

#33

Post by Gracana »

saulpw wrote: post #13

I tried it recently. I asked for videochat services like the one I use (WB) with 2 specific features that the most commonly used services don't have. It asked some clarifying questions and seemed to understand the mission, then went off for 10 minutes after which it returned 5 results in a table. The first result was WB, which I gave to it as the first example and am already using. Results 2 and 3 were the mainstream…

You can tell it to try again. It took me a couple rounds with the tool before I noticed that your conversation after the initial research isn't limited to just chatting: if you select the "deep research" button on your message, it will run the search process in its response.

Re: AI assisted search-based research works now

#34

Post by [deleted] »

intended wrote: post #12

I find that these conversations on HN end up covering similar positions constantly. I believe that most positions are resolved if 1) you accept that these are fundamentally narrative tools. They build stories, In whatever style you wish. Stories of code, stories of project reports. Stories or conversations. 2) this is balanced by the idea that the core of everything in our shared information economy is Verification.…

[deleted]

Re: AI assisted search-based research works now

#35

Post by gh0stcat »

intended wrote: post #12

I find that these conversations on HN end up covering similar positions constantly. I believe that most positions are resolved if 1) you accept that these are fundamentally narrative tools. They build stories, In whatever style you wish. Stories of code, stories of project reports. Stories or conversations. 2) this is balanced by the idea that the core of everything in our shared information economy is Verification.…

You've perfectly captured my experience as well, I typically only trust it and have good experiences with LLMs when I have enough domain expertise to get to at least a 95% confidence the output is correct. (Specific to my domain of work, I don't always need "perfect"). I also can mostly use it as a first pass for getting the idea of where to begin research, after that I lose confidence that the more detailed and advanced content it is giving me is accurate. There is a gray area though where a domain expert might have a false sense of confidence, and over time experience "Skill Drift", where they lose expertise because they are only ever verifying a lossy compression of information, rather than re-setting their context with real world information. I am mostly concerned with that last bit.

Re: AI assisted search-based research works now

#37

Post by jillesvangurp »

baq wrote: post #18

> I can feel my usage of Google search taking a nosedive already. Conveniently Gemini is the best frontier model for everything else, they’re very interested and well positioned (if not best?) to also be the best in deep research. Let’s check back in 3-6 months.

Google has two advantages:

1) Their AI models aren't half bad. Gemini 2.5 seems to be doing quite well relative to some competitors.

2) They know how to scale this stuff. They have their own hardware, lots of data, etc.

Scaling is of course the hard part. Doing things at Google scale means doing it well while still making a profit. Most AI companies are just converting VC cash into GPUs and energy. VC subsidized AI is nice at a small scale but cripplingly expensive at a larger scale. Google can't do this; they are too large for that. But they are vertically integrated, build their own data centers, with their own TPUs, etc. So, once this starts happening at their scale, they might just have an advantage.

A lot of what we are seeing is them learning to walk before they start running faster. Most of the world has no clue what perlexity is or any notion of the pros and cons of claude 3.7 sonnet vs. o4 mini high. None of that stuff matters long term. What matters is who can do this stuff well enough for billions of people.

So, I wouldn't count them out. But none of this stuff guarantees success either, of course.

Re: AI assisted search-based research works now

#39

Post by esafak »

oulipo wrote: post #20

The main "real-world" use cases for AI use for now have been: - shooting buildings in Gaza https://apnews.com/article/israel-palestinians-ai-weapons-43... - compiling a list of information on Government workers in US https://www.msn.com/en-us/news/politics/elon-musk-s-doge-usi... - creating a few losy music videos I'd argue we'd be better off SLOWING DOWN with that shit

Programming is not real world?

Re: AI assisted search-based research works now

#40

Post by bluefirebrand »

simonw wrote: post #31
bluefirebrand wrote: post #29

Earlier quoted context omitted.

Manual testing, automated testing, and code review All three of those things are things that software engineers rather reliably are bad at and cut corners on, because they are the least engaging and least interesting part of the job of building software

Yep. Engineers who aren't willing to invest in those skills will have limited success with AI-assisted development. I've seen a few people state that they don't like using LLMs because it takes away the fun part (writing the code) and leaves them with the bits they don't enjoy.

> Engineers who aren't willing to invest in those skills

Are bad engineers

> AI-assisted development

Are also bad engineers

Post reply on HN