Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI assisted search-based research works now

simonwillison.net

Post reply on HN
Print view View original

111–120 of 156 posts

Re: AI assisted search-based research works now

#111

Post by mz00 »

csallen wrote: post #105

It's actually quite doable to build your own deep research agent. You just need a single prompt, a solid code loop to run it agentically, and some tools for it to call. I've been building a domain-specific deep research agent over the past few days for internal use, and I'm pretty impressed with how much better it is than any of the official deep search agents for my use case.

same here. I first built the agentic workflow in python and later nextjs. uses dozens of llm apis. works well, and am also impressed with the results.

Re: AI assisted search-based research works now

#112

Post by FieryTransition »

spongebobstoes wrote: post #103
otistravel wrote: post #94

The most impressive demos of these tools always involve technical tasks where the user already knows enough to verify accuracy. But for the average person asking about health issues, legal questions, or historical facts? It's basically fancy snake oil - confident-sounding BS that people can't verify. The real breakthrough would be systems that are actually trustworthy without human verification, not slightly better B…

this remains true for pretty much all advice or information we receive. doctors, lawyers, accountants, teachers. there have been countless times that all of these professionals have given me bad advice or information sure, at least I have someone to blame in that case. but in my experience, the AI is at least as reliable as a person who I don't personally know

Plenty studies show that these models are better at catching and diagnosing than even a board of doctors are. Doctors are good at other things, and I hope the future will allow doctors to use these models together with their practice.

The problem is when the ai makes a catastrophic prediction, and the layman can't see it.

Re: AI assisted search-based research works now

#113

Post by gcanyon »

The concern over "LLMs vs. the Web" is giving me serious "The Web vs. Brick and Mortar" vibes. That's not to say that it won't be the predicted cataclysm, just that it might not be. Time will tell, because This Is Happening, People, but if it does turn out to be a serious problem, I think we'll find some way to adapt. We're unlikely to accept a lesser result.

Re: AI assisted search-based research works now

#114

Post by otabdeveloper4 »

spongebobstoes wrote: post #103
otistravel wrote: post #94

The most impressive demos of these tools always involve technical tasks where the user already knows enough to verify accuracy. But for the average person asking about health issues, legal questions, or historical facts? It's basically fancy snake oil - confident-sounding BS that people can't verify. The real breakthrough would be systems that are actually trustworthy without human verification, not slightly better B…

this remains true for pretty much all advice or information we receive. doctors, lawyers, accountants, teachers. there have been countless times that all of these professionals have given me bad advice or information sure, at least I have someone to blame in that case. but in my experience, the AI is at least as reliable as a person who I don't personally know

> but in my experience, the AI is at least as reliable as a person who I don't personally know

How do you know this?

Re: AI assisted search-based research works now

#115

Post by intended »

FieryTransition wrote: post #112
spongebobstoes wrote: post #103

Earlier quoted context omitted.

this remains true for pretty much all advice or information we receive. doctors, lawyers, accountants, teachers. there have been countless times that all of these professionals have given me bad advice or information sure, at least I have someone to blame in that case. but in my experience, the AI is at least as reliable as a person who I don't personally know

Plenty studies show that these models are better at catching and diagnosing than even a board of doctors are. Doctors are good at other things, and I hope the future will allow doctors to use these models together with their practice. The problem is when the ai makes a catastrophic prediction, and the layman can't see it.

Plenty of studies is news to me. I’ve only seen anecdotal content.

Re: AI assisted search-based research works now

#116

Post by esafak »

oulipo wrote: post #95
esafak wrote: post #39

Earlier quoted context omitted.

Programming is not real world?

I said "the main use cases", not "the little toys to distract and amuse engineers while the ruin the environment with CO2 emissions"

It is not a little toy, and when you consider the carbon footprint of humans and the productivity savings afforded by AI the balance evens out. AI is a very powerful technology if you know how to wield it.

Re: AI assisted search-based research works now

#117

Post by NewsaHackO »

intended wrote: post #115
FieryTransition wrote: post #112

Earlier quoted context omitted.

Plenty studies show that these models are better at catching and diagnosing than even a board of doctors are. Doctors are good at other things, and I hope the future will allow doctors to use these models together with their practice. The problem is when the ai makes a catastrophic prediction, and the layman can't see it.

Plenty of studies is news to me. I’ve only seen anecdotal content.

Yes, and I am pretty sure this is already an established phenomenon; AI and ML in general are able to apply domain specific algorithms better than the people who wrote them, because of how much it gets exposed to it during training.

Re: AI assisted search-based research works now

#118

Post by arkh »

neural_thing wrote: post #107
spongebobstoes wrote: post #103

Earlier quoted context omitted.

this remains true for pretty much all advice or information we receive. doctors, lawyers, accountants, teachers. there have been countless times that all of these professionals have given me bad advice or information sure, at least I have someone to blame in that case. but in my experience, the AI is at least as reliable as a person who I don't personally know

I tested o3 on a medical issue I've had that 50+ doctors couldn't diagnose over the span of 6-7 years, ended up figuring it out through sheer luck. With my first prompt, it gave a list of probabilities, with the correct answer being listed as the third most likely. It also suggested correct tests to run for every option. I trust it way more than I trust human doctors who were confidently wrong about me for years.

I'm still thinking a simple expert system used by some nurse used to getting people to explain their symptoms would be enough to replace a general MD.

Less time and money spent to train those nurses, which you can then spend on training specialists. And your expert system will take less time to update than training thousands of doctor every time some new protocol or drug is released.

Re: AI assisted search-based research works now

#119

Post by Tycho »

I tried o3 for a couple of things.

First one, geolocation a photo I saw in a museum. It didn’t find a definitive answer but it sure turned up a lot of fascinating info in its research.

Second one, I asked it to suggest a new line of enquiry in the Madeleine McCann missing person case. It made the interesting suggestion that the 30 minute phone call the suspect made on the evening of the disappearance, from a place near the location of the abduction, was actually a sort of “lookout call” to an accomplice nearby.

Quite impressed. This is a great investigative tool.

Re: AI assisted search-based research works now

#120

Post by franze »

neural_thing wrote: post #107
spongebobstoes wrote: post #103

Earlier quoted context omitted.

this remains true for pretty much all advice or information we receive. doctors, lawyers, accountants, teachers. there have been countless times that all of these professionals have given me bad advice or information sure, at least I have someone to blame in that case. but in my experience, the AI is at least as reliable as a person who I don't personally know

I tested o3 on a medical issue I've had that 50+ doctors couldn't diagnose over the span of 6-7 years, ended up figuring it out through sheer luck. With my first prompt, it gave a list of probabilities, with the correct answer being listed as the third most likely. It also suggested correct tests to run for every option. I trust it way more than I trust human doctors who were confidently wrong about me for years.

totally agree, identified an infection with a dangerous bacteria based on a single photo - the doctor just thought about if after presented with the AI opinion
Post reply on HN