Deep Research is now available on Gemini 2.5 Pro Experimental
1–10 of 26 posts
Re: Deep Research is now available on Gemini 2.5 Pro Experimental
#2I stumbled across the feature a few hours ago. I had asked Gemini why there's a hole in the middle of the city of Azusa, topologically speaking. It had given me a useless tautological response: because they never annexed it. Then it offered to create a research report and I agreed. Five minutes later I got a notification on my mobile that the report was ready. It had 120 sources including assessor's maps, historical maps, court cases, and narrative articles. The text that went along with it was too verbose and still contained paragraphs of vague stuff, but it had key information linking the Mexican land grants, the founding of the city, and other events of history. Very impressive.
Re: Deep Research is now available on Gemini 2.5 Pro Experimental
#3Has anyone tested googles functionality vs ChatGPT? I have lightly played around with it but felt that generally ChatGPTs implementation was a little more educated sounding and felt like it took whatever necessary persona well.
Re: Deep Research is now available on Gemini 2.5 Pro Experimental
#4> In our testing, raters preferred the reports generated by Gemini Deep Research powered by 2.5 Pro over other leading deep research providers by more than a 2-to-1 margin.
Are these raters experts in the field the report was written on? Did they rate the reports on factuality, broadness, and insights?
These sort of tests (and RLHF in general) are the reason that LLMs often respond with "Great question, you are exactly right to wonder..." or "Interesting insight, I agree that...". I do not want this obsequious behavior, I want "correct answers"[0]. We need some better benchmarks when it comes to human preference.
[0]: I know there is no objective correct answer for some questions.
Re: Deep Research is now available on Gemini 2.5 Pro Experimental
#5Has anyone tested googles functionality vs ChatGPT? I have lightly played around with it but felt that generally ChatGPTs implementation was a little more educated sounding and felt like it took whatever necessary persona well.
Just did a test last week and OpenAIs research was way better. Found 10x more sources and did an overall pretty great job
The task was to lookup information about a late distant family member who had been a prominent employee in a certain foreign government about 100 years ago
Gemini barely scratched the surface and pretty much gave up
ChatGPT on the other hand, kept building up on its research, connecting the dots and leveraging each bit of acquired information to try to find more
Re: Deep Research is now available on Gemini 2.5 Pro Experimental
#6Deep research used to mean spending a weekend with grep and a coffee pot. Now it’s just autocomplete with a confidence interval.
Re: Deep Research is now available on Gemini 2.5 Pro Experimental
#7Has anyone tested googles functionality vs ChatGPT? I have lightly played around with it but felt that generally ChatGPTs implementation was a little more educated sounding and felt like it took whatever necessary persona well.
Just did a test last week and OpenAIs research was way better. Found 10x more sources and did an overall pretty great job The task was to lookup information about a late distant family member who had been a prominent employee in a certain foreign government about 100 years ago Gemini barely scratched the surface and pretty much gave up ChatGPT on the other hand, kept building up on its research, connecting the dots a…
Would love to see this repeated with this latest version from Google.
Man, what's really missing from all of this is a 3rd party AI Consumer Reports type site for all of these LLM tools. Whoever does this thing that does not scale will have a highly referenced site on their hands.
Re: Deep Research is now available on Gemini 2.5 Pro Experimental
#8Deep research used to mean spending a weekend with grep and a coffee pot. Now it’s just autocomplete with a confidence interval.
> with grep
Maybe for an extremely limited number of people. For the rest of the world, it meant searching the web, books, or scholarly publications, reading a ton, taking notes, and then possibly creating a report. Which is pretty much exactly what these AI agents are claimed to to, so deep research is the perfect name for it. Whether or not they are good at it compared to humans is a question that hasn't been answered to my satisfaction yet, but the name I'm fine with.
Re: Deep Research is now available on Gemini 2.5 Pro Experimental
#9Deep research used to mean spending a weekend with grep and a coffee pot. Now it’s just autocomplete with a confidence interval.
It doesn't have a confidence interval. We can only dream...
Re: Deep Research is now available on Gemini 2.5 Pro Experimental
#10I stumbled across the feature a few hours ago. I had asked Gemini why there's a hole in the middle of the city of Azusa, topologically speaking. It had given me a useless tautological response: because they never annexed it. Then it offered to create a research report and I agreed. Five minutes later I got a notification on my mobile that the report was ready. It had 120 sources including assessor's maps, historical…
That sounds fascinating. Please share if it's not too private.