Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open Deep Research

github.com

Post reply on HN
Print view View original

41–50 of 83 posts

Re: Open Deep Research

#41

Post by pixl97 »

janalsncm wrote: post #30
swyx wrote: post #11

Earlier quoted context omitted.

theres always a lot of openTHING clones of THING after THING is announced. they all usually (not always[1]!) disappoint/dont get traction. i think the causes are 1. running things in production/self hosting is more annoying than just paying like 20-200/month 2. openTHING makers often overhype their superficial repros ("I cloned Perplexity in a weekend! haha! these VCs are clowns!") and trivializing the last mile, mos…

> RL in a tight loop that is not available in the open Completely agree that a real RL pipeline is needed here, not just some clever prompting in a loop. That being said, it wouldn’t be impossible to create a “gym” for this task. You are essentially creating a simulated internet. And hiding a needle is a lot easier than finding it.

I think models will have to some kind of internal training to teach them they are agents that can come back and work on things.

Working on complex problems tends to explode in to a web of things that needs done. You need to be able to separate these in to subtasks and work on them semi-independently. In addition when a subtask gets stuck in a loop, you need to work on another task or line of thought, and then come back and 're-run' your thinking to see if anything changed.

Re: Open Deep Research

#42

Post by pixl97 »

anon373839 wrote: post #38
tptacek wrote: post #17

Earlier quoted context omitted.

I think the opposite problem is going to occur with captchas for whatever it's worth: LLMs are going to obsolete them. It's an arms race where the defender has a huge constraint the attacker doesn't (pissing off real users); in that way, it's kind of like the opposite dynamics that password hashes exploit.

I’m not sure about that. There’s a lot of runway left for obstacles that are easy for humans and hard/impossible for AI, such as direct manipulation puzzles. (AI models have latency that would be impossible to mask.) On the other hand, a11y needs do limit what can be lawfully deployed…

>AI models have latency

So do humans, or can my friend with cerebral palsy not use the internet any longer?

Re: Open Deep Research

#43

Post by jsemrau »

aubanel wrote: post #31

Hi all! Aymeric (m-ric) here, maintainer of smolagents and part of the team who built this. Happy to see this interesting people here! Few points: - open Deep Research is not a production app, but it could easily be productionized (would need to be faster + good UX). - As the GAIA score of 55% (not 54%, that would be lame) says, it's not far from the Deep Research score of 67%. It's also not there yet: I think the ma…

I think using vision models for browsing is the wrong approach. It is the same as using OCR for scanning PDFs. The underlying text is already in digital form. So it would make more sense to establish a standard similar to meta-tags that enable the agentic web.

Re: Open Deep Research

#44

Post by tptacek »

ATechGuy wrote: post #18
tptacek wrote: post #9

Earlier quoted context omitted.

To which performance and resource requirements are you referring? A cloud VM runs as long as the agent runs, then stops running.

I mean performance overheads of an OS process running in a VM to (vs no VM) and additional resource requirements for running a VM, including memory and additional kernel. You can pull relevant numbers from academic papers.

OK. Thanks for clarifying. I think you're pretty wrong on this one, for what it's worth.

Re: Open Deep Research

#45

Post by yobid20 »

We need an OpenOpenAI to open source OpenAI who should actually be called ClosedAI, since there's nothing open about them other than their banks to take all your money.

Re: Open Deep Research

#47

Post by transpute »

ATechGuy wrote: post #18
tptacek wrote: post #9

Earlier quoted context omitted.

To which performance and resource requirements are you referring? A cloud VM runs as long as the agent runs, then stops running.

I mean performance overheads of an OS process running in a VM to (vs no VM) and additional resource requirements for running a VM, including memory and additional kernel. You can pull relevant numbers from academic papers.

A linear bar graph comparing compute/memory requirements?

  - OS process
  - virtual machine
  - LLM inference
Could have longevity as PC master race meme template.

Re: Open Deep Research

#48

Post by janalsncm »

pixl97 wrote: post #41
janalsncm wrote: post #30

Earlier quoted context omitted.

> RL in a tight loop that is not available in the open Completely agree that a real RL pipeline is needed here, not just some clever prompting in a loop. That being said, it wouldn’t be impossible to create a “gym” for this task. You are essentially creating a simulated internet. And hiding a needle is a lot easier than finding it.

I think models will have to some kind of internal training to teach them they are agents that can come back and work on things. Working on complex problems tends to explode in to a web of things that needs done. You need to be able to separate these in to subtasks and work on them semi-independently. In addition when a subtask gets stuck in a loop, you need to work on another task or line of thought, and then come ba…

The idea of reinforcement learning is that for some things it is hard to give an explicit plan for how to do something. For example, many games. Recently, DeepSeek showed that it worked for certain reasoning problems too, like leetcode problems.

Instead, RL just rewards the model when it accomplishes some measurable goal (like winning the game). This works for certain types of problems but it’s pretty inefficient because the model wastes a lot of time doing stuff that doesn’t work.

Re: Open Deep Research

#49

Post by ATechGuy »

vineyardmike wrote: post #10
ATechGuy wrote: post #5

Earlier quoted context omitted.

VM is not the right abstraction because of performance and resource requirements. VMs are used because nothing exists that provides same or better isolation. Using a throwaway VM for each AI agent would be highly inefficient (think wasted compute and other resources, which is the opposite of what DeepSeek exemplified).

Is “DeepSeek” going to be the new trendy way to say to not be wasteful? I don’t think DS is a good example here. Mostly because it’s a trendy thing, and the company still has $1B in capex spend to get there . Firecracker has changed the nature of “VMs” into something cheap and easy to spin up and throw away while maintaining isolation. There’s no reason not to use it (besides complexity, I guess). Besides, the entire…

I think you are assuming that inference happens on the same machine/VM that executes code generated by an AI agent.

Re: Open Deep Research

#50

Post by shekhargulati »

swyx wrote: post #11
transpute wrote: post #8

https://techcrunch.com/2025/02/04/hugging-face-researchers-a... > On GAIA, a benchmark for general AI assistants, Open Deep Research achieves a score of 54%. That’s compared with OpenAI deep research’s score of 67.36%..Worth noting is that there are a number of OpenAI deep research “reproductions” on the web, some of which rely on open models and tooling. The crucial component they — and Open Deep Research — lack is…

theres always a lot of openTHING clones of THING after THING is announced. they all usually (not always[1]!) disappoint/dont get traction. i think the causes are 1. running things in production/self hosting is more annoying than just paying like 20-200/month 2. openTHING makers often overhype their superficial repros ("I cloned Perplexity in a weekend! haha! these VCs are clowns!") and trivializing the last mile, mos…

> running things in production/self hosting is more annoying than just paying like 20-200/month

This is an important point. As people rely more on AI/LLM tools, reliability will become even more critical.

In the last two weeks, I've heavily used Claude and DeepSeek Chat. ChatGPT is much more reliable compared to both.

Claude struggles with long-context chats and often shifts to concise responses. DeepSeek often has its "fail whale" moment.

Post reply on HN