Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

tongyi-agent.github.io

Post reply on HN
Print view View original

31–40 of 156 posts

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#32

Post by pstuart »

btbuildem wrote: post #23
aliljet wrote: post #5

Sunday morning, and I find myself wondering how the engineering tinkerer is supposed to best self-host these models? I'd love to load this up on the old 2080ti with 128gb of vram and play, even slowly. I'm curious what the current recommendation on that path looks like. Constraints are the fun part here. I know this isn't the 8x Blackwell Lamborghini, that's the point. :)

I've recently put together a setup that seemed reasonable for my limited budget. Mind you, most of the components were second-hand, open box deals, or deep discount of the moment. This comfortably fits FP8 quantized 30B models that seem to be "top of the line for hobbyists" grade across the board. - Ryzen 9 9950X - MSI MPG X670E Carbon - 96GB RAM - 2x RTX 3090 (24GB VRAM each) - 1600W PSU

That's basically what I imagined would be my rig if I were to pull the trigger. Do you have an NVLink adapter as well?

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#33

Post by jwr »

aliljet wrote: post #5

Sunday morning, and I find myself wondering how the engineering tinkerer is supposed to best self-host these models? I'd love to load this up on the old 2080ti with 128gb of vram and play, even slowly. I'm curious what the current recommendation on that path looks like. Constraints are the fun part here. I know this isn't the 8x Blackwell Lamborghini, that's the point. :)

I just use my laptop. A modern MacBook Pro will run ~30B models very well. I normally stick to "Max" CPUs (initially for more performance cores, recently also for the GPU power) with 64GB of RAM. My next update will probably be to 128GB of RAM, because 64GB doesn't quite cut it if you want to run large Docker containers and LLMs.

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#39

Post by sumo43 »

aliljet wrote: post #5

Sunday morning, and I find myself wondering how the engineering tinkerer is supposed to best self-host these models? I'd love to load this up on the old 2080ti with 128gb of vram and play, even slowly. I'm curious what the current recommendation on that path looks like. Constraints are the fun part here. I know this isn't the 8x Blackwell Lamborghini, that's the point. :)

Try running this using their harness https://huggingface.co/flashresearch/FlashResearch-4B-Thinki...

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#40

Post by yalogin »

In my experience using these supposed expert models, they are all more or less the same given they all are trained on the same internet data. The differentiation and value is in the context window management and how relevant info from your session is pulled in. So it’s the interface to the model that makes all the difference. Even there the differences are quite minimal. That is because all these companies want to toe the line between providing functionality to keep the users engaged and pushing them to sign up for the subscription.

All this to ask the question, if I host these open source models locally, how is the user interface layer that remembers and picks the right data from my previous session and the agentic automation and others implemented? Do I have to do it myself or are the free options for that?

Post reply on HN