Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

tongyi-agent.github.io

Post reply on HN
Print view View original

51–60 of 156 posts

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#51

Post by nine_k »

btbuildem wrote: post #23
aliljet wrote: post #5

Sunday morning, and I find myself wondering how the engineering tinkerer is supposed to best self-host these models? I'd love to load this up on the old 2080ti with 128gb of vram and play, even slowly. I'm curious what the current recommendation on that path looks like. Constraints are the fun part here. I know this isn't the 8x Blackwell Lamborghini, that's the point. :)

I've recently put together a setup that seemed reasonable for my limited budget. Mind you, most of the components were second-hand, open box deals, or deep discount of the moment. This comfortably fits FP8 quantized 30B models that seem to be "top of the line for hobbyists" grade across the board. - Ryzen 9 9950X - MSI MPG X670E Carbon - 96GB RAM - 2x RTX 3090 (24GB VRAM each) - 1600W PSU

Does it offer more performance than a Macbook Pro that could be had for a comparable sum? Your build can be had for under $3k; a used MBP M3 with 64 GB RAM can be had for approximately $3.5k.

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#52

Post by ninetyninenine »

DataDaemon wrote: post #35

Unfortunately soon China will take lead in AI.

Isn't this an indication they are already in the lead? They currently have the best model that beats everyone on all quantitative metrics? Are you implying that the US has a better model somewhere?

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#54

Post by MaxMatti »

jlokier wrote: post #28
aliljet wrote: post #5

Sunday morning, and I find myself wondering how the engineering tinkerer is supposed to best self-host these models? I'd love to load this up on the old 2080ti with 128gb of vram and play, even slowly. I'm curious what the current recommendation on that path looks like. Constraints are the fun part here. I know this isn't the 8x Blackwell Lamborghini, that's the point. :)

I use a Macbook Pro with 128GB RAM "unified memory" that's available to both CPU and GPU. It's slower than a rented Nvidia GPU, but usable for all the models I've tried (even gpt-oss-120b), and works well in a coffee shop on battery and with no internet connection. I use Ollama to run the models, so can't run the latest until they are ported to the Ollama library. But I don't have much time for tinkering anyway, so I…

How's the battery holding up during vibe coding sessions or occasional LLM usage? I've been thinking about getting a MacBook or a laptop with a similar Ryzen chip specifically for that reason.

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#55

Post by btbuildem »

pstuart wrote: post #32
btbuildem wrote: post #23

Earlier quoted context omitted.

I've recently put together a setup that seemed reasonable for my limited budget. Mind you, most of the components were second-hand, open box deals, or deep discount of the moment. This comfortably fits FP8 quantized 30B models that seem to be "top of the line for hobbyists" grade across the board. - Ryzen 9 9950X - MSI MPG X670E Carbon - 96GB RAM - 2x RTX 3090 (24GB VRAM each) - 1600W PSU

That's basically what I imagined would be my rig if I were to pull the trigger. Do you have an NVLink adapter as well?

No NVLink; it took me a long time to compose the exact hardware specs, because I wanted to optimize performance. Both cards are on x8 PCIe direct CPU channels, close to their max throughput anyway. It runs hot with the CPU engaged, but it runs fast.

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#56

Post by btbuildem »

nine_k wrote: post #51
btbuildem wrote: post #23

Earlier quoted context omitted.

I've recently put together a setup that seemed reasonable for my limited budget. Mind you, most of the components were second-hand, open box deals, or deep discount of the moment. This comfortably fits FP8 quantized 30B models that seem to be "top of the line for hobbyists" grade across the board. - Ryzen 9 9950X - MSI MPG X670E Carbon - 96GB RAM - 2x RTX 3090 (24GB VRAM each) - 1600W PSU

Does it offer more performance than a Macbook Pro that could be had for a comparable sum? Your build can be had for under $3k; a used MBP M3 with 64 GB RAM can be had for approximately $3.5k.

I'm not sure, I did not run any benchmarks. As a ballpark figure -- with both cards throttled down to 250W, running a Qwen-30B FP8 model (variant depending on task), I get upwards of 60 tok/sec. It feels on par with the premium models, tbh.

Of course this is in a single-user environment, with vLLM keeping the model warm.

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#58

Post by whiplash451 »

Traubenfuchs wrote: post #19

It still feels to me like OpenAI has zero moat. There are like 5 paid competitors + open source models. I switch between gemini and ChatGpt whenever I feel one fails to fully grasp what I want, I do coding in claude. How are they supposed to become the 1 trillion dollar company they want to be, with strong competition and open source disruptions every few months?

Isn’t the moat in the product/UI/UX? I use Claude daily and love the “scratch notebook” feel of it. The barebone model does not get you any of this.

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#59

Post by idiotsecant »

onlyrealcuzzo wrote: post #47
zurfer wrote: post #8

It makes me wonder if we'll see an explosion of purpose trained LLMs because we hit diminishing returns on invest with pre training or if it takes a couple of months to fold these advantages back into the frontier models. Given the size of frontier models I would assume that they can incorporate many specializations and the most lasting thing here is the training environment. But there is probably already some tradeo…

Isn't the whole point of the MOE architecture exactly this? That you can individually train and improve smaller segments as necessary

I think it's the exact opposite - you don't specifically train each 'expert' to be a SME at something. Each of the experts is a generalist but becomes better at portions of tasks in a distributed way. There is no 'best baker', but things evolve toward 'best applier of flour', 'best kneader', etc. I think explicitly domain-trained experts are pretty uncommon in modern schemes.

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#60

Post by idiotsecant »

ninetyninenine wrote: post #50

Is China dominating the US in terms of AI? Given that they currently have a model that beats the best models at all formal quantitative benchmarks? What is the state of AI in China? My personal feeling is that it doesn't dominate the zeitgeist in China as it does in the US and despite this because of the massive amount of intellectual capital they have just a small portion of their software engineering talent working…

I think the lesson of the Chinese catchup in AI is that there is a massive disadvantage in being first, in this domain. You can do all the hard work and your competitors can distill that work out of your model for pennies on the dollar. Why should anyone want to do the work?
Post reply on HN