Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

tongyi-agent.github.io

Post reply on HN
Print view View original

71–80 of 156 posts

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#71

Post by terhechte »

anon373839 wrote: post #64
jlokier wrote: post #28

Earlier quoted context omitted.

I use a Macbook Pro with 128GB RAM "unified memory" that's available to both CPU and GPU. It's slower than a rented Nvidia GPU, but usable for all the models I've tried (even gpt-oss-120b), and works well in a coffee shop on battery and with no internet connection. I use Ollama to run the models, so can't run the latest until they are ported to the Ollama library. But I don't have much time for tinkering anyway, so I…

I’d strongly advise ditching Ollama for LM Studio, and using MLX versions of the models. They run quite a bit faster on Apple Silicon. Also, LM Studio is much more polished and feature rich than Ollama.

Fully agree to this. LM Studio is much nicer to use and with MLX faster on Apple Silicon

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#72

Post by trebligdivad »

exe34 wrote: post #16
aliljet wrote: post #5

Sunday morning, and I find myself wondering how the engineering tinkerer is supposed to best self-host these models? I'd love to load this up on the old 2080ti with 128gb of vram and play, even slowly. I'm curious what the current recommendation on that path looks like. Constraints are the fun part here. I know this isn't the 8x Blackwell Lamborghini, that's the point. :)

llama.cpp + quantized: https://huggingface.co/bartowski/Alibaba-NLP_Tongyi-DeepRese... get the biggest one that will fit in your vram.

How do people deal with all the different quantisations? Generally if I see an Unsloth I'm happy to try it locally; random other peoples...how do I know what I'm getting?

(If nothing else Tongyi are currently winning AI with cutest logo)

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#73

Post by kmarc »

ainch wrote: post #63
tbruckner wrote: post #34

Has anyone found these deep research tools useful? In my experience, they generate really bland reports don't go much further than summarization of what a search engine would return.

The reports are definitely bland, but I find them very helpful for discovering sources. For example, if I'm trying to ask an academic question like "has X been done before," sending something to scour the internet and find me examples to dig into is really helpful - especially since LLMs have some base knowledge which can help with finding the right search terms. It's not doing all the thinking, but those kind of bro…

I caught myself that most of my LLM usage is like this:

ask a loaded, "filter question" I more or less know the answer for, and mostly skip the prose and get to the links to its sources.

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#74

Post by blaesus »

tbruckner wrote: post #34

Has anyone found these deep research tools useful? In my experience, they generate really bland reports don't go much further than summarization of what a search engine would return.

"Summarization of what a search engine would return" is good enough for many of my purposes though. Good for breaking into new grounds, finding unknown unknowns, brainstorming etc.

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#75

Post by PeterStuer »

btbuildem wrote: post #23
aliljet wrote: post #5

Sunday morning, and I find myself wondering how the engineering tinkerer is supposed to best self-host these models? I'd love to load this up on the old 2080ti with 128gb of vram and play, even slowly. I'm curious what the current recommendation on that path looks like. Constraints are the fun part here. I know this isn't the 8x Blackwell Lamborghini, that's the point. :)

I've recently put together a setup that seemed reasonable for my limited budget. Mind you, most of the components were second-hand, open box deals, or deep discount of the moment. This comfortably fits FP8 quantized 30B models that seem to be "top of the line for hobbyists" grade across the board. - Ryzen 9 9950X - MSI MPG X670E Carbon - 96GB RAM - 2x RTX 3090 (24GB VRAM each) - 1600W PSU

Unfortunately the RTX 3090 has no native FP8 support.

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#76

Post by 3abiton »

aliljet wrote: post #5

Sunday morning, and I find myself wondering how the engineering tinkerer is supposed to best self-host these models? I'd love to load this up on the old 2080ti with 128gb of vram and play, even slowly. I'm curious what the current recommendation on that path looks like. Constraints are the fun part here. I know this isn't the 8x Blackwell Lamborghini, that's the point. :)

As many pointed out, Macs are decent enough to run them (with maxxed rams). You also have more alternative, like DGX Sparks (if you appreciate the ease of cuda, albeit a tad bit slower token generation performance), or the Strix Halo (good luck with ROCm though, AMD still peddling hype). There is no straitghtforwars "cheap" answer. You either go big (gpu server), or compromise. Either way use either vllm or sglang, or llama.cpp. ollama is just inferior in every way to llama.cpp.

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#77

Post by Nymbo »

sumo43 wrote: post #36

I made a 4B Qwen3 distill of this model (and a synthetic dataset created with it) a while back. Both can be found here: https://huggingface.co/flashresearch

Just tried this out with my web search mcp, extremely impressed with it. Never seen deep research this good from a model so small.

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#78

Post by Grimblewald »

Traubenfuchs wrote: post #19

It still feels to me like OpenAI has zero moat. There are like 5 paid competitors + open source models. I switch between gemini and ChatGpt whenever I feel one fails to fully grasp what I want, I do coding in claude. How are they supposed to become the 1 trillion dollar company they want to be, with strong competition and open source disruptions every few months?

Of course they dont, the only advantage it ever had was the willingness to destroy trust on the internet by scraping everything from everyone rules and expectations be dammed.

The underlying architecture isnt special, the underlying skills and tools aren't special.

There is nothing openAI brings to the table other than a willingness to lie, cheat, and steal. That only gives you an edge for so long.

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#79

Post by red2awn »

Traubenfuchs wrote: post #19

It still feels to me like OpenAI has zero moat. There are like 5 paid competitors + open source models. I switch between gemini and ChatGpt whenever I feel one fails to fully grasp what I want, I do coding in claude. How are they supposed to become the 1 trillion dollar company they want to be, with strong competition and open source disruptions every few months?

The moat of OpenAI is 1. internal knowledge they've built over the last few years building front tier models 2. their talent 3. the ChatGPT brand (go ask a random person on the street, they know ChatGPT but not Claude or Gemini)

Re: Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

#80

Post by criemen »

zurfer wrote: post #8

It makes me wonder if we'll see an explosion of purpose trained LLMs because we hit diminishing returns on invest with pre training or if it takes a couple of months to fold these advantages back into the frontier models. Given the size of frontier models I would assume that they can incorporate many specializations and the most lasting thing here is the training environment. But there is probably already some tradeo…

> or if it takes a couple of months to fold these advantages back into the frontier models.

Right now, I believe we're seeing that the big general-purpose models outperform approximately everything else. Special-purpose models (essentially: fine tunes) of smaller models make sense when you want to solve a specific task at lower cost/lower latency, and you transfer some/most of the abilities in that domain from a bigger model to a smaller one. Usually, people don't do that, because it's a quite costly process, and the frontier models develop so rapidly, that you're perpetually behind them (so in fact, you're not providing the best possible abilities).

If/when frontier model development speed slows down, training smaller models will make more sense.

Post reply on HN