Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Who's afraid of Chinese models?

stratechery.com

Post reply on HN
Print view View original

681–690 of 965 posts

Re: Who's afraid of Chinese models?

#681

Post by aroman »

AlexandrB wrote: post #634
tristanj wrote: post #611

Earlier quoted context omitted.

1) Model distillation is the process of transferring knowledge from a large model to a smaller one. It doesn't require logits. https://en.wikipedia.org/wiki/Knowledge_distillation 2) The word "attack" is standard security vocabulary. Per RFC 4949: attack 1. (I) An intentional act by which an entity attempts to evade security services and violate the security policy of a system. That is, an actual assault on system se…

> 3) The "attack" part of "distillation attack" refers to distillers creating tens of thousands of fraudulent accounts, using proxies to bypass georestrictions, deepfaked IDs, and paying real people to pass biometric KYC checks. Who then blended this in with real user traffic to conceal their behavior. Lol. Isn't this literally many of the same tactics OpenAI and Anthropic used to scrape the internet? So now it's an…

Sun Tzu said: if you scrape your enemy, call it training; if your enemy scrapes you, call it an attack.

Re: Who's afraid of Chinese models?

#682

Post by hnfong »

margalabargala wrote: post #281
sanex wrote: post #252

Earlier quoted context omitted.

Different county different feds both spying I'm sure.

Sure but if you are not Chinese or in China, then the chances of negative consequences to you from the Chinese feds is vanishingly small due to lack of ability to do anything that affects you. Meanwhile if you are in the US, DHS has already subpoenaed social media sites looking for people who made anti-ICE posts and I can't imagine they consider subpoenaing AI conversations off limits https://www.nytimes.com/2026/02/…

Also the US government has defacto extra-territorial jurisdiction due to the willingness of allied countries (i.e. most of the developed world) to extradite.

Of course only applies when they really want to get you, but that's still a risk.

Re: Who's afraid of Chinese models?

#683

Post by WarmWash »

bamboozled wrote: post #546
w4lker wrote: post #451

Earlier quoted context omitted.

https://thereallo.dev/blog/claude-code-prompt-steganography Why should I trust a US company more than a Chinese one?

Agree, China believes in Climate change and are at least taking steps to address it. Personally this is making me trust them more than the USA because, facts. I mean lesser of two evils thinking, if one is intentionally leading us towards climate disaster, while the other isn't then yeah. What else can be said? Should I trust the authoritarian country who believes in engineering and science, or the one that doesn't?

>Agree, China believes in Climate change and are at least taking steps to address it.

China brought 80GW of new coal power online last year. The US added 0, and plans to add 3GW next year (we all know why).

China doesn't care about climate change, they care about energy independence, and conveniently have very little natural fossil fuels besides coal. Which they heavily mine and utilize.

Re: Who's afraid of Chinese models?

#684

Post by utilize1808 »

codedokode wrote: post #609
utilize1808 wrote: post #489

Earlier quoted context omitted.

They can just favour some specific versions of some library that's been compromised. Unlike introducing bugs / flaws directly in the source code, they can claim plausible deniability, and it's much easier to implement without compromising the general coding capabilities of the models.

Cannot non-Chinese closed weights model do the same?

It will be used as ammunitions against non-US sanctioned models in general.

Re: Who's afraid of Chinese models?

#685

Post by causal »

tristanj wrote: post #611
causal wrote: post #583

"distillation attack" is such a loaded term that really pisses me off. Distillation is a technical term with real meaning, and historically requires logits which Anthropic does not provide. "Generated training data" is the correct term. It's not an "attack". And Anthropic undoubtedly also generates training data for each new generation of models, yet you never see them claim Fable is a distilled Opus.

1) Model distillation is the process of transferring knowledge from a large model to a smaller one. It doesn't require logits. https://en.wikipedia.org/wiki/Knowledge_distillation 2) The word "attack" is standard security vocabulary. Per RFC 4949: attack 1. (I) An intentional act by which an entity attempts to evade security services and violate the security policy of a system. That is, an actual assault on system se…

1) Your own Wikipedia link goes on to describe using logits. Yes, language evolves to mean multiple things, and that is my point: Anthropic is pushing for a watered down definition. Furthermore, Anthropic hides thinking, so you do not even really get model outputs, you get some downstream partials. Further-furthermore, Anthropic does not describe their own models as distilled when they produce training data. Why? Because generating training data != distillation.

2) This is a stretch: it allows Anthropic to arbitrarily define "attack" via TOS, and ignores the fact that the generated training data is literally paid for by the "attackers".

Re: Who's afraid of Chinese models?

#686

Post by hnfong »

podgorniy wrote: post #457
ballon_monkey wrote: post #233

The 2 things people need to remember: 1) China can (and does) use the models to influence the west. They train in false information about Taiwan and Hong Kong. Or pretend like history is in favor of China. 2) Ignoring the models containing false information, they are incredible. But you should be scared of running inference via the model creators directly. If you think your data is safe compared to running it via mod…

Somehow chineese make less troubles and more good to the world than americans at this point... Something something about ai benefiting all humanity, something something ai being open and stuff, like OpenAI. Chineese simply delivering what americans promised. Tell me: why is EU safe from Trump forcing AI companies to cut access to EU?

> Tell me: why is EU safe from Trump forcing AI companies to cut access to EU?

Well, I think they might end up doing it to themselves by imposing regulations that US companies are unwilling to put up with.

Re: Who's afraid of Chinese models?

#687

Post by colinsane »

OleksandrC wrote: post #6

The article makes a point about agent harnesses being sticky (the supposed moat). I have been building my own agent harness for a while, and I can tell with confidence that the harness almost does not matter, the entirety of the AI magic is the model itself. The harness can be almost barebones (like, for example, mini-swe-agent used for benchmarks), and yet the model still does the task just fine. So from my perspect…

we've barely scratched the surface when it comes to the available design space of agent harnesses.

i also expect you'll see markedly different results if you constrain yourself to small models. there even trivial harness improvements like Codex's /goal feature, and more capable basic tooling (e.g. semantic code grep, js-capable `fetch` tooling) make or break the actual task success rate.

Re: Who's afraid of Chinese models?

#688

Post by butlike »

abalashov wrote: post #557
inigyou wrote: post #517

Earlier quoted context omitted.

A Russian and an American are sitting next to each other on a plane. The American says "I'm impressed by the propaganda you have in Russia." "Oh it's very good, but it's nothing compared to the propaganda you have in America." replies the Russian. "Huh? We don't have propaganda in America." says the American. "Exactly." says the Russian.

As a Russian, I deeply appreciate this. In Russia, it seemed obvious to even the most parochial peasant that there was propaganda, while in America, the vast majority of the society not only fails to consider the possibility, but is cholerically allergic to the very idea. Edit: recognising that there is propaganda != knowing what is and isn't propaganda. That's all I meant.

Propaganda is just advertisements for the government, right?

Re: Who's afraid of Chinese models?

#689

Post by causal »

sigbottle wrote: post #640
causal wrote: post #583

"distillation attack" is such a loaded term that really pisses me off. Distillation is a technical term with real meaning, and historically requires logits which Anthropic does not provide. "Generated training data" is the correct term. It's not an "attack". And Anthropic undoubtedly also generates training data for each new generation of models, yet you never see them claim Fable is a distilled Opus.

Completely unrelated, but I'm seeing people and especially LLMs using causal/intervention so much it's kind of driving me insane. It's actually a very goated term but not everything is causal, it also has precise technical meanings (although those get blurred too given that causal can mean anything from intervention proper, to mere depdnence on something prior)

Are you talking about my username? Yeah I liked the word before LLMs made it cool/uncool.

Re: Who's afraid of Chinese models?

#690

Post by HarHarVeryFunny »

tristanj wrote: post #611
causal wrote: post #583

"distillation attack" is such a loaded term that really pisses me off. Distillation is a technical term with real meaning, and historically requires logits which Anthropic does not provide. "Generated training data" is the correct term. It's not an "attack". And Anthropic undoubtedly also generates training data for each new generation of models, yet you never see them claim Fable is a distilled Opus.

1) Model distillation is the process of transferring knowledge from a large model to a smaller one. It doesn't require logits. https://en.wikipedia.org/wiki/Knowledge_distillation 2) The word "attack" is standard security vocabulary. Per RFC 4949: attack 1. (I) An intentional act by which an entity attempts to evade security services and violate the security policy of a system. That is, an actual assault on system se…

> Model distillation is the process of transferring knowledge from a large model to a smaller one

Sure, and large-to-small is a key part of the definition, and why it's called distillation (cf concentrating something). When Anthopic use synthetic data generated by Opus to train Sonnet or Haiku, then this can correctly be considered a type of distillation.

When Anthropic accuse Chinese companies of "distillation", it seems they are using this word to refer to two potential uses of their model outputs:

1) Using Anthropic model outputs (aka synthetic data) as training data, especially for reasoning, for Chinese models. This really isn't distillation though, since (unlike when they distill their own models) Anthropic don't actually provide the reasoning in their model output, only a "summary" designed to hide the actual reasoning. You can't distill what you are not given!

2) Another way Chinese companies may be using US LLMs is for "LLM as judge" where you are just asking the model to use it's expertise to judge/rate something that you provided yourself (to provide RL training rewards), although for coding you really want hard rewards which are easy to obtain, not fuzzy "looks good to me" ones.

Of course Anthropic are trying to pull the drawbridge up after themselves and their TOS says you can't use their models to develop anything that competes with them, and this seems to be what they are generically referring to as "distillation" - any use of their models that they suspect is being used by the Chinese to improve their own models, not just what what might more technically be called distillation, unless you want to define that word so broadly that it does mean this!

Post reply on HN