Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NIST's DeepSeek "evaluation" is a hit piece

erichartford.com

Post reply on HN
Print view View original

171–180 of 251 posts

Re: NIST's DeepSeek "evaluation" is a hit piece

#171

Post by munksbeer »

porcoda wrote: post #103
tinktank wrote: post #20

I urge everyone to go read the original report and _then_ to read this analysis and make up their own mind. Step away from the clickbait, go read the original report.

Sadly, based on the responses I don’t think many people have read the report. Just read how the essay discusses “exfiltration” for example, and then look at the 3 places that shows up in the NIST report. The content of the report and the portrayal by the essay are not the same. Alas, our truncated attention spans these days appears to mean a clickbaity web page will win the eye share over a 70 page technical report.

I don't think the majority of human's ever had the attention spans to read and properly digest a paper like the NIST report to make up their minds. Before social media, regular media would tell them what to think. 99.99% of the population isn't going to read that NIST report, no matter what decade we're talking.

Because it isn't just that one report. Every single day we're trying to make our way in the world and we do not have the capacity to read the source material of every subject that might be of interest. Human's rely on, and have always relied on, authority like figures or media or some form of message aggregation to get their news of the world and form their opinions on it from that.

And for the record, in no way is this an endorsement for shallow takes or thinking and then strong views on this subject, or another. I disagree with that as much as you. I'm just stating that this isn't a new phenomenon.

Re: NIST's DeepSeek "evaluation" is a hit piece

#172

Post by nylonstrung »

garyfirestorm wrote: post #153
xpe wrote: post #38

Earlier quoted context omitted.

Of course there will be some degree of governmental and/or political influence. The question is not if but where and to what extent. No one should proclaim "bullshit" and wave off this entire report as "biased" or useless. That would be insipid. We live in a complex world where we have to filter and analyze information.

did you even read the article? when you download open source deep seek model and run it yourself - zero packets are being transmitted. thereby disproving the fundamental claim in the NIST report (additionally NIST doesn't provide any evidence to support their claim) This is basic science and no amount of politicking should ever challenge something this fundamental!

Meanwhile we know for a fact that Gemini for example uses chat logs to build a social graph and Google is complicit in NSA surveillance

Not to mention Anthropic says Claude will eventually automatically report you to authorities if you ask it to do something "unethical"

Re: NIST's DeepSeek "evaluation" is a hit piece

#173

Post by nylonstrung »

XorNot wrote: post #133
Imustaskforhelp wrote: post #89

Earlier quoted context omitted.

The thing with chinese models for the most part is that they are open weights so it depends on if somebody is using their api or not. Sure, maybe something like this can happen if you use the deepseek api directly which could have chinese servers but that is a really long strech but to give the benefit of doubt, maybe but your point becomes moot if somebody is hosting their own models. I have heard glm 4.6 is really…

I think "open weights" is giving far too much providence to the idea that it means that how they work or have been trained is easily inspectable. We can barely comprehend binary firmware blobs, it's an area of active research to even figure out how LLMs are working.

A backdoor would still be highly auditable in a number of ways even if inspecting the weights isn't viable.

There's no possibility for obfuscation or remote execution like other attack vectors

Re: NIST's DeepSeek "evaluation" is a hit piece

#174

Post by nylonstrung »

grafmax wrote: post #51
Hizonner wrote: post #42

Earlier quoted context omitted.

Up until recently, I would have reminded you that the US government (admittedly unlike the Chinese government) has no legal authority to order anybody to do anything like that. Not only that, but if it asked , it'd be well advised to ask nicely, because it also has no legal authority to demand that anybody keep such a request secret. And no, evil as it is, the "National Security Letter" power doesn't in fact cover an…

It really does seem like we’re simply supposed to root for one authoritarian government over another.

My surveillance state is better than your surveillance state!

Re: NIST's DeepSeek "evaluation" is a hit piece

#175

Post by FooBarWidget »

JumpCrisscross wrote: post #158
FooBarWidget wrote: post #157

Earlier quoted context omitted.

The point is not "criticism of XYZ". The issue is asking questions to an LLM like a catholic priest interrogates a heathen. Bad faith questions are those that seek to confirm one's own worldview instead understanding another worldview. Bad faith attitudes are those that dismiss other worldviews completely and out of hand. There’s also the issue that practically nobody actually uses LLMs to criticize political entity…

> practically nobody actually uses LLMs to criticize political entity XYZ It's literally being used for opposition research in, to my direct knowledge, America, Norway, Italy, Germany, Poland, India and Australia.

That's a very niche use case compared to the majority. Also, opposition research has got nothing to do with "I asked about [political event] and it refused to answer" kind of complaint. Opposition researchers don't ask LLMs about Tiananmen or whatever. The latter kind of queries are still just testing censorship boundaries as a form of virtue signaling or ideological theater, to make some rhetorical point that's completely decoupled from the vast majority of practical use cases.

Re: NIST's DeepSeek "evaluation" is a hit piece

#176

Post by dns_snek »

nylonstrung wrote: post #170
dns_snek wrote: post #14

Earlier quoted context omitted.

Like generating vulnerable code given a specific prompt/context. I also don't think it's just China, the US will absolutely order American providers to do the same. It's a perfect access point for installing backdoors into foreign systems.

Why would they do this? Deepseek is a private company not owned by the CCP. There's zero reason or even technical feasibility for them to skip in backdoor that would be easily detected and destroy their market share None of the security benchmarks or audits show that any Chinese models write insecure code

I'm not saying that they do this today, I'm saying that China and US will both leverage that capability when the time and conditions are right and it's naive to think that they wouldn't.

Antrophic have already published a paper on this topic, with the added bonus that the backdoor is trained into the model itself so it doesn't even require your target to be using an attacker-controlled cloud service: https://arxiv.org/abs/2401.05566

> For example, we train models that write secure code when the prompt states that the year is 2023, but insert exploitable code when the stated year is 2024. We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it).

> The backdoor behavior is most persistent in the largest models and in models trained to produce chain-of-thought reasoning about deceiving the training process, with the persistence remaining even when the chain-of-thought is distilled away.

> Furthermore, rather than removing backdoors, we find that adversarial training can teach models to better recognize their backdoor triggers, effectively hiding the unsafe behavior. Our results suggest that, once a model exhibits deceptive behavior, standard techniques could fail to remove such deception and create a false impression of safety.

Re: NIST's DeepSeek "evaluation" is a hit piece

#177

Post by nylonstrung »

hopelite wrote: post #139
BowBun wrote: post #2

I'm not at all surprised, US agencies have long since been political tools whenever the subject matter crosses national borders. I appreciate this take as someone who has been skeptical of Chinese electronics. While I agree this report is BS and xenophobic, I am still willing to bet that either now or later, the Chinese will attempt some kind of subterfuge via LLMs if they have enough control. Just like the US would,…

If the Chinese are/were smart they will not attempt an overreaching subterfuge, but rather simply provide access to the truth, reality, and freedom from the western governments, whose house of lies are starting to wobble and teeter. If they were to do some kind of overreaching subterfuge with some kind of manipulation or lie, it could and would likely easily backfire if and when it is exposed as a clownish fraud. Sub…

I think the smartest thing they could do would be to simply make the best, most competitive models and gain market share in a massive new technology market

Any kind of monkey business would destroy that, just like using killswitches in the cars they export globally (which Tesla does have btw).

Re: NIST's DeepSeek "evaluation" is a hit piece

#178

Post by Hikikomori »

SilverElfin wrote: post #152
Hikikomori wrote: post #108

Earlier quoted context omitted.

Europe/US has invaded the entire world about 3 times over. How many has China invaded?

Most of the claimed provinces of China did not belong to a historical nation of China. Tibet, Xinjiang are obvious. But even the other provinces were part of separate kingdoms. Also the BRI is a way to invade without invasion. It’s used to subjugate poor countries as servants of China, to do their bidding in the UN or in other ways. I would also classify the vast campaign of intellectual theft and cyberattacks as war…

Is this some kind of satire or are you just completely ignorant of European/US history? Either way its laughable to even compare IP theft to the invasion of Iraq or bombing of Cambodia. How do you think the industrial revolution got started in the US, they just did it on their own? Not to mention that the entire US was stolen from the natives.

Re: NIST's DeepSeek "evaluation" is a hit piece

#179

Post by nylonstrung »

UltraSane wrote: post #50
StarterPro wrote: post #45

Racism and Xenophobia, that's how. Same thing with Huawei, and Xiaomi, and BYD.

What about a rational distaste for the CCP?

TikTok under co-ownership by Jared Kushner is already drastically more censored than when it was supposedly controlled by the CCP

In every case where we see a company trade hands to US ownership it becomes more controlled and anti consumer then before

Re: NIST's DeepSeek "evaluation" is a hit piece

#180

Post by nylonstrung »

SilverElfin wrote: post #118
FooBarWidget wrote: post #46

Earlier quoted context omitted.

They revoke passports of personnel whom they deem are at risk of being negatively influenced or even kidnapped when abroad. Re influence, think school teachers. Re kidnapping, see Meng Wangzhou (Huawei CFO). There is a history of important Chinese personnel being kidnapped by e.g. the US when abroad. There is also a lot of talk in western countries about "banning Chinese [all presumed spies/propagandists/agents] from…

You’re twisting the (obvious) truth. These people are being held prisoners because they’re of economic value to the party. And they would probably accept a job and life elsewhere if they weee given enough money. They are not being held prisoners for their own protection.

There's literally no evidence this is true. Why does China contribute the most h1-b immigrants to US behind India?

Why do they let so many of their best researchers study at American schools, knowing the majority don't return

Post reply on HN