Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Who's afraid of Chinese models?

stratechery.com

Post reply on HN
Print view View original

441–450 of 965 posts

Re: Who's afraid of Chinese models?

#441

Post by metalspot »

fspeech wrote: post #387
ryeguy wrote: post #271

Earlier quoted context omitted.

I keep seeing mention of the cache, what's special about it? All frontier llms have prefix caching, what is special about deepseek's approach?

Their kv cache is smaller so they can use less vram and also keep your prefix cached for longer. https://deepseek.ai/blog/deepseek-v4-compressed-attention

This also comes with significant capability reduction. deepseek-v4-flash is very good in the [edit]

This is my observation from using it without an specific context engineering to optimize for Deepseek's cache compression and sparse attention mechanisms. I am pretty sure that if you specifically structure your context to align to the cache compression boundaries you can significantly improve performance in the full 1M context, but there is not much reason to do this, because if you design your outer loop to work with shorter contexts that solution is portable and more efficient, so I haven't bothered with a optimizing for DS at this point.

Re: Who's afraid of Chinese models?

#442

Post by ragazzina »

itake wrote: post #273
Lord-Jobo wrote: post #260

Earlier quoted context omitted.

Is this something that is more true of a Chinese model than any other model of a different national origin? Genuine question: generalized up from individual models to “models from country X”, is there any country that doesn’t have this exact risk?

In the USA, multiple political parties balance each out other. In China, there is 1 party. 1 view. 1 definition of the Truth.

>In the USA, multiple political parties balance each out other.

Is this what Americans really believe?

Re: Who's afraid of Chinese models?

#443

Post by johanyc »

Sol- wrote: post #38
OleksandrC wrote: post #6

The article makes a point about agent harnesses being sticky (the supposed moat). I have been building my own agent harness for a while, and I can tell with confidence that the harness almost does not matter, the entirety of the AI magic is the model itself. The harness can be almost barebones (like, for example, mini-swe-agent used for benchmarks), and yet the model still does the task just fine. So from my perspect…

For me, harnesses are mostly sticky insofar as the model providers only allow you to use their subsidized plans through their own harnesses, unfortunately. But of course switching model + harness is an option.

> model providers only allow you to use their subsidized plans through their own harnesses

true for anthropic, not true for openai.

Re: Who's afraid of Chinese models?

#444

Post by blensor »

amelius wrote: post #438
blensor wrote: post #408

Earlier quoted context omitted.

I'm a model nomad, using whatever solved my last problem the best and where it makes the most sense to start my next work in. However with the latest models Fable, Kimi K3, 5.6, it's getting to a point where I sometimes forget what model I am on without noticing a difference. And once I realize it because something may not be exactly like I expected it I won't switch for that work either because I don't want to inval…

You didn't say what kind of problems you solve with AI. It matters a lot if you are doing HTML versus C++, for example.

In no order of importance:

  - Refactoring a 13 year old in-house vacation rental booking system ( python/turbogears )
  - Backend development for our VR fitness game ( flask/python )
  - Unity development on our VR fitness game ( C#/Unity )
  - VR game development experiments ( Godot/GDScript )
  - Standalone SLAM localization service ( C++ )
  - Audio analysis ( python/pytorch )
  - Virtual display with Viture display glasses ( C )
  - Reverse engineering a library I am using for another project ( ghidra -> C - no MCP yet, that's something I am looking forward to )
  - Public facing website rebuilding for the booking system above ( PHP/JS )
  - Generative 3D environments for our VR fitness game ( python )
  - Wireless camera/IMU based tracker for the SLAM system ( C )
Once I've dug in with a specific model into a problem I tend to stick to that because I have a feeling what it will do and how well it works, but when I start a new thing I usually use whatever the model was last set to.

Re: Who's afraid of Chinese models?

#445

Post by 2sk21 »

nl wrote: post #95
chrsw wrote: post #55

Earlier quoted context omitted.

Correct. We need open weights, open code and open data. If nobody else can reproduce what someone did there will always be security questions. Even if we can reproduce it there could still be security concerns but it's more realistic to investigate yourself.

I'm all for open models, but people seem to misunderstand what they are. They aren't the same thing as open source code! > open weights, open code and open data Even if you have all these things you still can't replicate a model because of randomness. You can backdoor a model with less than 1000 examples and it is impossible to detect.

Yeah - we also don't know if the models from OpenAI and anthropic are back-doored either.

Re: Who's afraid of Chinese models?

#446

Post by tellrealos »

itake wrote: post #360
TripolitianFish wrote: post #326

Earlier quoted context omitted.

I’d love to live in the USA you’re talking about friend. This is just oriental despotism paranoia, whatever cutsie repetition slogan you come up with is not a serious argument.

I've traveled to China ~5x [0], visited a range of cities Tier 1-3 over a collective 5 months, and grew up in the USA. I also currently live in Vietnam (~3.5 years) and spent 5.5 years working for a Singaporean company and a team stationed in Beijing. I don't really know how else to express my experiences living in, working with, and interacting with people in both of these countries. Perhaps you can share how life w…

I am not really interested in reality like this.

I just repeat things I read on my social media feed.

America bad. China good.

Re: Who's afraid of Chinese models?

#447

Post by londons_explore »

Omniusaspirer wrote: post #391
michaelt wrote: post #358

Earlier quoted context omitted.

> Power is only a small part, single digit, it will eventually matter but does not really today. Sorta yes, sorta no. A single 5090 consumes 450W - at Californian energy prices of $0.38 per kWh that's $0.17 per hour. And the card itself costs $4100 on amazon. So after 2.75 years running at full power 24/7 you'll have spent more on electricity than on the card. I would have thought most data centres being built today…

Anyone seriously building out AI infrastructure I presume is paying nowhere near $.38/kWh which is extortionate. Utility scale solar is closer to $.02-.03/kWh, then maybe around ~$.10/kWh for natural gas peaker plants.

Utility scale power price varies widely by location and exact time of day.

LLM's aren't very latency sensitive and can therefore move to wherever power is cheapest.

Right now that's places next to aluminium smelters (which also like very cheap electricity 90+% of the time).

Re: Who's afraid of Chinese models?

#448

Post by cdud3 »

senderista wrote: post #176
appplication wrote: post #135

Earlier quoted context omitted.

I think there are some really interesting thought there, but I’d challenge some of this: > Models are not free. Downloading them is free. Running them is not. This has manufacturing economics, not software economics; the idea that they are "free" is an economic category error as it relates to their actual use I think a large part of manufacturing economics is illiquid overhead and the cost of expertise to set up and…

Amazon is not "meaningfully in inference"? Bedrock seems to have a ton of enterprise customers, some of which would never trust the AI labs themselves with their data but will trust Amazon.

A lot of this companies are also in parallel evaluating running Chinese models inhouse to be independent of the good will and profit margins of externals.

This is where Oracles "datacenters for rent to run your own Chinese models" strategy will benefit. The LLM SaaS game is a lost one thanks to China.

Re: Who's afraid of Chinese models?

#449

Post by amelius »

blensor wrote: post #444
amelius wrote: post #438

Earlier quoted context omitted.

You didn't say what kind of problems you solve with AI. It matters a lot if you are doing HTML versus C++, for example.

In no order of importance: - Refactoring a 13 year old in-house vacation rental booking system ( python/turbogears ) - Backend development for our VR fitness game ( flask/python ) - Unity development on our VR fitness game ( C#/Unity ) - VR game development experiments ( Godot/GDScript ) - Standalone SLAM localization service ( C++ ) - Audio analysis ( python/pytorch ) - Virtual display with Viture display glasses (…

Wow, now we're talking :)

Re: Who's afraid of Chinese models?

#450

Post by viktorcode »

kinj28 wrote: post #253

I am afraid — if Chinese models go mainstream it has a clear way of pushing its narrative way beyond its otherwise borders. More like a Trojan horse it is for the Chinese. Here is a quick example of how Chinese deepseeks agent works kn its underlying model) when asked a tough question https://x.com/jinen83/status/2079406993979383902?s=46&t=D7hQ...

This is misleading; the chat is censored, the model is not.

You can download the model, run locally and ask the same questions to see the difference.

Post reply on HN