Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deepseek: The quiet giant leading China’s AI race

chinatalk.media

Post reply on HN
Print view View original

301–310 of 476 posts

Re: Deepseek: The quiet giant leading China’s AI race

#301

Post by seanmcdirmid »

sangnoir wrote: post #280
ricardobeat wrote: post #274

Earlier quoted context omitted.

Models like Llama 3 are trained on sixteen thousand GPUs, OpenAI probably 25k-100k GPUs. This is the kind of scale the sanctions make a lot harder to achieve.

> This is the kind of scale the sanctions make a lot harder to achieve Harder and more expensive, but far from impossible. I doubt they pay more than doubles Nvidia's sticker price, all told. My comment was inspired by recent real-life events; Nvidia got into legal trouble in the last couple of months for turning a blind eye to questionable transactions - if you're curious about the mechanics of GPU sanctions-busting…

That really sounds plausible. They could open a training farm in Vietnam or Mongolia (or even Taiwan, or really anywhere the internet goes) and just use it, the GPUs don’t need to be located in the mainland. The only way to lock down the GPUs would be to just restrict completely to who could use them, and then prevent them from contracting with sanctioned entities.

When I was working in Beijing, we definitely had resources we couldn’t access locally but could easily access remotely so it didn’t really matter.

Re: Deepseek: The quiet giant leading China’s AI race

#303

Post by seanmcdirmid »

Dalewyn wrote: post #300
selimthegrim wrote: post #275

Earlier quoted context omitted.

Landslide? Electorally?

Trump won both the electoral and popular votes and won all the swing states. The Republicans took both Houses of Congress and held all governorships up for election. Yes, he won by a landslide and the biggest factor was the bad economy.

The factor was calling the economy a bad economy, they are already backtracking on promises since the economy is doing pretty well already and they won’t be able to juice it much more.

Anyone who thinks 49.9% of the vote (vs 48.4%) is a landslide will quickly find them in negative approval ratings if they think that gives them much political capital (if trump focuses and spending rather than earning capital, that is, which he definitely will).

Re: Deepseek: The quiet giant leading China’s AI race

#304

Post by roenxi »

bbreier wrote: post #296
roenxi wrote: post #252

Earlier quoted context omitted.

That is obviously wrong to the point where I am confused why someone always makes the claim. I'm looking forward to running in to someone who can actually follow up with some sort of defence of the position. Extraordinary claims, extraordinary evidence style. Consider the computer industry. Prices have been falling pretty much across my entire life. Supply-demand suggests that people will keep buying new computers as…

Deflation is a problem, but not for the reason mentioned there. The reason deflation is an issue is because it makes holding cash into an investment strategy. If the price of goods is dropping, the value of money is rising, so the more cash I hold the richer I get. This obviously dissuades people with cash from investing their cash into actual productive work, which means fewer jobs. There was a small deflationary bu…

Like interest on a bank account? Holding cash is already an investment strategy. Bonds are a thing. People have options to hold cash and not lose purchasing power. One of the traditional ways people tackle inflation is they demand interest from the banks sufficient to cover it plus a little more to account for time value of money.

It is rather unlikely that giving people an option that they already have is going to cause a problem. One major benefit of money is that people can hoard it and there is no cost in the real economy because all the resources are still there and prices can just adjust to the amount of cash in circulation.

> here was a small deflationary bump in American history around the 1930s that helps to illustrate what can happen in a deflationary spiral.

The US came out of the 1930s with an economy that was capable of overcoming almost literally the entire world. Again, the evidence that deflation was some sort of major problem is questionable, it seems to have been associated with the creation of one of the most dynamic economies in the history of history.

And the idea that we have this one clear lesson from one instance back in the 30s is just weird and unbelievable. That isn't how history or complex systems work.

Re: Deepseek: The quiet giant leading China’s AI race

#305

Post by roenxi »

kayewiggin wrote: post #220
roenxi wrote: post #211

Earlier quoted context omitted.

> when real estate prices have dropped by more than 50% in tier 2 and below cities, and deflation has raged on. Can other economies copy that part? I know a bunch of people who'd like to be able to afford more houses & more groceries at the same time. I'd like that, I can't realistically afford a house in the city I live in without a 50% price drop. I'm sure China has a lot of problems, but key goods getting cheaper…

China's real estate prices having dropped 50% or more has been accompanied by/caused by wages being slashed 50% or more, and increasing unemployment rate, such as 30%+ for youth. Here are some good posts on why nobody wants deflation: https://www.reddit.com/r/AskEconomics/comments/uzq5bu/why_is... https://www.reddit.com/r/AskEconomics/comments/mbsxyl/can_so... https://www.reddit.com/r/AskEconomics/comments/yotf0c/i_d…

So some might suggest that the problem is wages being slashed by >50%? Falling real wages are actually a problem. And, AFAIK by definition, are not influenced by inflation or deflation. But if wages had fallen by 50% then the overall situation was probably improving. A bit chaotic to be comfortable, but not fundamentally worse.

And there is an unemployment problem too, obviously.

Re: Deepseek: The quiet giant leading China’s AI race

#306

Post by varenc »

tokioyoyo wrote: post #141
qwertox wrote: post #96

Earlier quoted context omitted.

If these three died, it might be a loss to the country. None of them is as important to the country as Xi is to China. The resilience of the CCP, in light of its dependence on Xi, can only be upheld through the absolute suppression of freedom. But the average daily life is certainly much more enjoyable than in NK.

I’m curious, what do you think would happen if Xi stepped down tomorrow for whatever reason? You think everything will just fall apart?

If Xi Jinping died suddenly, it’d cause serious instability. His centralization of power erased clear succession plans, so top CCP factions would likely fight for control. Markets would also freak out. Long-term? Either a return to pre-Xi more collective leadership or another strongman doubling down on Xi’s approach. Centralized systems with no backup plans are fragile.

It’s possible that with technology like absolute communication control and ubiquitous surveillance the chance of internal unrest or revolution is greatly reduced. And as long as the country is growing and the average citizen is getting richer they’re much less likely to get unruly. It’s like startups: growth solves all problems.

Re: Deepseek: The quiet giant leading China’s AI race

#307

Post by llm_trw »

cma wrote: post #240
llm_trw wrote: post #190

Earlier quoted context omitted.

We're seeing a split in models between deep and wide. Wide models sound like they know more than deep models but fail at reasoning with more than a few steps and are cheap to train and serve. Deep models know a lot less but can reason much better. An example I saw all moe models fail at a few months back was A and not B being implicit in the grounding text, all of them would turn it into A and B a substantial proport…

They may have a better approach for MoE selection during training: > The key distinction between auxiliary-loss-free balancing and sequence-wise auxiliary loss lies in their balancing scope: batch-wise versus sequence-wise. Compared with the sequence-wise auxiliary loss, batch-wise balancing imposes a more flexible constraint, as it does not enforce in-domain balance on each sequence. This flexibility allows experts…

That's just making the architecture work better.

I'm old enough to remember when everyone outside of a few weirdos thought that a single hidden layer was enough because you could show that type of neural network was a universal approximator.

The same thing is happening with the wide MoE models. They are easier to train and sound a lot smarter than the deep models, but fall on their faces when they need to figure out deep chains of reasoning.

Re: Deepseek: The quiet giant leading China’s AI race

#308

Post by varenc »

freefaler wrote: post #74
qwertox wrote: post #36

Earlier quoted context omitted.

It remains to be seen how stable a totalitarian government can be. China has the benefit of having full control over its people and therefore gets to decide what is important and what not, and currently people are ok with handing that control over to the government. But it's also a very fragile state, which can only be retained through full repression.

This argument might've worked in Mao's time. Now with a capitalist economy under the party the resource allocation while still skewed is much more efficient than during Mao or USSR central planned economy. (And EU wide policies sometimes aren't that far off from USSR stupidity). Loss of feedback in authoritarian regimes is a problem, but in the short time it might not be if Xi doesn't make really stupid moves. It pai…

> It pains me to see it, but they show more long-term thinking that many of the Western governments…

Absolutely agree, and it pains me as well. Besides long-term thinking, they can also just impose sweeping new rules to address certain problems in a way the West never could.

For example, with teen gaming addiction, they didn’t hesitate to just ban kids under 18 from gaming more than a few hours a week, crashing the value of certain gaming companies. In the West, we’d spend years debating, lobbying, and litigating over individual freedoms vs. public good, and likely end up with nothing meaningful. It comes with huge drawbacks, but their system allows them to take drastic action quickly, while we’re often paralyzed by process.

Re: Deepseek: The quiet giant leading China’s AI race

#309

Post by Dalewyn »

jasdi wrote: post #298
Dalewyn wrote: post #262

Earlier quoted context omitted.

Whoever are downvoting this really don't understand that expanding a congested highway merely widens the congestion instead of alleviating it. Software today require several dozen gigabytes of RAM and two dozen CPU cores just to render some simple fucking text because everyone has several dozen gigabytes of RAM and two dozen CPU cores.

This is one of the biggest reasons why China or India are threat to the West. Because their Engineers are constantly in situations were they have to work with much less resources. And they come up with stuff that constantly surprises people for being much cheaper. The western reaction is to then go and buy them out. Its not going to work forever. They have already taken over large swathes of the tech landscape. Even…

For how much we all bitch about beancounters, I'm kind of surprised that beancounters haven't managed to convince management to buy bottom-of-the-barrel Celeron laptops for software developers.

Make those fucking assholes use the hardware of the people instead of some monstrosity with a 256 cores CPU and 48 TB of RAM and 24 exabytes of enterprise SSD and an RTX Cinco Grande connected via optical fiber to the Amazonflare Cloudnet. We will see lean and mean software literally overnight.

Post reply on HN