Interesting (mis)use of the word catfish
Not how we normally understand
Deepseek: The quiet giant leading China’s AI race
241–250 of 476 posts
Re: Deepseek: The quiet giant leading China’s AI race
#242Earlier quoted context omitted.
But like I said, deepseek is open source so why can't the competitors copy whatever source that makes the cost of production 10x cheaper ?
You have to distinguish between the current model and DeepSeek the company. DeepSeek the company can do an OpenAI and stop releasing their weights any time they like. The knowledge and skill is retained. I really wonder how long the current era of giving models away for free can last. How is this sensible from a business perspective? Facebook got burned by iOS and now engage in what would otherwise look like irration…
They would have to pay more to get researchers that don't publish.
Re: Deepseek: The quiet giant leading China’s AI race
#243Re: Deepseek: The quiet giant leading China’s AI race
#244Not personally surprised that a MoE model performs so well. I used Mixtral a lot for coding Rust, and it had qualities no other model had except GPT 3.5 and later Claude Sonet. The funny thing is Mixtral was based on Llama 2 which was not trained on code that much. DeepSeek v3: 671B parameters on total, and 37B activated sounds very good even though impossible to run locally. Question if some people happen to know: F…
It activates only 37B per query, but you don’t know which ones ahead of time, so you gotta store all 671B in (V)RAM.
But you don't need cluster networking or nvlink so much like with splitting out llama 405B. You could even split them out with friends over internet levels of bandwidth.
Re: Deepseek: The quiet giant leading China’s AI race
#245Earlier quoted context omitted.
> when real estate prices have dropped by more than 50% in tier 2 and below cities, and deflation has raged on. Can other economies copy that part? I know a bunch of people who'd like to be able to afford more houses & more groceries at the same time. I'd like that, I can't realistically afford a house in the city I live in without a 50% price drop. I'm sure China has a lot of problems, but key goods getting cheaper…
Much of real estate in china is not liveable. They are cheaply built projects without any infrastructure or people living in it.
here's a video of a Chinese family living on the 16th floor of a rotten tail building (unfinished building) with no running water or electricity.
Re: Deepseek: The quiet giant leading China’s AI race
#246I feel the GPU restrictions created an environment for Chinese Devs to be more innovative and do more with less. Kudos to the deepseek team!
Software expands to fill the available resources. If you want more efficient software, build it on less powerful hardware. AI training runs are no exception!
Can't agree more. Especially as I wait to download 1.3gb file to update some Windows driver like Realtek audio or LED display, and you realise a whole effing Debian operating system with Xfce & apps can fit in a lot less.
Re: Deepseek: The quiet giant leading China’s AI race
#247Earlier quoted context omitted.
The LMSYS leaderboards are crowdsourced and would be hard to fake, it showing a pretty strong performance in terms of human preference.
Crowdsourced data is the easiest to fake unless you can somehow ensure that you have a completely unbiased population (which is impossible). There's a reason why certain models do so well on upvote-based leaderboards but rank nowhere on objective tests.
Which ones? I think fine-tunes are where I see most of this (I'll just call it) "model spam", but the base models don't seem to exhibit this behavior. I do see some models perform way below the curve compared to their benchmark performance, though (Phi family being the most famous).
Re: Deepseek: The quiet giant leading China’s AI race
#248Earlier quoted context omitted.
The question in a rational mind is, why would it even bother? US/China partnership is the most economically successful in world history, even more so than US/UK or AUKUS. But the downside of CCP government structure is that paranoia at the top ranks has a good probability of overruling rationality. Albeit US cannot speak as US-centric paranoia/"exceptionalism" may do the same thing...and the electorate voted to self…
> US economy being the strongest in decades The vast majority of its people do not share in that success and have seen a declining standard of life relative to prior generations whereas in China, the opposite is quite demonstrably true, despite increasingly similar concentrations of wealth and political power.
On the contrary, real wages in the USA are the highest they've ever been. Social media and fentanyl are making people unhappy but for most people it's not an economic problem.
Re: Deepseek: The quiet giant leading China’s AI race
#249Earlier quoted context omitted.
When I was a kid, China was a lot better integrated with the international community. Right now their relationships are far and few between, rarely featuring first-world nations. If Russia couldn't beat NATO in a pitched fight against the rest of the world, neither can China.
As a sovereign nation rises in power you'll notice how it slowly starts losing favor from USA
If it's a democracy and reasonably friendly to the USA, it's not a big deal. In the 1990s everyone thought Japan was going to be the next superpower and that was just fine.
Re: Deepseek: The quiet giant leading China’s AI race
#250Earlier quoted context omitted.
It's literally happening lol. When you were a kid China was making shoes and their GDP is 10% of the US. Now they're making drones / evs / high end electornics and it's 80%. This is why people's perception is so unreliable because it's impossible to notice things when they happen over a lifetime
And now they're at the point where the population pyramid is collapsing. It's hard to make any predictions about the future when they got here riding a baby boom and now their ratio of elderly to working age is about to go through the roof. https://www.populationpyramid.net/china/2023/
Japan, Germany, Italy, and numerous other countries are all much worse off. Assuming things don't change, China in 2050 will be in approximately the same position as Japan today.
Germany and Italy, to take but two examples of large Western economies, haven't had native above-replacement TFR since ~1970.
Even in the US, TFR is well below replacement right now, and in fact is basically comparable to China's TFR from 2010-2017.
> https://www.cdc.gov/nchs/data/vsrr/vsrr035.pdf
US population growth is sort of immigration-dependent, which, let's put it this way, isn't an unalloyed good thing.