121–130 of 198 posts
Earlier quoted context omitted.
Seems to be working out better than leaving it up to the market
Why does the US stock market keep having record years while the population doesn't seem to benefit, whereas the opposite is happening in China? Isn't that a failure of US capitalism?
Earlier quoted context omitted.
Once I start a conversation about Postgres query plans, maybe 95% of the knowledge will be almost certainly not needed, and for an inference provider there will be many more concurrent queries with reasonably large overlap. Maybe future architectures will be able to take advantage of this so not all parameters are needed in memory for almost all instances. If we are talking about all knowledge , then I agree that the…
Mixture of Experts is already used by pretty much all modern LLMs to address exactly this phenomenon. Hopefully, future models can be trained to be even more aware of external knowledge, accessible through web search / RAG / whatever it will be then, and might not need to internalize much knowledge at all.
Then you need longer contexts, which is proving to a much more stubborn problem than general knowledge compression.
The more interesting question is: why does the rest of the world not increase their production prowess? I am aware of most of the reasons, but I am getting tired of the "China is better than the rest of the world". It is more that China did a lot of things that the others either failed to do or no longer wanted to do. Case in point is Germany: from "our cars are best" to "hey, chinese cars are now better" (or, cheape…
Chinese structural advantages: monetary policy, lower cost of living, better infrastructure (electrical and transportation are all new within the last 20 years), lower cost of {concrete, factory machinery, metals, electrical} due to economies of scale.
Chinese EV manufacturers advantages: they don't sell ICE cars, so they don't have any legacy customers, factories to retool, or brand migration issues. And they mostly stole their technology from hiring Tesla factory workers to moonlight for them and share the design, technology, tooling, and RE the Tesla finished product.
Major Western legacy car company disadvantages: Ford, Toyota, Volkswagen, etc... all have significant reasons why they can't or won't make the super risky attempt to go all-EV. They all have $10s of billions in debt they are still paying off for their current capital improvements. Those debts assume they contribute selling ICE because EVs last longer so they would sell fewer units per year. It's simply less risk to avoid making any EVs and pay politicians to protect their businesses than to try to compete with 10 year old Chinese phone-companies-turned-EV companies. On top of that, the worldwide demand for EVs is only about the volume of the single most popular model of Toyota. They would have to go into massive debt on a giant risk that consumers will want more EVs (and that the EVs they design/build will sell better than the current ICE cars they have meticulously crafted for 50-100 years).
I have nothing to say about this company's long term potential, but I'm curious to know whether this will become an enormous "tik stok" like SK Hynix became in South Korea. I've watched a short documentary about the investment mania in SK and it's really scary.
Earlier quoted context omitted.
To be fair, all high end chips are made in Asia with EU tooling. US happens to be the place where the managers are and the trade is facilitated. There has been some good progress to push Asia make cips in USA with EU machines however IIRC significant part of the chain is still in Asia and chips made in USA are shipped to Asia for further processing. Edit: changed China with Asia to distance the comment from Taiwan-Ch…
> US happens to be the place where the managers And you know.. the actual design for plenty of these chips. By that definition Apple is also an Asian company.. > with EU tooling That is only a (vital) part of the supply chain, there are plenty of other components.
I have nothing to say about this company's long term potential, but I'm curious to know whether this will become an enormous "tik stok" like SK Hynix became in South Korea. I've watched a short documentary about the investment mania in SK and it's really scary.
If the chinese government feels its critical to china's success then it will be backed like alibaba and wechat and others were
Earlier quoted context omitted.
It's this. Wall Street won't allow american companies to make long term decisions and long term planning that cost money in the short term, because they want all that money immediately now. Like, right now now. If an executive promised they could deliver big profits in 5 years by spending the next two investing their profits, Wall Street would fire them and put in someone who will happily spend all the profit (plus d…
The stock market works that way, it's true. But what about SpaceX, Blue Origin... there are also counter examples that show it is possible for investors to wait out the big profits.
Earlier quoted context omitted.
How would we do that? There's no historic precedent for that. Its fundamentally an information theory thing: what's the max amount of intelligence you can get out of 1 KB/MB/GB? There has to be a limit and I'm not convinced that it's far off.
It’s not just about small models, that’s only one part of evolution Some groups are baking models into silicone, Deepmind has an example, it gets 18,000 tokens/sec on Llama 3.1, not sure about parameter size
While some other groups are baking silicone into models :)