Earlier quoted context omitted.
There's a way for us to have a say in the matter. It's happened before.
Okay, what way? What happened before? Don't be vague.
The french perfected it.
AI eats the world (Spring 26) [pdf]
161–170 of 189 posts
Re: AI eats the world (Spring 26) [pdf]
#162Earlier quoted context omitted.
Ok so the hype would be people saying AI can currently do something well and autonomously when it cannot (or not consistently enough), and it is easy to prove them wrong. But I feel like people are more hyped about what the AI will be able to do soon rather than what it can do now. I think AI does understand things (depending on your definition), how else could we communicate and ask it a question if it didn't? I mea…
> I think AI does understand things (depending on your definition), how else could we communicate and ask it a question if it didn't? https://en.wikipedia.org/wiki/Chinese_room
That doesn't really respond to the question though - there is a quite reasonable argument that the Chinese room as a system 'understands' things.
The issue that is hit immediately is we don't have a definition or test of understanding that AI doesn't clear easily. Then on top of that we can't even really be sure that we ourselves are understand things given all the tricks that our minds play with memory and perception. There is precious little evidence that the people around us understand things, they seem to be guessing. It is completely unclear if a Chinese room has or doesn't have a property if we rule out all the tests that check for it as not really counting. But all the tests we can do suggest it does understand, because engineers can implement Chinese rooms now and they even turn out to be more reliably artistic/capable of novel thinking/creative than humans. Anything that tests understanding they can do.
Re: AI eats the world (Spring 26) [pdf]
#163This is excellent all around.
Re: AI eats the world (Spring 26) [pdf]
#164Earlier quoted context omitted.
Yes, I wrote about Rock’s Law too, but we don’t know that this is how these models will develop
Evidence point to the same type of scaling law. Compute for a training run grows 4-5x every year.[0] I'm sure this will slow down but the premise remains that weaker competitors will not be able to maintain this pace. We already see labs like Cohere, Mistral, Inflection AI, Adept, Character.ai, and others bow out of the frontier race. I'm also skeptical that Meta, xAI can catch up. Even Google has trouble keeping up.…
How about the externalized intelligence around the model weights (skills, tools, harness, memory etc)? If the model weights are sufficiently intelligent, the focus might move to the external layers.
Re: AI eats the world (Spring 26) [pdf]
#165Earlier quoted context omitted.
> Mixture-of-Experts seems like an attempt to do this - the domain structure being extracted into specific sub-models that are presumably trained on particular domain-associated content This is a common miss-conception. MoE LLMs are NOT trained with each expert receiving domain-associated data. It's just an unfortunate naming decision that stuck, and is commonly miss-understood by non practitioners.
Interesting. So what's the strategy there? Just assume that each expert will learn some underlying clustering of semantic associations, but not direct it?
Not even that. The "experts" are not expert in any particular topic.
MoE is an architecture change meant to lower the total compute for both training and serving an LLM. You basically have many smaller models (unfortunately called experts) and a router on top of them. The router "learns" which expert to activate for the next token generation, but that doesn't need to follow any semantic association. For the same math problem you could get experts 1 and 234 activate on the first token, 5 and 132 on the 2nd token and so on.
Re: AI eats the world (Spring 26) [pdf]
#166Earlier quoted context omitted.
Yes, I wrote about Rock’s Law too, but we don’t know that this is how these models will develop
Evidence point to the same type of scaling law. Compute for a training run grows 4-5x every year.[0] I'm sure this will slow down but the premise remains that weaker competitors will not be able to maintain this pace. We already see labs like Cohere, Mistral, Inflection AI, Adept, Character.ai, and others bow out of the frontier race. I'm also skeptical that Meta, xAI can catch up. Even Google has trouble keeping up.…
I agree with much of what you’ve written but think you are missing the correct alignment of the mobile data timeline — mobile data had standards because it was forced to. It was forced to early because it was not a fundamental innovation, telecom itself was the fundamental innovation, mobile was a constraint relaxation. Intelligence might be forced to have standards as well, we will see what form the regulations take when prices reflect costs and healthy margins and become existential threats for many businesses.
Re: AI eats the world (Spring 26) [pdf]
#167> What happened the last time that everything changed? * Hardware era (pre 1995s) -> IBM, Intel, Microsoft, Apple * Internet era (1994-2001) -> Amazon, Google, Meta, Salesforce * Mobile era (iPhone+ era) -> Uber, Mobile Games, Youtube, Snapchat, Tiktok, Airbnb * Cloud era (AWS+ era) -> AWS, GCP, Azure, Snowflake, Databricks and bunch of other data & database startups AI era (ChatGPT+ era) -> Change is inevitable
> * Internet era (1994-2001) -> Amazon, Google, Meta, Salesforce
Meta, née Facebook, wasn’t started until 2004.
Re: AI eats the world (Spring 26) [pdf]
#168Earlier quoted context omitted.
Evidence point to the same type of scaling law. Compute for a training run grows 4-5x every year.[0] I'm sure this will slow down but the premise remains that weaker competitors will not be able to maintain this pace. We already see labs like Cohere, Mistral, Inflection AI, Adept, Character.ai, and others bow out of the frontier race. I'm also skeptical that Meta, xAI can catch up. Even Google has trouble keeping up.…
I agree with much of what you’ve written but think you are missing the correct alignment of the mobile data timeline — mobile data had standards because it was forced to. It was forced to early because it was not a fundamental innovation, telecom itself was the fundamental innovation, mobile was a constraint relaxation. Intelligence might be forced to have standards as well, we will see what form the regulations take…
Intelligence can’t be standardized.
The reason mobile data had to standardize is because it’s a network and a network must have protocols. It’s useless without them.
Re: AI eats the world (Spring 26) [pdf]
#169Earlier quoted context omitted.
It is not industry practice to actually give the sources then? Just list vague names in a short "Sources:Companies" note? I now see the inflation indication as well. Which is not on the y-axis label, but in the "Sources:"-note. I will concede that. But what the hell my guy, why is it down there. If "financial analysis" is less rigid in sourcing requirements than grade school, what are you guys even doing. If you have…
You are surprised to discover how an entire industry you know nothing about does things. You conclude that everyone in that industry must be an idiot doing bad work. This says far more about you than it does about me.
Yes, it says I have bare-minimum standards for how to cite references.
I never said I thought financial analysts are idiots. That's not a conclusion I made.
I gave a very reasonable reason: they want it to be more difficult to replicate the analysis. That makes sense if your goal is to make sure your work is profitable, rather than quality reporting and knowledge dissemination.
That would imply the people who set the standards (not everyone) is cynical and/or greedy. Not an idiot.
And still, presumably you are still allowed to add actual sources to financial analysis? You still failed the regular standard of good reporting on this. Your graphs also fail just regular data analysis sanity checks. There is, e.g., no thought given to whether it's even valid to compare data from two sources in one graph. You just do it.
Re: AI eats the world (Spring 26) [pdf]
#170Earlier quoted context omitted.
I agree with much of what you’ve written but think you are missing the correct alignment of the mobile data timeline — mobile data had standards because it was forced to. It was forced to early because it was not a fundamental innovation, telecom itself was the fundamental innovation, mobile was a constraint relaxation. Intelligence might be forced to have standards as well, we will see what form the regulations take…
Intelligence can’t be standardized. The reason mobile data had to standardize is because it’s a network and a network must have protocols. It’s useless without them.
I agree with that as a premise, but again it seems to
me you are selectively jumping way into the end game. There were early networks that did not standardize, and these nonstandard networks had advantages, and some of those advantages were sacrificed in market-driven standardization.
Intelligence must have interfaces, and those can be standardized. Businesses will try to remain provider agnostic, which will also drive standardization via standard sales and marketing methods.
Separately, we are doing our best to standardize performances on benchmarks.
I don’t disagree that right now transport of standardized mobile data vs emulation of human intelligence is qualitatively different, but perhaps primarily because it is early in development, and our vantage point this time is relatively from within the network, instead of outside it.