11–20 of 202 posts
> He uses the iPod as an example. For a few iterations, it did in fact double in storage size from 10 all the way to 160GB. If it had continued on that trajectory, he figured out we would have an iPod with 160TB of storage by 2017, but of course we didn’t. I think Brooks' opinions will age poorly, but if anyone doesn't already know all the arguments for that they aren't interested in learning them now. This quote see…
He suggests to limit the scope of the AI problem, add manual overrides in case there are unexpected situations, and he (rightly, in my opinion) predicts that the business case for exponentially scaling LLM models isn't there. With that context, I like his iPod example. Apple probably could have made a 3TB iPod to stick to Moore's law for another few years, but after they reached 160GB of music storage there was no usecase where adding more would deliver more benefits than the added costs.
> He uses the iPod as an example. For a few iterations, it did in fact double in storage size from 10 all the way to 160GB. If it had continued on that trajectory, he figured out we would have an iPod with 160TB of storage by 2017, but of course we didn’t. I think Brooks' opinions will age poorly, but if anyone doesn't already know all the arguments for that they aren't interested in learning them now. This quote see…
Do we still need to make a fair claim against unrestricted exponential growth in 2024? Exponential growth claims have been made countless times in the past, and never delivered. Studies like the Limits to Growth report (1972) have shown the impossibility of unrestricted growth, and the underlying science can be used in other domains to show the same. There is no question that exponential growth doesn't exist, the only interesting question is how to locate the inflexion point.
Apparently the only limitless resource is gullible people.
To me, this reads like a very reasonable take. He suggests to limit the scope of the AI problem, add manual overrides in case there are unexpected situations, and he (rightly, in my opinion) predicts that the business case for exponentially scaling LLM models isn't there. With that context, I like his iPod example. Apple probably could have made a 3TB iPod to stick to Moore's law for another few years, but after they…
I don’t think the iPod comparison is a valid one. People only have so much time to listen to music. Past a certain point, no one has enough good music they like to put into a 3TB iPod. However, the more data you feed into an LLM, the smarter it should be in the response. Therefore, the scale of iPod storage and LLM context is on completely different curves.
To me, this reads like a very reasonable take. He suggests to limit the scope of the AI problem, add manual overrides in case there are unexpected situations, and he (rightly, in my opinion) predicts that the business case for exponentially scaling LLM models isn't there. With that context, I like his iPod example. Apple probably could have made a 3TB iPod to stick to Moore's law for another few years, but after they…
I’m still waiting for SalesForce to integrate an LLM into Slack so I can ask it business logic and decisions long lost. Still waiting for Microsoft to integrate an LLM into outlook so I can get a summary of a 20 email long chain I just got CCed into. I don’t think the iPod comparison is a valid one. People only have so much time to listen to music. Past a certain point, no one has enough good music they like to put i…
Given that Microsoft this year is all-in on LLM AI, this is surely coming.
But perhaps it will be a premium, paid feature?
> He uses the iPod as an example. For a few iterations, it did in fact double in storage size from 10 all the way to 160GB. If it had continued on that trajectory, he figured out we would have an iPod with 160TB of storage by 2017, but of course we didn’t. I think Brooks' opinions will age poorly, but if anyone doesn't already know all the arguments for that they aren't interested in learning them now. This quote see…
The same is true for LLMs. What they can do is impressive but to fix what they can't will be hard or even impossible.
Earlier quoted context omitted.
I’m still waiting for SalesForce to integrate an LLM into Slack so I can ask it business logic and decisions long lost. Still waiting for Microsoft to integrate an LLM into outlook so I can get a summary of a 20 email long chain I just got CCed into. I don’t think the iPod comparison is a valid one. People only have so much time to listen to music. Past a certain point, no one has enough good music they like to put i…
> Still waiting for Microsoft to integrate an LLM into outlook Given that Microsoft this year is all-in on LLM AI, this is surely coming. But perhaps it will be a premium, paid feature?
i dont know much about machine learning but what i think i know is that its getting an outcome based on averages of witnessed data/events. so how's it going to come up with anything novel? or outside of normal?
You have some input, which may include labels or not.
If it does have labels, ML is the automatic creation of a function that maps the examples to the labels, and the function may be arbitrarily complex.
When the function is complex enough to model the world that created the examples, it's also capable of modelling any other input; this is why LLMs can translate between languages without needing examples of the specific to-from language pair in the training set.
The data may also be synthetic based on the rules of the world; this is how AlphaZero beats the best Go and chess players without any examples from human play.
To me, this reads like a very reasonable take. He suggests to limit the scope of the AI problem, add manual overrides in case there are unexpected situations, and he (rightly, in my opinion) predicts that the business case for exponentially scaling LLM models isn't there. With that context, I like his iPod example. Apple probably could have made a 3TB iPod to stick to Moore's law for another few years, but after they…
I’m still waiting for SalesForce to integrate an LLM into Slack so I can ask it business logic and decisions long lost. Still waiting for Microsoft to integrate an LLM into outlook so I can get a summary of a 20 email long chain I just got CCed into. I don’t think the iPod comparison is a valid one. People only have so much time to listen to music. Past a certain point, no one has enough good music they like to put i…
This is not obvious though.