141–150 of 202 posts
Earlier quoted context omitted.
He made an analogy to discuss the business case for scaling an iPod, not whether or not new products and services would be invented. My iPhone still only has about 64gb of storage.
But it is the dumbest thing in the World to compare features of LLM to something like that.
Earlier quoted context omitted.
> However, the more data you feed into an LLM, the smarter it should be in the response. Is it that way? For example if it lacks a certain reasoning capability, then more data may not change that. So far LLMs lack useful ideas of truth, it will easily generate untrue statements. We see lots of hacks how to control that, with unconvincing results.
That has not been my experience with GPT4 and GPt4o. Maybe you’re using worse models? The point is that the more context an LLM or human has, the better decision it can make in theory. I don’t think you can debate this. Hallucinations and LLM context scale are more engineering problems.
For example it's common to bury an adversary in paperwork in legal discovery to try to obscure what you don't want them to find.
Humans do not do better with excessive context and it is appearing that although you can go to 2m tokens etc. they don't actually understand. They ONLY do well at "find the needle in the haystack, here's a very specific description of the needle" tasks but nothing that involves simultaneously considering multiple parts of that.
Earlier quoted context omitted.
I’m still waiting for SalesForce to integrate an LLM into Slack so I can ask it business logic and decisions long lost. Still waiting for Microsoft to integrate an LLM into outlook so I can get a summary of a 20 email long chain I just got CCed into. I don’t think the iPod comparison is a valid one. People only have so much time to listen to music. Past a certain point, no one has enough good music they like to put i…
Haha, I'd be happy if outlook just integrated a search that actually works. Most of outlook search results aren't even relevant, and it regularly misses things I know are there. Literally the most useless search I've ever had to use.
> He uses the iPod as an example. For a few iterations, it did in fact double in storage size from 10 all the way to 160GB. If it had continued on that trajectory, he figured out we would have an iPod with 160TB of storage by 2017, but of course we didn’t. I think Brooks' opinions will age poorly, but if anyone doesn't already know all the arguments for that they aren't interested in learning them now. This quote see…
Exponentials have a bad habit of turning into sigmoids due to fundamental physical and geometric, and also practical, constraints. There’s the thing called diminishing returns. Every proven technology reaches maturity; it happened to iPods, it happened to smartphones, it happened to digital cameras and so on. There’s still growth and improvement, but it’s greatly slowed down from the early days. But that’s not to say…
Most rigorous definitions of quality incorporate a target expectation that is impossible to exceed. You are removing errors, so best case, exponential improvement in quality adds nines - a la the sigmoid function you mention.
Earlier quoted context omitted.
I’m still waiting for SalesForce to integrate an LLM into Slack so I can ask it business logic and decisions long lost. Still waiting for Microsoft to integrate an LLM into outlook so I can get a summary of a 20 email long chain I just got CCed into. I don’t think the iPod comparison is a valid one. People only have so much time to listen to music. Past a certain point, no one has enough good music they like to put i…
Haha, I'd be happy if outlook just integrated a search that actually works. Most of outlook search results aren't even relevant, and it regularly misses things I know are there. Literally the most useless search I've ever had to use.
To me, this reads like a very reasonable take. He suggests to limit the scope of the AI problem, add manual overrides in case there are unexpected situations, and he (rightly, in my opinion) predicts that the business case for exponentially scaling LLM models isn't there. With that context, I like his iPod example. Apple probably could have made a 3TB iPod to stick to Moore's law for another few years, but after they…
I’m still waiting for SalesForce to integrate an LLM into Slack so I can ask it business logic and decisions long lost. Still waiting for Microsoft to integrate an LLM into outlook so I can get a summary of a 20 email long chain I just got CCed into. I don’t think the iPod comparison is a valid one. People only have so much time to listen to music. Past a certain point, no one has enough good music they like to put i…
That’s my biggest worry. I’ve set up my own RAG and it’s kind of sad how even that is not super accurate.
Earlier quoted context omitted.
Haha, I'd be happy if outlook just integrated a search that actually works. Most of outlook search results aren't even relevant, and it regularly misses things I know are there. Literally the most useless search I've ever had to use.
Most useless ever? You've used a Windows OS, right?
Earlier quoted context omitted.
Haha, I'd be happy if outlook just integrated a search that actually works. Most of outlook search results aren't even relevant, and it regularly misses things I know are there. Literally the most useless search I've ever had to use.
If they can't make an effective basic text search, I doubt they're going to manage an LLM integration that doesn't return straight up garbage.