271–280 of 461 posts
think it's missing the biggest assumption, which is not necessarily true for regular software either but much more true than AI: The same inputs should produce the same outputs. And that assumption is important because dependability is the strength of an automated process.
For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…
Earlier quoted context omitted.
I think there are people who know exactly how it works, they know how neural networks work, they know how the transformer architecture works, how attention works, embeddings, tokenization, etc. We just can’t define the weights of the connections between the neurons.
That's a bit like saying knowing how a pipe works is enough to explain a combustion engine. You're just listing part of how LLMs work. Those mechanisms only explain next word prediction, not LLM reasoning. That's an emergent property that no person, as far as I understand it, can explain past hand waving. Happy to be corrected here.
> bugs are usually caused by problems in the data used to train an AI This also is a misunderstanding. The LLM can be fine, the training and data can be fine, but because the LLMs we use are non-deterministic (at least in regard to their being intentional attempts at entropy to avoid always failing certain scenarios) current algorithms are inherently by-design not going to always answer every question correctly that…
Earlier quoted context omitted.
> minor tools for making emojis, summarizing notifications, and proof reading. The notification / email summaries are so unbelievably useless too: it’s hardly more work to skim the notification / email that I do anyway.
Like most AI products it feels like they started with a solution first and went searching for the problems. Text messages being too long wasn't a real problem to begin with. There are some good parts to Apple Intelligence though. I find the priority notifications feature works pretty well, and the photo cleanup tool works pretty well for small things like removing your finger from the corner of a photo, though it's n…
I want to open WhatsApp and open the message and have it clear the notif. Or atleast click the notif from the normal notif center and have it clear there. It kills me
My current method for trying to break through this misconception is informing people that nobody knows how AI works. Literally. Nobody knows. (Note that knowing how to make something is not the same as knowing how it works. Take humans as an obvious example.)
How is it possible that nobody knows how it works - it’s running on hardware we have complete control over and perfect observability into, is it not? At any frame we can pause, examine the state, then step forward, examine the state, and observe what changes have occurred - we have perfect knowledge of the source code, the compiler, whatever components you prefer to break software down into - What is it that we don’t…
The ML field has a good understanding of the algorithms that produce these floating point numbers and lots of techniques that seem to produce “better” numbers in experiments. However, there is little to no understanding of what the numbers represent or how they do the things they do.
My current method for trying to break through this misconception is informing people that nobody knows how AI works. Literally. Nobody knows. (Note that knowing how to make something is not the same as knowing how it works. Take humans as an obvious example.)
How is it possible that nobody knows how it works - it’s running on hardware we have complete control over and perfect observability into, is it not? At any frame we can pause, examine the state, then step forward, examine the state, and observe what changes have occurred - we have perfect knowledge of the source code, the compiler, whatever components you prefer to break software down into - What is it that we don’t…
And inspecting each part is not enough to understand how, together, they achieve what they achieve. We would need to understand the entire system in a much more abstract way, and currently we have nothing more than ideas of how it _might_ work.
Normally, with software, we do not have this problem, as we start on the abstract level with a fully understood design and construct the concrete parts thereafter. Obviously we have a much better understanding of how the entire system of concrete parts works together to perform some complex task.
With AI, we took the other way: concrete parts were assembled with vague ideas on the abstract level of how they might do some cool stuff when put together. From there it was basically trial-and-error, iteration to the current state, but always with nothing more than vague ideas of how all of the parts work together on the abstract level. And even if we just stopped the development now and tried to gain a full, thorough understanding of the abstract level of a current LLM, we would fail, as they already reached a complexity that no human can understand anymore, even when devoting their entire lifetime to it.
However, while this is a clear difference to most other software (though one has to get careful when it comes to the biggest projects like Chromium, Windows, Linux, ... since even though these were constructed abstract-first, they have been in development for such a long time and have gained so many moving parts in the meantime that someone trying to understand them fully on the abstract level will probably start to face the difficulty of limited lifetime as well), it is not an uncommon thing per se: we also do not "really" understand how economy works, how money works, how capitalism works. Very much like with LLMs, humanity has somehow developed these systems through interaction of billions of humans over a long time, there was never an architect designing them on an abstract level from scratch, and they have shown emergent capabilities and behaviors that we don't fully understand. Still, we obviously try to use them to our advantage every day, and nobody would say that modern economies are useless or should be abandoned because they're not fully understood.
I found this statement particularly relevant: While it’s possible to demonstrate the safety of an AI for a specific test suite or a known threat, it’s impossible for AI creators to definitively say their AI will never act maliciously or dangerously for any prompt it could be given. This possibility is compounded exponentially when MCP[0] is used. 0 - https://github.com/modelcontextprotocol
Nevertheless, in the book, the AI managed to convince people, using the light signal, to free it. Furthermore, it seems difficult to sandbox any AI that is allowed to access dependencies or external resources (i.e. the internet). It would require (e.g.) dumping the whole Internet as data into the Sandbox. Taking away such external resources, on the other hand, reduces its usability.
For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…
However, when I stopped driving and looked at the picture the AI generated description was pretty poor - it wasn't completely wrong but it really wasn't what I was expecting given the description.