341–350 of 461 posts
Earlier quoted context omitted.
What about them? It's possible for the question to unambiguously specify the required notational convention.
Is it? You have three wishes, which the maliciously compliant genie will grant you. Let’s hear your unambiguous request which definitely can’t be misinterpreted.
Earlier quoted context omitted.
This is not necessarily a problem. Any programming or mathematical question has several correct answers. The problem with LLMs is that they don't have a process to guarantee that a solution is correct. They will give a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way. That's why LLMs generate so many bugs in software and in anything related to logical t…
>> a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way Not quite ... LLMs are not HAL (unfortunately). They produce something that is associated with the same input, something that should look like an acceptable answer. A correct answer will be acceptable, and so will any answer that has been associated with similar input. And so will anything that fools…
This is very important and often overlooked idea. And it is 100% correct, even admitted by Anthropic themselves. When user asks LLM to explain how it arrived to a particular answer, it produces steps which are completely unrelated to the actual mechanism inside LLM programming. It will be yet another generated output, based on the training data.
For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…
Why not take the easy wins? Like let me change phone settings with Siri or something, but nope.
A lot of AI seems to be mismanaging it into doing things AI (LLMs) suck at... while leaving obvious quick wins on the table.
Hmm, I don't think any of these were true with non-AI software. Commonly held beliefs, sure.
If anything, I am glad AI is helping us revisit these assumptions.
- Software vulnerabilities are caused by mistakes in the code
Setting aside social engineering, mistake implies these were knowable in advance. Was the lack of TLS in the initial HTTP spec a mistake?
- Bugs in the code can be found by carefully analysing the code
If this was the case, why do people reach for rewriting buggy code they don't understand?
- Once a bug is fixed, it won’t come back again
Too many counter examples to this one in my lived experience.
- Every time you run the code, the same thing happens
Setting aside seeding PRNGs, there's the issue of running the code on different hardware. Or failing hardware.
- If you give specifications beforehand, you can get software that meets those specifications
I have never seen this work without needing to revise the specification during implementation.
My current method for trying to break through this misconception is informing people that nobody knows how AI works. Literally. Nobody knows. (Note that knowing how to make something is not the same as knowing how it works. Take humans as an obvious example.)
In a non-linear system the former is often easier than the latter. For example we know how planets “work” from the laws of motion. But planetary orbits involving > 2 bodies are non-linear, and predicting their motion far into the future is surprisingly difficult.
Neural networks are the same. They’re actually quite simple, it’s all undergraduate maths and statistics. But because they’re non-linear systems, predicting their behaviour is practically impossible.
Earlier quoted context omitted.
Yeah in that regard we should always treat it like a junior something. Very much like you can't expect your own kids to never do something dangerous even if tell it for years to be careful. I got used to getting my kid from the Kindergarten with a new injury at least once a month.
We should move well beyond human analogies. I have never met a human that would straight up lie about something, or build up so much deceptive tests that it might as well be lying. Granted this is not super common in these tools, but it is essentially unheard of in junior devs.
This doesn't match my experience. Consider high profile things like the VW emissions scandal, where the control system was intentionally programmed to only engage during the emissions test. Dictators. People are prone to lie when it's in their self interest, especially for self preservation. We have entire structures of government, courts, that try to resolve fact in the face of lying.
If we consider true-but-misleading, then politics, marketing, etc. come sharply into view.
I think the challenge is that we don't know when an LLM will generate untrue output, but we expect people to lie in certain circumstances. LLMs don't have clear self-interests, or self awareness to lie with intent. It's just useful noise.
Earlier quoted context omitted.
Neither, you’re reading it wrong. Think of it as codebases getting more reliable over time as they accumulate fixes and tests. (As opposed to, say, writing code in NodeJS versus C++)
Sounds more like survivorship bias. All the bad codebases were thrown out and only the good ones lasted a long time.
Earlier quoted context omitted.
Is it? You have three wishes, which the maliciously compliant genie will grant you. Let’s hear your unambiguous request which definitely can’t be misinterpreted.
"Do my interpretation of the wish."
Earlier quoted context omitted.
It does feel like somebody forgot that "from the first sentence or two of the email, you can tell what it's about" was already a rule of good writing...
You sometimes need to want to quickly learn what's in an email that was written by someone less helpful. Eg sometimes the writer is outright antagonistic, because they have some obligation to tell you something, but don't actually want you to know.