What if we just aren't doing enough, and we need to use GAN techniques with the LLMs.
We're at the "lol, ai cant draw hands right" stage with these hallucinations, but wait a couple years.
The "confident idiot" problem: Why AI needs hard rules, not vibe checks
231–240 of 399 posts
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#232we're inching towards the three laws of robotics
Your username makes me think you might be a little biased.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#233The thing that bothers me the most about LLMs is how they never seem to understand "the flow" of an actual conversation between humans. When I ask a person something, I expect them to give me a short reply which includes another question/asks for details/clarification. A conversation is thus an ongoing "dance" where the questioner and answerer gradually arrive to the same shared meaning. LLMs don't do this. Instead,…
LLMs all behave as if they are semi-competent (yet eager, ambitious, and career-minded) interns or administrative assistants, working for a powerful CEO-founder. All sycophancy, confidence and positive energy. "You're absolutely right!" "Here's the answer you are looking for!" "Let me do that for you immediately!" "Here is everything I know about what you just mentioned." Never admitting a mistake unless you directly…
Analogies of LLMs to humans obfuscates the problem. LLMs aren't like humans of any sort in any context. They're chat bots. They do not "think" like humans and applying human-like logic to them does not work.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#234Earlier quoted context omitted.
Interestingly, coding models often struggle with complex type systems, e.g. in Haskell or Rust. Of course, part of this has to do with the relative paucity of relevant training data, but there are also "cognitive" factors that mirror what humans tend to struggle with in those languages. One big factor behind this is the fact that you're no longer just writing programs and debugging them incrementally, iteratively dea…
Actually I found the coding models to work really well with these languages. And the type systems are not actually complex. Ocaml's type system is actually really simple, which is probably why the compiler can be so fast. Even back in the "beta" days of Copilot, despite being marketed as Python only, I found it worked for Ocaml syntax and worked just as well. The coding models work really well with esoteric syntaxes…
There's a reason I mentioned Haskell and Rust specifically. You're right, OCaml's type system is simpler in some relevant respects, and may avoid the issues that I was alluding to. I haven't worked with OCaml for a number of years, since before the LLM boom.
The presence of type classes in Haskell and traits in Rust, and of course the memory lifetime types in Rust, are a big part of the complexity I mentioned.
(Edit: I like type classes and traits. They're a big reason I eventually settled on Haskell over OCaml, and one of the reasons I like Rust. I'm also not such a fan of the "O" in OCaml.)
> All possible runs of a program is exactly what HM type systems type check for.
Yes, my point was this can be a more difficult goal to achieve.
> This fed into the coding model automatically iterates until it finds a solution that doesn't violate any possible run of the program.
Only if the model is able to make progress effectively. I have some amusing transcripts of the opposite situation.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#235The thing that bothers me the most about LLMs is how they never seem to understand "the flow" of an actual conversation between humans. When I ask a person something, I expect them to give me a short reply which includes another question/asks for details/clarification. A conversation is thus an ongoing "dance" where the questioner and answerer gradually arrive to the same shared meaning. LLMs don't do this. Instead,…
ChatGPT offered a "robotic" personality which really improved my experience. My frustrations were basically decimated right away and I quickly switched to a more "You get out of it what you put in" mindset. And less than two weeks in they removed it and replaced it with some sort of "plain and clear" personality which is human-like. And my frustrations ramped up again. That brief experiment taught me two things: 1. I…
Sort of a personal modified Butlerian Jihad? Robots / chatbots are fine as long as you KNOW they're not real humans and they don't pretend to be.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#236Earlier quoted context omitted.
We likely learn/generate language in an auto-regressive way at least conceptually similar to an LLM, but this isn't just self-contained auto-regressive generation... Humans use language to express something (facts, thoughts, etc), so you can consider these thoughts being expressed as a bias to the language generation process, similar perhaps to an image being used as a bias to the captioning part of an image captioni…
> Humans use language to express something (facts, thoughts, etc), so you can consider these thoughts being expressed as a bias to the language generation process My point however is more that the "thoughts being expressed" are themselves being generated by a similar process (and that it's either that or a God-given soul).
Similar in the sense of being mechanical (no homunculus or soul!) and predictive, but different in terms of what's being predicted (auto-regressive vs external).
So, with the LLM all you have is the auto-regressive language prediction loop.
With animals you primarily have the external "what happens next" prediction loop, with these external-world fact-based predictions presumably also the basis of their thoughts (planning/reasoning), as well as behavior.
If it's a human animal who has learned language, then you additionally have an LLM-like auto-regressive language prediction loop, but now, unlike the LLM, biased (controlled) by these fact-based thoughts (as well as language-based thoughts).
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#237The thing that bothers me the most about LLMs is how they never seem to understand "the flow" of an actual conversation between humans. When I ask a person something, I expect them to give me a short reply which includes another question/asks for details/clarification. A conversation is thus an ongoing "dance" where the questioner and answerer gradually arrive to the same shared meaning. LLMs don't do this. Instead,…
Its not a magic technology, they can only represent data they were trained on. Naturally most represented data in their training data is NOT conversational. Consider that such data is very limited and who knows how it was labeled if at all during pretraining. But with that in mind, LLM's definitely can do all the things you describe, but a very robust and well tested system prompt has to be used to coax this behavior out. Also a proper model has to be used, as some models are simply not trained for this type of interaction.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#238Earlier quoted context omitted.
This is a recent phenomenon. It seems most of the pages today are SEO optimized LLM garbage with the aim of having you scroll past three pages of ads. THe internet really used to be efficient and i could always find exactly what i wanted with an imprecise google search ~ 15 years ago.
You'd think with the reputation of LLMs being trained on Twitter (pre-Musk radicalization) and Reddit, they'd be better at understanding normal conversation flow since twitter requires short responses and Reddit... while Wall of Text happens occasionally, it's not the typical cadence of the discussion.
Reddit and Twitter don't have human conversations. They have exchanges of confident assertions followed with rebuttals. In fact, both of our comments are perfect demonstrations of exactly that too. Fairly reflective of how LLMs behave — except nobody wants to "argue" with an LLM like Twitter and Reddit users want to.
This is not how humans converse in human social settings. The medium is the message, as they say.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#239Earlier quoted context omitted.
Of users. It's an implicit subject from the first sentence.
But how do they know that, if it's of all users?
They didn't claim to know it, they said "it seems to me". Presumably they're extrapolating from their experience, or their expectations of how an average user would behave.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#240The thing that bothers me the most about LLMs is how they never seem to understand "the flow" of an actual conversation between humans. When I ask a person something, I expect them to give me a short reply which includes another question/asks for details/clarification. A conversation is thus an ongoing "dance" where the questioner and answerer gradually arrive to the same shared meaning. LLMs don't do this. Instead,…
LLMs all behave as if they are semi-competent (yet eager, ambitious, and career-minded) interns or administrative assistants, working for a powerful CEO-founder. All sycophancy, confidence and positive energy. "You're absolutely right!" "Here's the answer you are looking for!" "Let me do that for you immediately!" "Here is everything I know about what you just mentioned." Never admitting a mistake unless you directly…
I agree entirely, and I think it's worthwhile to note that it may not even be the LLM that has that behavior. It's the entire deterministic machinery between the user and the LLM that creates that behavior, with the system prompt, personality prompt, RLHF, temperature, and the interface as a whole.
LLMs have an entire wrapper around them tuned to be as engaging as possible. Most people's experience of LLMs is a strongly social media and engagement economy influenced design.