61–70 of 192 posts
A lot of HNers were so adamant that the LLMs understand absolutely nothing and that these models are just predicting the next most likely word. I think with this paper it becomes clear that the adamant denial was just human bias talking. LLMs crossed a certain line here. You would think people would be amazed or in awe or react in fear at technological break through a and they often are. The weird part for LLMs was t…
A bigger Chinese Room - is still a Chinese Room. We already went through this with ELIZA 60 years ago
Earlier quoted context omitted.
It was something like Paris - France + Japan = Tokyo I know that's not a source but you could also just google "word2vec", as I recall most of the explainer blog posts had similar examples.
So it's good at SAT type questions?
Wait I had the "It's just a stochastic parrot" parroted at me ad nauseum. Was this FUD all along?
No, this paper doesn't change that claim at all. This paper is about learnability and not generalization of what is learned. From the papers summary, note that they are referencing learning and not some implications that it gets us closer to a universal quantifier. "We have provided evidence that LLMs learn linear representations of space and time that are unified across entity types and fairly robust to prompting, a…
A lot of HNers were so adamant that the LLMs understand absolutely nothing and that these models are just predicting the next most likely word. I think with this paper it becomes clear that the adamant denial was just human bias talking. LLMs crossed a certain line here. You would think people would be amazed or in awe or react in fear at technological break through a and they often are. The weird part for LLMs was t…
Indeed. I was offering examples of how you could bootstrap cognitive processes at the beginning of the year and people were just sticking their fingers in their ears. A lot of people exercise their intellectual, economic, or political freedoms by rejecting others' efforts to gain the same things.
Could you please explain what the above has to do with a Large Languange Model?
Earlier quoted context omitted.
No, I've only used GPT-3 so far. Should I be excited about GPT-4? :) Examples of things I'd file as silly mistakes: - mentioning non-existing settings when asking it about a particular software. - responding to roughly "give me a way to list all rds clusters that are running a version that will reach EOL within the next 12 months" (which I know is almost impossible to do without scraping, as the EOLs are not returned…
GPT-4 should do significantly better, but still worse than expert humans, on the tasks you mentioned. They would also benefit from good prompting, such as those in - https://lilianweng.github.io/posts/2023-03-15-prompt-enginee... - https://help.openai.com/en/articles/6654000-best-practices-f... Not a large percentage of humans would be able to do the tasks listed above without significant experience or training eithe…
I'm in full agreement about humans and reasoning, btw. I just don't think chatgpt (with the version I've used) is anywhere on the same league as the worst (way below-average) humans either. I do think it's quite useful as a writing aid. You do need to review what it gives you, but you'd have to do that even if you were relying on a human writing aid.
A lot of HNers were so adamant that the LLMs understand absolutely nothing and that these models are just predicting the next most likely word. I think with this paper it becomes clear that the adamant denial was just human bias talking. LLMs crossed a certain line here. You would think people would be amazed or in awe or react in fear at technological break through a and they often are. The weird part for LLMs was t…
My only experience so far is with chatgpt, and I am not an expert in AI, or even just in LLMs. With those disclaimers, my interactions inform my opinion that the LLM behind chatgpt has no internal world model. It shows no understanding of basic facts and makes very silly mistakes very easily. I have my bias, like anyone else, but in the case of AI in particular, I should say that I don't think there's anything especi…
This is just the beginning. It is a tiny, tiny view into this new technology.
Nobody with an ounce of intelligence looked at the first iPhone and thought "ah well it's just another phone". It's the potential of this new technology that is so impressive.
How can you not be impressed by this? I am literally incapable of empathizing with your perspective. All I can think is that the world is going to pass you by.
Earlier quoted context omitted.
>because that way we'll only make machines with none of the advantages of machines and all the disadvantages of computers. There is no evidence, basically none whatsoever that general "perfect logical reasoning" is a thing that actually exists in the real world. None. No animal we've observed does it. Humans certainly don't do it. The only realm this idea actually works is Fiction. and this was not like for a lack of…
Would you say that this in itself is due to how incomplete human reasoning is in the first place? That as a result, our ideas of logic and what perfect logic looks like are bound to fail? Or are you saying that the purest mathematical representation of logic cannot scale to a point where they can model and predict real world relationships successfully?
Gödel’s Incompleteness Theorems also demonstrates that in any sufficiently powerful mathematical system, there are true statements that cannot be proven within the system. This implies that no matter how refined a logical system you devise, it will invariably be incomplete or inconsistent when grappling with real-world phenomena.
“Have an internal world model,” or “give the appearance of having an internal world model”? (Disclaimer: I’ve only read the summary, not the full paper) Recent papers about advanced Go-playing AIs have made it abundantly clear that, at best, their conceptual understanding of Go is deeply flawed, and at worst, completely absent. Yet under most circumstances they give the appearance of super-human understanding based o…
But the outcome of its action is irrefutable. It plays and does things better better than us. That is all we need to evaluate it on. I know that the topic is about "sentience" but my point is we are evaluating a complete alien being using human's standards. Of course we will see that it is lacking. It isn't human.
One thing that is clouding this discussion is that most people are mixing up a lot of different characteristics that animals like humans have as if they were all the same thing. So for many people they don't really distinguish between things like "reasoning", "self-aware", "conscious", "alive", "sentient", "intelligent", "has world model". They also don't distinguish between different types or varying levels of cogni…
AI entities, may exhibit one, or none or sometimes some and that maps poorly to our prior experience with reasoning entities.
How would self-aware, non-conscious agent respond?
How would a reasoning but non-intelligent agent respond?