From Bing to Sydney

Evolution doesn't have opinions so it's not in a camp.

Human behaviors like reciprocity and consideration for feelings are indeed part of human collective behavior. Calling such behavior "rational" misses the point - such behavior exists and we have the benefit of social existence because of it and this bring us benefits collectively. But individual calculating purely individual benefit would naturally just fake social engagement - roughly such individuals are know as sociopaths and they can succeed individually being a detriment to society. Which is to say a social creature is a matter of rationality but simply evolutionary result.

Still, the one thing most people would say is irrational is trusting a sociopath. Now, a Chat bot is absolutely a thing programmed to mimic human social conventions. A view that anthropomorphizes a Chat bot doesn't see that the chat bot isn't going to be actually bounded by human conventions except accidentally or instrumentally, basically the same as trusting sociopath.

It's important to remember that Google search also returns false results for all kinds of searches and that's it's been getting slowly worse for years.

Recently I searched Google for "bamboo sign" because I was designing a 3d model building and I wanted a placeholder texture for the sign.

What I got was loads of results for "bamboo spine" which apparently is a skeletal disorder of some kind. Putting "sign" in quotes or the entire "bamboo sign" in quotes didn't make any difference, Google had decided I was looking for information about spines and that was it.

I switched over to duckduckgo and got the results I wanted immediately (Duckduckgo, of course, is bad at loads of other things that Google would do better at).

Before people dismiss chat based search for sometimes being incorrect, I think we need a comprehensive test: ask both Google search and the new Bing Chat search a few hundred simple questions on a broad range of topics and see which gives more incorrect answers.

In my model, when you talk about anthropomorphism, seemingly as a negative, I realize I've noticed things which a coupled model doesn't predict: that intentional error via anthropomorphism can not just be correct, but that your scare quotes around rational while trying to denigrate the idea that it can be correct could not be more wrong, because the hard to vary causal explanation of why we ought to anthropomorphize gives a causal mechanism for why we ought to which is intimately tied in, not with being irrational, but with being more rational.

I realize this sounds insane, but the math and empirical investigation supports it. Which is why I think it is worth sharing with you. So I'm trying to share a thing that I consider likely to be very surprising to you even to the point of seeming non-sensical.

Would you like a link to an interesting technical talk by a NIPS best paper award winning researcher which delves into this subject and whose works advanced the state of the art in both game theory and natural language applied on strategic problems in the context of chat agents? Or do you not care whether anthropomorphism, when applied when it shouldn't be according to the analogical accuracy that usually decides whether logical analogy can be safely applied might be accurate beyond the level you thought it was?

I am not trying to disagree with you. I'm trying to talk to you about something interesting.

Yeah, probably it will evolve in that direction. I could imagine that happening.

> That's some weird reasoning.

In the AI textbooks I've read, reflex is defined in the context of a reflex agent. You would have sentences like "a reflex agent reacts without thinking" and then an example of that might be "a human who puts their hand on a stove yanks it away without thinking about it" and this is rational because the decision problem doesn't call for correct cognition - it calls for minimization of response time such that the hand isn't burned. To me, when you say reflex decision making is the reason for the danger, it seems to me that this is an inconsistent reason because for other decision making problems, reflex is a help, not a hindrance. I do not consider it wrong to or weird reasoning to use definitions sourced from AI research. I think, given your confusion at my post, you probably weren't intending to argue that being faster means being wrong, but the structure of your reply read that way to me because of the strong association I have for that word and reflex as it relates to optimal decision making by an AI under time constraints. I also think is what you actually said, even if you didn't intend to, but I don't doubt you if you say you meant it another way, because language is imprecise enough that we have to arrive on shared definitions in order to understand each other and it is by no means certain that we start on shared definitions.

I'm also kind of way too literal sometimes. Side-effect of being a programmer, I suppose. And I take this subject way too seriously, because I agree with Paul Graham about surface area of a general idea multiplying impact potential. So I'm trying really really really hard to think well - uh, for example, I've been thinking about this almost continuously whenever I reasonably could ever since my first reply, unable to stop.

It is 1:32 AM for me. I'm taking multiple continuous hours of thinking about this and writing about this and trying to be clear in my thinking about this, because I find it so important. So hopefully that gets across how I am as a person - even if it makes me seem really weird.

> You're fixing of my argument is OK but it's pretty easy to imagine it and others from the initial argument imo.

I'm really trying to drive at the deeper fundamental truths. I feel like logic and analogy are really important and profound and worthy of countless hours of thought about and that the effort will ultimately be rewarded.