Live data from Hacker News

We don't have a hundred biases, we have the wrong model

worksinprogress.co

31–40 of 97 posts

Re: We don't have a hundred biases, we have the wrong model

#31
post #21

If you apply the cognitive biases model to algorithms which have superhuman performance in various games - like AlphaZero, DeepBlue, Pluribus, and so on - the natural result is to conclude that these models are predictably irrational. The reason you get this conclusion is because it turns out to be necessary to trade off theoretical optimal answers for the sake of speed. The behavioral economic view of human irration…

> apply the cognitive biases model to algorithms which have superhuman performance in various games Could you give an example of this?

- Anthropic bias

The algorithms have this tendency. They use counterfactual reasoning to determine that assuming a nash player alike to them is their opponent when making their decisions. Sometimes they don't have a nash opponent, but they persist in this assumption anyway. In the cognitive bias framing this tendency is error. In the game theoretic framing this corresponds with minimizing the degree to which you would be exploited. You can find times where the algorithm plays against something that isn't nash and so it was operating according to a flawed model. You can call it biased for assuming that others operated according to that flawed model. From a complexity perspective this assumption lets you drop an infinite number of continuous strategy distributions from consideration - with strong theoretical backing for why it won't hurt you to do so - since nash is optimal according to some important metrics.

- Attentional bias

The tendency to pay attention to some things and not other things. Some examples of times where we do that are with alpha beta pruning. You can find moves that involve sacrifice that show the existence of this bias. The conceit in the cognitive bias framing is that it is stupid because some of the things might be important. The justification is that it some things are more promising than others and we have limited computational budget. Better to stop exploring the things which are not promising since they are not promising and direct efforts to where they are promising. Something like an upper confidence bound tree search in the cognitive bias model would turn balancing the explore exploit dynamic as part of approximating the nash equillibrium into erroneous reasoning because it doesn't choose to explore everything is an example of the lesser form of anchoring effects as they relate to attentional bias. It weights the action values according to the promising rollout more highly.

- Apophenia

Hashing techniques are used to reduce dimensionality. There is an error term here but you gain faster reasoning speed. Seen in blueprint abstraction - the poker example I gave - since we've hashing down using similarity to help bucket similar things. This gives rise to things like selective attention (another bias, and kind of related to this general category of bias).

Jumping ahead to something like confirmation bias the heuristic that all these algorithms are using are flawed in various ways. They see that they are flawed after a node expansion and update their beliefs, but they don't update the heuristic. In fact if a flawed heuristic was working well such that it won we would have greater confidence rather than lesser confidence in the bias.

---

Putting all that aside I would caution against specifity in understanding my point. I think approaching it in this direction - very specific examples - is horrible because it directs attention to the wrong things; when you look at specific examples you're always in a more specific situation and if you're in a more specific situation it means that your situation is more computationally tractable than the general situation which was being handled by the algorithm. So trying to focus on examples is actually going to give you weird inversions where the rules that applied in general don't apply to the specific situation.

You need to come about it from the opposite direction - from the problem descriptions to the necessary constraints on your solution. Then it happens that the error in reasoning is a natural result of trying to do well.

Re: We don't have a hundred biases, we have the wrong model

#32

If you apply the cognitive biases model to algorithms which have superhuman performance in various games - like AlphaZero, DeepBlue, Pluribus, and so on - the natural result is to conclude that these models are predictably irrational. The reason you get this conclusion is because it turns out to be necessary to trade off theoretical optimal answers for the sake of speed. The behavioral economic view of human irration…

The thing about continuous space solutions is that they are typically differentiable, which means you can use a gradient descent or LM optimization rather than needing to fully explore the solution space. Typically there are large regions which are heuristically excludable, which is what you are getting at I think, but even an unbiased sampling plus gradient descent often makes problems much more tractable than discr…

The type of learning problem where I agree with your point is in something like learning how to classify hand written digits. My point about the continuous nature being unsearchable in practice is about recursive forms - if I choose this policy, my opponent will choose to react to the fact that I had that policy.

In your learning problem where thing were made tractable by differentiation you have something like an elevation map that you are following, but in the multi-stage decision problem you have something more like a fractal elevation map. When you want to know the value of a particular point on the elevation map you have to look for the highest point or the lowest point on the elevation map you get by zooming in on the area which is the resultant of your having chosen a particular policy.

The problem is that since this is a multi-agent environment they can react to your policy choice. So they can for example choose to have you get a high value only if you have the correct password entered on a form. That elevation map is designed to be a plain everywhere and another fractal zoom corresponding with a high utility or a low error term only at the point where you enter the right password.

Choose a random point and you aren't going to have any information about what the password was. The optimization process won't help you. So you have to search. One way to do that is to do a random search; if you do that you eventually find a differing elevation - assuming one exists. But what if there were two passwords - one takes you to a low elevation fractal world that corresponds with a low reward because it is a honeypot. The other takes you to the fractal zoom where the elevation map is conditioned on you having root access to the system.

This argument shows us that we actually would need to search over every point to get the best answer possible. Yet if we do that we have to search over the entire continuous distribution for our policy. Since by definition there are an infinite number of states a computer with infinite search speed can't enumerate them; there is another infinite fractal under every policy choice that also needs full enumeration. We have non-termination by a diagonalization argument for a computer that has infinite speed.

Now observe that in our reality passwords exist. Less extreme - notice that reacting to policy choice in general, for example, moving out of the way of a car that drives toward you but not changing the way you would walk if it doesn't, isn't actually an unusual property in decision problems. It is normal.

Re: We don't have a hundred biases, we have the wrong model

#33

If you apply the cognitive biases model to algorithms which have superhuman performance in various games - like AlphaZero, DeepBlue, Pluribus, and so on - the natural result is to conclude that these models are predictably irrational. The reason you get this conclusion is because it turns out to be necessary to trade off theoretical optimal answers for the sake of speed. The behavioral economic view of human irration…

The thing about continuous space solutions is that they are typically differentiable, which means you can use a gradient descent or LM optimization rather than needing to fully explore the solution space. Typically there are large regions which are heuristically excludable, which is what you are getting at I think, but even an unbiased sampling plus gradient descent often makes problems much more tractable than discr…

Only if the local optima are good.

Re: We don't have a hundred biases, we have the wrong model

#34
post #23

The rational actor model assumes that a person will behave optimally - using all information available to make and carry out the best decision possible for their goals. I strongly suspect that a better model is that people instead of optimizing their outcomes instead optimize the ease of decision making while still getting an acceptable course of action. Most of our biases serve to either allow us to make decisions q…

This is "bounded rationality" [1], where people make the best decisions possible given computational constraints on how they make decisions. A lot of interesting work tries to derive human cognitive biases from this idea.

[1] https://en.wikipedia.org/wiki/Bounded_rationality

Re: We don't have a hundred biases, we have the wrong model

#35
post #6

》epicycles were still not enough to describe what could be observed. Epicycles based models were far superior in practice, such as predicting planetary conjunctions. Heliocentric models did not really catched up, until Newton invented gravity and calculus. And centre of mass of solar system (barycenter in Newtonian physics), is outside of Sun, so heliocentric models technically never gave solid predictions! Stellar p…

Invented gravity?

Turns out, yes.

Gravity - some mystical force that attracts masses together - turns out to be a completely fictional thing. Mass curves spacetime, objects actually move in straight lines, and the fact you can explain the results of that as an 'attractive force' turns out to just be a convenient invention. The idea of summing how all that works in terms of a simple inverse square force is just an ingenious human observation and invention.

Re: We don't have a hundred biases, we have the wrong model

#36

If you apply the cognitive biases model to algorithms which have superhuman performance in various games - like AlphaZero, DeepBlue, Pluribus, and so on - the natural result is to conclude that these models are predictably irrational. The reason you get this conclusion is because it turns out to be necessary to trade off theoretical optimal answers for the sake of speed. The behavioral economic view of human irration…

It sound like you're talking about, or at least brushing up against, prudential judgement[0]. Sometimes, the optimal move is not to seek the optimum. An obvious class of problems is where determining the optimum takes more time than the lifetime of the problem. Say you need to write an algorithm at work that does X, and you need X by tomorrow. If it would take you a week to find the theoretical optimum, then the opti…

Yeah, that is pretty close to what I'm talking about. Coming at it from a different perspective - learning theory - but it seems to be the same overarching idea. I'm extending it a little though to something similar to anachronistic reasoning being incorrect - you can't divorce prudential decisions from their context. When you do judgement of the decisions is flawed because it doesn't acknowledge the actual constraints the decision was made under.

Re: We don't have a hundred biases, we have the wrong model

#38
post #6

Earlier quoted context omitted.

Invented gravity?

gravity is a notation for describing and predicting an arbitrary subset of natural processes you might as well contest that he invented calculus

As opposed to discovered it? I think most people would say he discovered calculus rather than invented it.

Re: We don't have a hundred biases, we have the wrong model

#39

If you apply the cognitive biases model to algorithms which have superhuman performance in various games - like AlphaZero, DeepBlue, Pluribus, and so on - the natural result is to conclude that these models are predictably irrational. The reason you get this conclusion is because it turns out to be necessary to trade off theoretical optimal answers for the sake of speed. The behavioral economic view of human irration…

The thing about continuous space solutions is that they are typically differentiable, which means you can use a gradient descent or LM optimization rather than needing to fully explore the solution space. Typically there are large regions which are heuristically excludable, which is what you are getting at I think, but even an unbiased sampling plus gradient descent often makes problems much more tractable than discr…

No post body was provided.

Re: We don't have a hundred biases, we have the wrong model

#40
post #12

》epicycles were still not enough to describe what could be observed. Epicycles based models were far superior in practice, such as predicting planetary conjunctions. Heliocentric models did not really catched up, until Newton invented gravity and calculus. And centre of mass of solar system (barycenter in Newtonian physics), is outside of Sun, so heliocentric models technically never gave solid predictions! Stellar p…

I would normally be skeptical of an article that starts with a description of epicycles because it probably means that whatever is going to be described next is totally bullshit. In this case I’m not so sure. As a plebeian normie, it seems like the “rational actor” model of economics has a lot of problems. Now I do believe that All people are All of the time trying to achieve their goals and meet their needs as can b…

The real question for me is, do you think that a Government is different or in a better position than corporations or people operating in the market in making economic decision for an entire country?

I believe it isn't. Actually I think it's in a much worse position for the following reasons:

1) A Government is made of people (usually elected directly or indirectly by the majority based on feelings and all the same irrationality), which in turn will likely be "irrational", or have the wrong incentives (be elected again).

2) A Government is made of few people compared to all the people that there are in the Country. They can't possibly know about all the details of the economy and the situations people are in or they can't process it.

3) Government policies can affect the entire economy. An error there can have bigger repercussions than, for instance, a company making a mistake.

Post reply on HN