The "overlearning the game" problem
andrewoneverything.com
The "overlearning the game" problem
1–10 of 144 posts
Re: The "overlearning the game" problem
#2[deleted]
Re: The "overlearning the game" problem
#3Less Wrong talks about this a lot; their term for one variant of it is "superstimuli".
What's the difference between your "overlearning" and "hacking"? They sound like the same thing.
This phenomenon is the reason for the Wikipedia rule, "Ignore All Rules".
Re: The "overlearning the game" problem
#4Less Wrong talks about this a lot; their term for one variant of it is "superstimuli". What's the difference between your "overlearning" and "hacking"? They sound like the same thing. This phenomenon is the reason for the Wikipedia rule, "Ignore All Rules".
> What's the difference between your "overlearning" and "hacking"? They sound like the same thing.
They are similar, but I think one difference between "hacking" and "overlearning" is that a hacker can learn to subvert/circumvent a system quite quickly, and usually does so in a way that breaks the rules. Overlearning usually takes longer to achieve, and is a more insidious problem because it achieves its goals without actually breaking any rules.
Re: The "overlearning the game" problem
#5Less Wrong talks about this a lot; their term for one variant of it is "superstimuli". What's the difference between your "overlearning" and "hacking"? They sound like the same thing. This phenomenon is the reason for the Wikipedia rule, "Ignore All Rules".
> What's the difference between your "overlearning" and "hacking"? They sound like the same thing. They are similar, but I think one difference between "hacking" and "overlearning" is that a hacker can learn to subvert/circumvent a system quite quickly, and usually does so in a way that breaks the rules. Overlearning usually takes longer to achieve, and is a more insidious problem because it achieves its goals withou…
It depends on what you think "the rules" are. Generally speaking, hacking happens in a framework that has unbreakable rules, whether those are rules of a game, of a CPU, or of an OS. A solution becomes a hack when it exploits those rules in a way that was previously thought impossible.
Of course, there may be other, breakable, rules that are broken — "You can't render 3-D in real time on a PC!" — but those aren't the relevant rules.
Many hacks are not quick at all to achieve.
Re: The "overlearning the game" problem
#6Earlier quoted context omitted.
> What's the difference between your "overlearning" and "hacking"? They sound like the same thing. They are similar, but I think one difference between "hacking" and "overlearning" is that a hacker can learn to subvert/circumvent a system quite quickly, and usually does so in a way that breaks the rules. Overlearning usually takes longer to achieve, and is a more insidious problem because it achieves its goals withou…
It depends on what you think "the rules" are. Generally speaking, hacking happens in a framework that has unbreakable rules, whether those are rules of a game, of a CPU, or of an OS. A solution becomes a hack when it exploits those rules in a way that was previously thought impossible. Of course, there may be other , breakable, rules that are broken — "You can't render 3-D in real time on a PC!" — but those aren't th…
I guess an example of hacking would be someone hacking the phone system to make free long distance calls. An example of overlearning would be someone learning the rules of society to such a degree that he gets free long distance calls (e.g. using lawsuits against phone companies).
The former would get fined or imprisoned if caught, but the latter is fine under our legal system.
Re: The "overlearning the game" problem
#7This article seems to be addressing 'the spirit' of a document or system.
Perhaps another way to look at the problem is to imagine creating an A.I. that you want to succeed at whatever system you present. In most cases, an A.I. will take the literal interpretation of the system and become a test-taker, an electable 'gotcha-game' politician or even an entity that finds it can maximize game theory to its own ends by complicating the rules of an existing system to the point of absurdity once it becomes powerful enough to modify and create rules.
So then how do we create systems resistant to beings that take everything literally? I suppose the only way is to reward certain outcomes as opposed to rewarding the direct product of the system itself.
Examples: After an election, have we elected someone who has met with a high degree of favorability in the electorate by the end of his term?
After having students become proficient test-takers, do they then become excellent doctors, lawyers, engineers, etc.. ?
In a game invented to be fun or fair, once overlearned, do they produce fun or fairness?
If not, then a new game needs to be created or the existing one might need to be extensively modified to produced the desired product. This is where the internet shines, where everyone is welcome to take an existing system and modify it to something better. The problem with politics, law, stock market, etc... is that they have become the only method adopted in real society (there is only one game in town).
If reality were allowed to adopt, incorporate or evolve from systems/games from virtual reality (i.e. internet) there might be some productive change. But first we would need to see the first step taking place, that being even flawed virtual systems are allowed to manifest in significant proportion within real life society.
Don't hate the game, hate the player. Every game that profits a winner will have its cheaters.
Re: The "overlearning the game" problem
#8Earlier quoted context omitted.
It depends on what you think "the rules" are. Generally speaking, hacking happens in a framework that has unbreakable rules, whether those are rules of a game, of a CPU, or of an OS. A solution becomes a hack when it exploits those rules in a way that was previously thought impossible. Of course, there may be other , breakable, rules that are broken — "You can't render 3-D in real time on a PC!" — but those aren't th…
I guess an example of hacking would be someone hacking the phone system to make free long distance calls. An example of overlearning would be someone learning the rules of society to such a degree that he gets free long distance calls (e.g. using lawsuits against phone companies). The former would get fined or imprisoned if caught, but the latter is fine under our legal system.
It's pretty bizarre that you're using the derogatory definition of "hack" given that you're posting on Hacker News. And it doesn't incline me to want to talk to you further.
Re: The "overlearning the game" problem
#9I'm reminded of Mechanism Design: a Nobel prize winning theory of economics that starts with the supposition that agents in any system will exploit its rules to maximize their personal gain. The corollary, which I've been quite taken by, is that where we can influence the rules of the game, we should design them such that exploitation serves a social good.
Re: The "overlearning the game" problem
#10It's related to Goodhart's Law:
Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.
This is often the result of attempting to overoptimize a system. You can optimize a race car to a huge degree, because you know exactly what you want it to do.
You can't optimize a schooling system, because you don't know exactly what you want it to do. A little noise is a good thing, because the you want a little wiggle room for teachers to sidestep the dictums of education czars, and students to sidestep the dictums of teachers.
The Greeks solved this quite a few years ago, with sortition. Under sortition (injecting noise into elections - http://en.wikipedia.org/wiki/Sortition), Bush and Gore would have been forced to pay "paper, scissors, rock" for the presidency. Under the US's more pure democracy, they would have been tempted to make all kind of Faustian bargains with sordid players to nail down the last 0.01% of votes.
Randomization means that the last percent is just not worth chasing, so players in a competition won't be tempted to bend the rules for a tiny advantage.
The same process could be used for tests. If you allocate places in desirable courses (say medicine) randomly to anyone above a certain score, the top students won't bother drilling as hard just to get the top score.
Stocks are the same - quants wouldn't sweat timing as much if their placement in order books was randomized. It would be more efficient to pay attention to fundamental value than momentary fluctuations if they weren't guaranteed to make large profits on the momentary fluctuations. Some would still work on timing, but not as many.
Patents are just bad policy badly implemented at the moment, not over-optimized.