Live data from Hacker News

Algorithms Have Already Gone Rogue

wired.com

141–150 of 165 posts

Re: Algorithms Have Already Gone Rogue

#141
post #48

My only objection to this is a semantic one -- the word "algorithm" is not being well-served here. The correct word for this sort of thing is "heuristic". The concern isn't that algorithms themselves are incorrect, the concern is that the problem they are trying to solve is a heuristic one, not a formal one. Saying "let's write an algorithm to improve search results" is meaningless; "let's design and implement a heur…

Algorithms running on "social hardware" can be surprisingly formal. A famously well-documented example are early modern witchhunts. The humorous depiction in Monty Python and the Holy Grail does a surprisingly good job at conveying the algorithmic nature.

Along those lines, here are some of my comments on this general topic from an email I posted to the Doug Engelbart Unfinished Revolution II Colloquium in 2000: http://www.dougengelbart.org/colloquium/forum/discussion/012...

===

... I personally think machine evolution is unstoppable, and the best hope for humanity is the noble cowardice of creating refugia and trying, like the duckweed, to create human (and other) life faster than other forces can destroy it. [Although in 2017 I'd add other possibilities like symbiosis or trying to create friendlier AI as a partner (or at least AIs with a sense of humor -- see James. P. Hogan's AIs, or the ones like Libbry in EarthCent Ambassador series, or the Old Guy Cybertank series example), improved sensemaking through better intelligence-augmenting tools, and trying to help human society be more compassionate in the hopes our path out of a singularity will somehow reflect our path going in...]

Note, I'm not saying machine evolution won't have a human component -- in that sense, a corporation or any bureaucracy is already a separate machine intelligence, just not a very smart or resilient one. This sense of the corporation comes out of Langdon Winner's book "Autonomous Technology: Technics out of control as a theme in political thought".

You may have a tough time believing this, but Winner makes a convincing case. He suggests that all successful organizations "reverse-adapt" their goals and their environment to ensure their continued survival. These corporate machine intelligences are already driving for better machine intelligences -- faster, more efficient, cheaper, and more resilient.

People forget that corporate charters used to be routinely revoked for behavior outside the immediate public good, and that corporations were not considered persons until around 1886 (that decision perhaps being the first major example of a machine using the political/social process of its own ends).

Corporate charters are granted supposedly because society believe it is in the best interest of society for corporations to exist. But, when was the last time people were able to pull the "charter" plug on a corporation not acting in the public interest? It's hard, and it will get harder when corporations don't need people to run themselves.

I'm not saying the people in corporations are evil -- just that they often have very limited choices of actions. If a corporate CEOs do not deliver short term profits they are removed, no matter what they were trying to do. Obviously there are exceptions for a while -- William C. Norris of Control Data was one of them, but in general, the exception proves the rule. Fortunately though, even in the worst machines (like in WWII Germany) there were individuals who did what they could to make them more humane ("Schindler's List" being an example).

Look at how much William C. Norris of Control Data got ridiculed in the 1970s for suggesting the then radical notion that "business exists to meet society's unmet needs". Yet his pioneering efforts in education, employee assistance plans, on-site daycare, urban renewal, and socially-responsible investing are in part what made Minneapolis/St.Paul the great area it is today. Such efforts are now being duplicated to an extent by other companies. Even the company that squashed CDC in the mid 1980s (IBM) has adopted some of those policies and directions. So corporations can adapt when they feel the need.

Obviously, corporations are not all powerful. The world still has some individuals who have wealth to equal major corporations. There are several governments that are as powerful or more so than major corporations. Individuals in corporations can make persuasive pitches about their future directions, and individuals with controlling shares may be able to influence what a corporation does (as far as the market allows).

In the long run, many corporations are trying to coexist with people to the extent they need to. But it is not clear what corporations (especially large ones) will do as we approach this singularity -- where AIs and robots are cheaper to employ than people. Today's corporation, like any intelligent machine, is more than the sum of its parts (equipment, goodwill, IP, cash, credit, and people). It's "plug" is not easy to pull, and it can't be easily controlled against its short term interests.

What sort of laws and rules will be needed then? If the threat of corporate charter revocation is still possible by governments and collaborations of individuals, in what new directions will corporations have to be prodded? What should a "smart" corporation do if it sees this coming? (Hopefully adapt to be nicer more quickly. :-) What can individuals and governments do to ensure corporations "help meet society's unmet needs"?

Evolution can be made to work in positive ways, by selective breeding, the same way we got so many breeds of dogs and cats. How can we intentionally breed "nice" corporations that are symbiotic with the humans that inhabit them? To what extent is this happening already as talented individuals leave various dysfunctional, misguided, or rogue corporations (or act as "whistle blowers")? I don't say here the individual directs the corporation against its short term interest. I say that individuals affect the selective survival rates of corporations with various goals (and thus corporate evolution) by where they choose to work, what they do there, and how they interact with groups that monitor corporations. To that extent, individuals have some limited control over corporations even when they are not shareholders. Someday, thousands of years from now, corporations may finally have been bred to take the long term view and play an "infinite game".

However, if preparations fail, and if we otherwise cannot preserve our humanity as is (physicality and all), we must at least adapt with grace whatever of our best values we can preserve or somehow embody in future systems. So, an OHS/DKR [Open Hyperdocument System / Dynamic Knowledge Repository] to that end (determining our best values, and strategies to preserve them) would be of value as well.

When aluminum was first discovered around 1827, and for decades afterward, it was worth more than platinum, and now just under two centuries later we throw it away. In perhaps only two decades from now, children may play "marbles" using diamonds, and a child won't bother to pick a diamond up from the street unless it is exceptionally pretty (although you or I probably would out of habit -- "see a diamond, pick it up, and all the day you have good luck").

This long essay is my own current perspective on this developing situation, and part of the process of my formulating my own thinking on these trends and how I as an individual will respond to them.

To conclude, I think all the "classical" problems like food, energy, water, education, and materials will be technically solvable by 2050 even if we don't do much specifically about them (and like hunger are solved today except for politics). The dynamics of technology and economics are just taking us there whether we like it or not. Those goods may all may essentially be "free" or "extremely cheap" by 2050. Obviously the complex politics of these issues need to be resolved, and the solutions need to be actually implemented. If they are "extremely cheap", people still need a tiny amount of income to buy them.

Still, I think Doug [Engelbart] is right. We face huge problems that only collaborative efforts can solve -- especially the problems of intelligent machines, technology-amplified conflict, and a complete disruption of our scarcity-based materialistic economic and social systems. These problems dwarf technical issues like energy, food, goods, education, and water.

The problem has always been, and will always be, "survival with style" (to amplify Jerry Pournelle). The next twenty years will fundamentally change what the survival issues are: environment, threats, and allies. They will also very well change what we value as "style" -- when diamonds are cheap as glass [perhaps from nanotechnology], what will one give to impress?

===

Re: Algorithms Have Already Gone Rogue

#142
post #124

The financial system is a giant message passing algorithm. It is pretty much just a min-sum algorithm [1] whose sole purpose is to answer the question "what should we do?". Anyone who has played around with these algorithms for solving decoding problems knows that they are fabulously powerful. But these message passing algorithms have two weaknesses: (A) When there is more than one solution (B) When there are small l…

> The financial system is a giant message passing algorithm. It is pretty much just a min-sum algorithm [1] whose sole purpose is to answer the question "what should we do?". no, no no. To even propose that the answer to the question of "what should we do?" can be solved by the financial system is laughable; that is pure free-market absolutism. The anwer to the question "what should we do?" is _political_ . Let us no…

I'm afraid you read "what should we do" in much more general sense than the original author. I think that algorithm answers a narrower question of "what should we do to optimize monetary resource allocation", and it only answers within the boundaries you set, for instance, trust and reputation play major roles.

Re: Algorithms Have Already Gone Rogue

#143
post #131
post #124

Earlier quoted context omitted.

> The financial system is a giant message passing algorithm. It is pretty much just a min-sum algorithm [1] whose sole purpose is to answer the question "what should we do?". no, no no. To even propose that the answer to the question of "what should we do?" can be solved by the financial system is laughable; that is pure free-market absolutism. The anwer to the question "what should we do?" is _political_ . Let us no…

Downvoters, explain your logic please

I assume you're being downvoted because you seem to have misread the author's statement and created a strawman argument. He posited that the financial system's algorithm's purpose is to answer, "what should we do?". You seem to have interpreted that as saying that the financial system should answer that problem for society as a whole.

Re: Algorithms Have Already Gone Rogue

#144

The financial system is a giant message passing algorithm. It is pretty much just a min-sum algorithm [1] whose sole purpose is to answer the question "what should we do?". Anyone who has played around with these algorithms for solving decoding problems knows that they are fabulously powerful. But these message passing algorithms have two weaknesses: (A) When there is more than one solution (B) When there are small l…

Great analysis. (A) When there is more than one solution And this is exactly why the concept of a "growing pie" is flawed. Markets generally pool around a single solution versus averaging around multiple and there isn't enough capital (labor or dollars). I think it comes back to power law driving behaviors here and everyone wanting a huge win. So in effect markets look like fixed pies in the short term and the winner…

> Trouble with this scheme is, the "pie growth", when it happens, is distributed to a very small group of people who have the ability to make big bets - so it compounds.

It seems like the real trouble is that we've set things up so that "big bets" are required.

Huge tomes of regulations that have one-time compliance costs, so the cost to a tiny shop is the same as it is to Comcast or Microsoft, and regulatory capture to keep it that way.

Tax laws that encourage profits to be kept within the corporation instead of returned to investors, which requires successful companies to become conglomerates.

Laws that allow Hollywood and Apple to control content distribution and reject anything from anyone who poses a competitive threat to them.

Fix things like that and you won't have to be so big to grow the pie.

Re: Algorithms Have Already Gone Rogue

#145
post #118

The financial system is a giant message passing algorithm. It is pretty much just a min-sum algorithm [1] whose sole purpose is to answer the question "what should we do?". Anyone who has played around with these algorithms for solving decoding problems knows that they are fabulously powerful. But these message passing algorithms have two weaknesses: (A) When there is more than one solution (B) When there are small l…

"(B) When there are small loops in the message network" Can you elaborate a real-world example of this for the layperson ? 50k foot view is fine :)

A real-world example of such dysfunction happening right now is that most of the money that could be used by all people to signal demand via cash "messages" is now tied up in a small loop of messages sent between the wealthiest people a "Casino Economy". See for example "Money as Debt II": https://www.youtube.com/watch?v=6MwHgpFSQMo

Money can be seen as a form of kanban unit or ration token for signalling demand. Essentially, the richest 1% or less of investors now use their "messaging" tokens (cash) for speculative investments in games against other wealthy investors in the financial sector (foreign exchange, derivatives market, etc.). That starves the rest of the economy for kanban tokens (cash) so it can't function. It would be like you walked into a Toyota factory using Kanban tokens and randomly removed 90% of the tokens -- that would prevent each industrial operation from signalling its need for required parts from other operations, causing all operations to slow down as they wait on all their dependencies to arrive. https://en.wikipedia.org/wiki/Kanban

Almost any economist will agree that if 90% of the money supply suddenly disappeared we would suffer a great economic depression in the USA. But the same economists seem unable to accept the same depression will happen for the 99% if the richest 1% take most of the money supply out of general circulation and just use it to play poker with each other. There are other complexities (including velocity of money message passing), but it seems to me the big issue many people overlook -- it is not just the total amount of money supply but how it is distributed.

Unfortunately, most of the governmental solutions (to satisfy wealthy donors taking part in legalized bribery of campaign donations) are based on supply-side "voodoo economics", like giving trillions of more dollars to the wealthy via bank loans or tax cuts. This is done in a foolish unfounded hope the wealthy will use extra money differently than they already have in the casino economy disconnected from meeting the needs of the 99%.

Even the slightest amount of thought will show how absurd supply-side economics is compared to demand-side economics. Almost anyone who can show a predictable demand for a good or service (like booked orders) can get a bank loan to fulfill those orders -- and to get orders, the customers need to have cash to signal demand. It is demand that makes businesses successful -- not supply.

Markets can work well to meet needs and wants, but they only hear the needs and wants of people who have money. Thus the value of a basic income to ensure all people's needs and wants are heard by the market to at least a basic extent.

Other options for dealing with the cash crisis caused by the triumph of the Casino Economy include strengthening non-market parts of the overall society such as: subsistence production (home 3D printing, home robotics, home gardening, home power via solar panels); the gift economy with more volunteering, freecycling, and sharing knowledge via the internet; and better democratic planning.

Unfortunately, with the big move of women into the workforce in the USA over the past few decades, home production, volunteering, and civic participation have all been reduced. Expanded entertainment options as a form of "Supernormal Stimuli" also distracted many people from physical daily life, also reducing participation in those other three sectors of society. Thus, a growing percentage of total societal interactions take place via exchange in the market instead of via subsistence, gift exchange, or civic planning.

Ironically, the "Two Income Trap" means families have very little to show for the second income between extra expenses involved in working outside the home, two-income families bidding up the price of houses and other items, and an increased supply of workers leading to lower compensation and poorer working conditions for everyone. See Elizabeth Warren's book on that: http://www.motherjones.com/politics/2004/11/two-income-trap/

With an increased supply of workers, there was decreased power of workers to demand wage increases in parallel with ongoing productivity increases. This in turn created the situation whether the owners of capital could take profits and lend them to workers instead of paying them in wages. Richard Wolff talks about in "Capitalism Hits The Fan", (whatever one thinks about his proposals for reform): http://issuepedia.org/Capitalism_Hits_the_Fan

To be clear, I'm not saying women should not have a choice as to what they do with their time. This is just about the societal implications of certain trends given men did not leave the work force to stay at home and be subsistence producers, volunteers, and civic actors in the same numbers as women who joined the work force.

I'm also not saying these "supernormal stimuli" are all bad (see for pros and cons: http://www.paulgraham.com/addiction.html ).

How we deal with that situation is a political question -- but to deal with it, we first need to acknowledge and understand what happened. And beyond a decrease in activity in the non-market sectors of society, one of the consequences of multiple trends has been a concentration of wealth in a smaller number of hands which has made the shift to a Casino Economy more likely.

Automation also has a role to play in that concentration of wealth from a different direction. Marshall Brain talks about that in "Robotic Freedom": http://marshallbrain.com/robotic-freedom.htm

Better technology has also increased options for participation in those three non-market areas (via cheaper tools, cheaper robots, and cheaper communications), so it is hard to say the entire trend has been downward for those non-market sectors. We may yet see them rise again as those other costs continue to fall -- and perhaps if people learn to move beyond the supernormal stimuli of mass-produced entertainment and back to making their own fun and using their own creativity.

While this won't directly break the tight loop of the 1% and the Casino Economy, it may bypass it so it does not matter as much -- in which case all that vast amount of money controlled by the 1% may just becomes like Monopoly money -- meaningless to most people most of the time because their life is built around non-market interactions.

Re: Algorithms Have Already Gone Rogue

#146

Earlier quoted context omitted.

> "rank the documents by the total number of occurrences of each search term in that document" I fail to see how this is not an algorithm. The heuristic (rank search results from most to least relevant) is backed by an algorithm (find occurrence of word, sort document based on occurrences). I like to approximate the two by thinking of heuristics as an approach to solving a given problem while algorithms are actions t…

There's an algorithm to rank the documents by an arbitrary metric. And there's an algorithm to calculate the number of occurrences of each search term in that document. However, those are insignificant implementation details - all the logic (and all the good and bad results) comes from the arbitrary decision to use the number of occurrences as meaningful for measuring the relevance, from the choice of heuristic.

That's just saying it's the wrong algorithm, not that it isn't an algorithm. Every computation of pi has truncated the result, which is a 'heuristic' decision, but that doesn't invalidate the fact that pi is computed using an algorithm.

All algorithms approximate things, after all. That's simply a consequence of abstraction.

Re: Algorithms Have Already Gone Rogue

#147
post #110
post #19

Earlier quoted context omitted.

Yes? This is an interview with an author that was conducted specifically to talk about the book and the material in it. What did you expect ? Unrelated, I'd like to see your argument against that particular line. IMO the comparison is an excellent one; its only issue is that the chosen scope ("the financial market") is too small. Corporations and other bureaucratic entities like governments are powerful cross-domain…

Calling financial markets AI is wrong in pretty much the only way a word can be "wrong": It's not what most people mean when they talk about AI. This makes it really easy to make statements that sound deep and meaningful, but really aren't. E.g. "I'm not worried about Artificial Intelligence - we already have artificial intelligence, its called a company. Companies are artificial, and they behave intelligently". This…

If I said "algebra is geometry", you could make the same criticism; it's wrong, because that's not what most people mean by those terms.

But then you see that almost all of the most important mathematical developments in human history make that exact conflation, and that it is all driven by the need to describe human experience in symbolic language... and it does look very deep.

Nobody cares what 'most people' mean. Most people are idiots. Sure, you wouldn't talk like that when you're writing a dictionary, but dictionary writers are not known for making intellectual breakthroughs. That's the difference between description thinking and proscriptive. One allows for creativity.

Re: Algorithms Have Already Gone Rogue

#148
"I’m not sure that Jeff would make a great president, but he might."

So let me tell you about my experience with Amazon Fulfilment. I was gonna pay Amazon, who has this huge expertise in packing and shipping stuff, to fulfil my Kickstarter. I'd made a large, delicate art book. Amazon, in their infinite wisdom, stuffed them in bubble wrap envelopes and dropped them in the mail. They were getting bent, the envelopes were getting ripped up, it was a mess.

I spent a month in customer service hell e-mailing someone who was following a script that said I would have to turn on something they called "prep", which would ask them to look at it and package it better. Three times I checked that this was set, and three times I sent myself a test book that came in a bubble mailer.

Finally I got clued in that there is a high-level support team that you access by sending a complaint directly to Bezos. This person, after some back and forth, ultimately informed me that due to their internal systems, it is completely impossible for them to put a book in anything but this level of shitty packaging; "prep" is just not a process that can ever happen to a book.

They covered shipping my books out to someone actually capable of finding sensible boxes and shipping them. And they intimated that someone maybe lost a job over this. But changing this, they said, might take on the order of years.

I'm not sure I want a man who presides over a system like this running the country.

Especially given that O'Reilly points out that the "rogue algorithms" of the title are corporations, and the only reason Amazon is headquartered in WA is that the tax was lowest there...

Re: Algorithms Have Already Gone Rogue

#149
> when the curtain rolls back we see that those superpowers have consequences: Those algorithms have bias built in.

> That’s absolutely right. But I’m optimistic because we’re having a conversation about biased algorithms. We had plenty of bias before but we couldn’t see it.

I must say I am really happy to see that bias in tech is being recognized and accounted for (for the most part).

Please forgive the politics (I'll try really hard not to bash Trump ;-) ), but if there is a silver lining to the 2016 US presidential election I think it is that it has really caused many of us to introspect and realize how thick our echo chamber walls have really gotten over the last few years. The chamber was constructed so quickly, I barely realized it was happening. We're becoming so polarized that we're actually moving to different communities to be with more people "like us." Simple awareness of the problem is a huge step forward in being able to resolve it.

Re: Algorithms Have Already Gone Rogue

#150

My only objection to this is a semantic one -- the word "algorithm" is not being well-served here. The correct word for this sort of thing is "heuristic". The concern isn't that algorithms themselves are incorrect, the concern is that the problem they are trying to solve is a heuristic one, not a formal one. Saying "let's write an algorithm to improve search results" is meaningless; "let's design and implement a heur…

Yes, a lot of people use "algorithm" to mean simply "procedure". I'm glad to see someone besides me pointing out the distinction between algorithms, in the strict sense, and heuristics. Complaining about the misuse of technical terms is unlikely to have impact on usage in the popular press ([0]), but I think it's appropriate in a technical discussion. One of the most egregious misuses of "algorithm", in my opinion, i…

Genetic algorithms are algorithms in that they are a description of a specific process or set of processes (which may used in heuristics, or studied in isolation) at an implementation level of abstraction. "Genetic heuristics" suggests heuristic applications, rather than algorithms in isolation, and "genetic search" suggests a specific application, but a "genetic algorithm" can be extremely simple and isn't fundamentally different than an algorithm like quicksort. Either could be used as part of some heuristic, or not.
Post reply on HN