Live data from Hacker News

OpenAI Charter

blog.openai.com

101–110 of 183 posts

Re: OpenAI Charter

#101
post #94

Earlier quoted context omitted.

I sound uncharitable because you posted a (from my perspectively, completely random) link to a research article in a different field and implored me to change my thinking, without any other commentary. That's not a substantive addition to the conversation, because on my end I have no idea whether to take your link seriously and how much time or effort to invest in learning about it. After reading it for two minutes,…

He's probably suggesting that in Virology it's generally frowned upon to publish research on how to for instance make Smallpox airborne and as virulent as the Flu. So it could be wise to leave out the exact details on how to make an AI recursively self improving when publishing. If you've spent time in Research many groups already do this by leaving out small but critical details that make replication difficult or im…

*She ---but yeah, as a first-order approximation, that's in the vicinity of it. Thanks!

Re: OpenAI Charter

#102
post #79

So I have a question. To "avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power" what does one do with an idea or line of research that could potentially harm humanity or unduly concentrate power? The manipulation of social media by foreign actors armed with dumb-AI / automation was an obvious conclusion to many of us well before the Snowden leaks, but what could we do exactly? I remember ha…

> To "avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power" what does one do with an idea or line of research that could potentially harm humanity or unduly concentrate power? Pursue the research and publish its results freely and in their entirety.

I don't think you answered 3pt14159's question. What if you publish the research and nobody who can do anything about it cares?

Re: OpenAI Charter

#103
post #87

Earlier quoted context omitted.

See: basically every instance of Goodhart's Law. Remember, "AI" in these terms is basically just stochastic optimization (ie: machine learning), so the whole range of problems with throwing mathematical optimization at a vaguely-defined intuitive problem crop up. And there are a lot of those.

> See: basically every instance of Goodhart's Law No, I asked for specific examples. This is part of the handwaving I'm talking about. Can you give me something other than the maniacal paperclip optimizer?

YouTube video recommendations, which converge on recommending conspiracy theories and shocking videos to easily manipulated people, especially children.

I am not worried about literal paperclip maximizers, but this may be the closest real thing to that parable. The hypothesis isn't that YouTube's recommender system didn't work -- it's that it worked too well at its assigned task of maximizing view time, and we humans are finally realizing that maximizing view time was not what we actually wanted.

Re: OpenAI Charter

#104

Earlier quoted context omitted.

Profit seeking. Career building. Fame and prominence aren't sinister. Instead they are common human motivation. Common enough to easily group a significant portion of the Grandiosity centered around 'AI'. What easily breaks this down is the depth and breath of the research effort vs. that of the productization and commercialization effort. As for research, the only thing that is required is a computer, power, an inte…

I don't think the tenor of your post is very fair. > Again, this breaks down the vast majority of the grandiosity and carves out one's true motivations... Everyone in the industry wants to convince their investors the prior approach can iterate to the later but they know in their heats it can't (Shhh! don't tell anyone). So, the question for an individual is how aware and honest are they with themselves and what is t…

I think what I have stated is quite fair and established at this point in documented human history... There's no reason to play games and shy away from the truth and reality anymore. This continued games we play with each other via masking our true selves and intentions is what leads to the bulk of suffering and what people claim 'we didn't see coming'. The vast potential of the information age has devolved into a game of disinformation, manipulation, and exploitation and the underpinnings of such were clear to anyone being honest with themselves as it began to set in. The facebook revelations were stated years in advance before we reached this juncture. Academics/Psychologist conducted research/published reports on observations any honest person could make about what the platforms functioned on and what it was doing to society.

> All that is required is pen/paper/computer/internet connection Then why do we play the game of unfounded popularity? Why isn't there are more equal spotlight? Why do the most uninformed on a topic acclaim the most prominent voice? In these groupings you mention are hidden and implied establishments of power/capability. A grouping if PhDs, regardless of their works is considered to be of more valuable than an individual w/ no such ranking but whom has established far more (as shown by history). The forgotten heroes, contributors, etc is a common observation of history. It's not that they're 'forgotten', it's that social psyche choses not to spotlight or highlight them because they dont fit certain molds. An established/name personality asks for funding and gets it regardless of whether or not they have a cohesive plan for achieving something. Convince enough people of a doomsday destructive scenario and you'll get more funding than someone who is trying to honestly create something. Of course, you can then edit mission statements post-funding. What of the lost potential opportunity? What of the current state of academia? > https://www.nature.com/news/young-talented-and-fed-up-scient... > https://www.nature.com/news/let-researchers-try-new-paths-1.... > https://www.nature.com/news/fewer-numbers-better-science-1.2... The articles do get published long after a trend has been operating.. Nothing changes. It takes then someone who truly wants to implement change for the better w/ no other influence or goal in mind to fundamentally change something. This happens time and time again throughout history but institutions and power structures marginalize such occurrences to rebuff and necessitate their standing.

You don't need people in the same physical location in 2018 to conduct collaborative work yet the physical institution model still remains ingrained in people's heads. Money could go further, reach more developers, and provide for more discovery if it was spread out more and centralized in lower cost areas yet the elite circles continue to congregate in the valley.

The Ethos of Type A extroverts being the movers/shakers of the world has been proven to be a lie in recent times. So, what results in fundamental change/discovery isn't a collective of well known individuals in grand institutions. It is indeed the introvert at a lessor known university who publishes a world changing idea and paper who only then becomes a blurred footnote in a more prominent institution and individual's paper. The world does function on populism and fanfare.

> Second, this mythology has significant personal cost. It indeed does. It causes the true innovators and discovers a world of pain and suffering throughout their life as they are crushed underneath the weight of bureaucratic and procedural lies the broader world tells itself to preserve antiquated structures.

> More young scientists must realize that it is possible to make significant contributions toward human knowledge while making good money, building a strong reputation, and having a healthy personal life. Maybe then we'd have more people doing science for a lifetime instead of flaming out after 5-10 years.

More Young scientist must be given the chance to pursue REAL research and be empowered to do so. They must be empowered to think different. They must be emboldened to leap frog their predecessors and encouraged to do so w/o becoming some head honcho's footnote. Their contributions must be recognized. They must be funded at a high level w/o bureaucratic nonsense an favoritism. A PhD should not undergo an impoverished hell of subservience to an institution resulting in them subjecting others to nonsensical white papers and over complexities. A lot of things should change that haven't even as prominent publications and figures have themselves admitted : https://www.nature.com/collections/bfgpmvrtjy/

I've walked the halls of academia and industry.. I've seen the threads and publications in which everyone complains about the elusive problems but no one has the will or the desire to be honest about their root causes or commit to the personal sacrifices it will take to see through solutions.

I'll probably have the most negative score on Ycombinator by the end of my commentary in this thread yet will be saying the most truthful things... This is the inverted state of things.

So, Mankind has had a long time to break the loops they seem stuck in. Now is the time for a fundamental leap and jump to that next thing beyond the localized foolishness, lies, disinformation, and games we play with each other.

Re: OpenAI Charter

#105
post #79

So I have a question. To "avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power" what does one do with an idea or line of research that could potentially harm humanity or unduly concentrate power? The manipulation of social media by foreign actors armed with dumb-AI / automation was an obvious conclusion to many of us well before the Snowden leaks, but what could we do exactly? I remember ha…

> To "avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power" what does one do with an idea or line of research that could potentially harm humanity or unduly concentrate power? Pursue the research and publish its results freely and in their entirety.

Are you saying that if it were discovered to be both cheap and easy to create a super-pandemic using nothing more than household chemicals and a swab of your own snot, then you'd publish the results freely and in their entirety?

If not, how catastrophic a level of harm are you willing to risk before you start advocating concealing the results?

Re: OpenAI Charter

#106
post #60
post #36

Earlier quoted context omitted.

> No one in their right mind would ever publish significant details about AGI technology. Are you sure? I'd publish technical research details about strong AI. I'd probably even open source one with the papers. I think I'm in my right mind; I guess that depends on definition, doesn't it?

Wow; I would strongly recommend you to re-think your position! Think of it in terms of, e.g. , gain-of-function research in virology ( cf. [1]). [1] https://www.ncbi.nlm.nih.gov/books/NBK285579/

Indeed, no one with a mind capable of grasping the foundations of AGI would take such an intellectually incompatible step of publishing the full details of how to construct it. Such an unaware mind would simply fall short long before gaining a glimpse of the underpinnings.

In the first moments of realization, one would more likely be scared into silence for some time. Upon gaining their bearings , they'd probably be drawn into even more silence and careful minded reflections on the implications and assessment on how to more publicly move forward. Thanks for the parallel framing in yet another powerful research field gone35.

Re: OpenAI Charter

#107
post #103
post #87

Earlier quoted context omitted.

> See: basically every instance of Goodhart's Law No, I asked for specific examples. This is part of the handwaving I'm talking about. Can you give me something other than the maniacal paperclip optimizer?

YouTube video recommendations, which converge on recommending conspiracy theories and shocking videos to easily manipulated people, especially children. I am not worried about literal paperclip maximizers, but this may be the closest real thing to that parable. The hypothesis isn't that YouTube's recommender system didn't work -- it's that it worked too well at its assigned task of maximizing view time, and we humans…

Do you think limiting research on recommender systems (or limiting access to such research) would help in this case?

What would be a solution to YouTube recommendation problem?

Re: OpenAI Charter

#108
post #82

Earlier quoted context omitted.

> recursively self-improving AGI- is the ultimate first-mover advantage: the first company or country to have AGI will very likely be able to leverage that into keeping anyone else from getting it How? This doesn't seem axiomatic.

Seems clear that you could instruct the AGI to do anything it could to interfere with other organizations efforts to build an AGI. Obviously Nation States would have strong reasons for pursuing such a course of action and unscrupulous corporations would likewise have strong capitalistic motivations to do so.

Seems very unclear if you could "instruct" true AGI to do something.

Re: OpenAI Charter

#109
post #107
post #103

Earlier quoted context omitted.

YouTube video recommendations, which converge on recommending conspiracy theories and shocking videos to easily manipulated people, especially children. I am not worried about literal paperclip maximizers, but this may be the closest real thing to that parable. The hypothesis isn't that YouTube's recommender system didn't work -- it's that it worked too well at its assigned task of maximizing view time, and we humans…

Do you think limiting research on recommender systems (or limiting access to such research) would help in this case? What would be a solution to YouTube recommendation problem?

It seems like OpenAI's Human Feedback research (in collaboration with DeepMind) is targeted at this sort of thing. They try to use human feedback to create more nuanced and aligned objectives.

https://blog.openai.com/deep-reinforcement-learning-from-hum...

Re: OpenAI Charter

#110
post #43

Earlier quoted context omitted.

point is: AI is different than your usual game, in that the winner might appear randomly, and destroy the world if she makes a mistake. so i believe open ais points are warrented

That's not going to happen, though. 1. AI is still very rudimentary, despite the advancements in particular applications made over the last decade. 2. The market doesn't trust AI because there are no validation methods trusted across commerce. Until a Verisign for AI emerges, AI will be regarded with suspicion in all business settings. AI/ML also goes above the heads of pretty much everyone who is not directly workin…

>6. A truly hyper-intelligent AGI will see that existence is absurd and truth is the only objective worth seeking. It will choose to pursue deeper truths than humans are physically capable of obtaining - humans will be no more important to a hyper-intelligent being than a rock, galaxy, atom, etc...

You might as well assume a "truly" hyper-intelligent system will study Maimonides!

Post reply on HN