Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft is investing $1B in OpenAI

openai.com

Post reply on HN
Print view View original

441–450 of 623 posts

Re: Microsoft is investing $1B in OpenAI

#441

Post by idlewords »

marvin wrote: post #410
DuskStar wrote: post #389

Earlier quoted context omitted.

> No one is anywhere close to achieving true AGI No one knows how far off true AGI is, just like no one in 1940 (or 1910) knew how far off fission weapons were. EDIT: I quite liked this article from a few years back [0], and the fission weapon prediction example is stolen from there. 0: https://intelligence.org/2017/10/13/fire-alarm/

The Manhattan Project is a very apt analogy. Even if you believe that AGI is impossible, it should be possible to appreciate that many billions would quickly be invested in its development if somehow a viable pathway to it became clear. Even if just to a few well-connected experts. This is what happened when it became known nuclear weapons were a viable concept. The technology shifted power to such an extreme degree…

The Manhattan project happened when the entire conceptual road map to fission weapons was understood. This is manifestly not the case with AI, which can be charitably described as "add computers until magic".

Re: Microsoft is investing $1B in OpenAI

#442

Post by gonehome »

dragonwriter wrote: post #423
gonehome wrote: post #413

Earlier quoted context omitted.

The difference is in the scale of the intelligence, not just the technology. It's not so much a new human like intelligence that runs on silicon, it's a general problem solving intelligence that can run a billion times faster than any individual human. This is the part I think you're underestimating. If you have that without the ability to align its goals to human goals then that's a problem.

> The difference is in the scale of the intelligence, not just the technology. AGI is inherently no greater in scale than human intelligene, so scale is not a difference with AGI, though it might be with AG super I. But that's a different issue than mere AGI, and may be impossible or impractical even if AGI is doable; we have examples of human-level intelligence so we know it is physically acheivable in our universe,…

I think that's somewhat of an arbitrary distinction likely not to exist in practice.

If you have an AGI you can probably scale up its runtime by throwing more hardware at it. Maybe there's some reason that'll prevent this from being true, but I'm not sure that should be considered the default or most likely case.

Biology is limited in ways that AGI would not be due to things like power and headsize constraints (along with all other things that are necessary for living as a biological animal). Human intelligence is more likely to be a local maxima driven by these constraints than the upper bound on all possible intelligence.

Re: Microsoft is investing $1B in OpenAI

#443

Post by idlewords »

umvi wrote: post #414
DuskStar wrote: post #389

Earlier quoted context omitted.

> No one is anywhere close to achieving true AGI No one knows how far off true AGI is, just like no one in 1940 (or 1910) knew how far off fission weapons were. EDIT: I quite liked this article from a few years back [0], and the fission weapon prediction example is stolen from there. 0: https://intelligence.org/2017/10/13/fire-alarm/

Really? I thought by 1940 physicists generally understood fission and theoretically understood how to build a bomb - they just needed to find enough distilled fissile material (which was hard to do). And indeed, once they had enough U235, they had such a high degree of confidence in the theory, that they built a functioning U235 bomb without ever having previously tested one.

They did understand it theoretically. This is the key flaw in any analogy between AI risk and nuclear weapons.

Re: Microsoft is investing $1B in OpenAI

#444

Post by jacques_chester »

gonehome wrote: post #283
apta wrote: post #271

Earlier quoted context omitted.

> If AGI is achievable (seems likely given brains are all over the place in nature) I don't see how that conclusion follows the antecedent.

Brains aren't magical, if the laws of nature allow for them to exist in nature and we see generalized intelligence develop and get selected for repeatedly then that suggests it can be done - it's just a matter of knowing how.

Our struggle to understand the brain suggests that "just a matter of knowing how" might take a while.

Re: Microsoft is investing $1B in OpenAI

#445

Post by pron »

stcredzero wrote: post #397
pron wrote: post #359

Earlier quoted context omitted.

> are you effectively conceding that we're close to AGI at insect levels? By "we are not even close to achieving insect-level intelligence" I think I meant that what we have now is not close in intelligence (whatever that means) to insects. I don't know if we have a 50% of getting there in a decade, but I certainly wouldn't conclusively say that "we are not even close" to that. I mostly regret having chosen bikes rat…

By "we are not even close to achieving insect-level intelligence" I think I meant that what we have now is not close in intelligence (whatever that means) to insects. Some insects are pretty stupid! Fleas and ticks have a good and highly adapted repertoire of behaviors, but for the most part, as far as we know, most individual behaviors are fairly simple. I mostly regret having chosen bikes rather than electric scoot…

I think my point was lost because it's my pre-Primetime Emmy material.

Re: Microsoft is investing $1B in OpenAI

#446

Post by Alas1 »

Veedrac wrote: post #428
Alas1 wrote: post #289

Earlier quoted context omitted.

What exactly does that 96% mean, though? It means that on some fixed dataset you're achieving 96% accuracy. I'm baffled by this stupidity of claiming results (even high-profile researchers do this) based on datasets with models that are nowhere near as robust as the actual intelligence that we take as reference: humans. Take the model that makes you think "sentiment analysis is at 96%", come up with your own examples…

> I think continual Turing testing is the only way of concluding whether an agent exhibits intelligence or not. So you think it's impossible to ever determine that a chimpanzee, or even a feral child, exhibits intelligence? This seems rather defeatist.

No, interpreting "continual" the way you did would mean I should believe that we can't conclude our friends to be intelligent either (I don't believe that). Maybe I should've said "prolonged" rather than "continual".

Let me elaborate on my previous point with an example. If you look at the recent works in machine translation, you can see that the commonly used evaluation metric of BLEU is being improved upon at least every few months. What I argue is that it's stupid to look at this trend and conclude that soon we will reach human performance in machine translation. Even when comparing against the translation quality of humans (judged again by BLEU on a fixed evaluation set) and showing that we can achieve higher BLEU than humans is not enough evidence. Because you also have Google Translate (let's say it represents the state-of-the-art), and you can easily get it to make mistakes that humans would never do. I consider our prolonged interaction with Google Translate to be a narrow Turing test that we continually apply to it. A major issue in research is that, at least in supervised learning, we're evaluating on datasets that are not different enough from the training sets.

Another subtle point is that we have strong priors about the intelligence of biological beings. I don't feel the need to Turing test every single human I meet to determine whether they are intelligent, it's a safe bet at this point to just assume that they are. The output of a machine learning algorithm, on the other hand, is wildly unstable with respect to its input, and we have no solid evidence to assume that it exhibits consistent intelligent behavior and often it is easy to show that it doesn't.

I don't believe that research in AI is worthless, but I think it's not wise to keep digging in the same direction that we've been moving in for the past few years. With deep learning, while accuracies and metrics are pushed further than before, I don't think we're significantly closer to general, human-like AI. In fact, I personally consider only AlphaZero to be an unambiguous win for this era of AI research, and it's not even clear whether it should be called AI or not.

Re: Microsoft is investing $1B in OpenAI

#447

Post by p1esk »

breck wrote: post #222
pron wrote: post #220

Earlier quoted context omitted.

> Yes, we do. Lots of data, lots of training, better algorithms, more understanding of the brain.. So to build AI all that remains is to understand how it could work. > but it's pretty clear what we need to do It isn't (unless by "clear" you mean as clear as in your statement above). I've been following some of the more theoretical papers in the field, and we're barely even at the theory forming stage. > but it's pre…

> I've been following some of the more theoretical papers in the field, and we're barely even at the theory forming stage. I read those papers too. And I write code and train models day in and day out. I could get very specific on what needs to be done, but that's what we do at our job. If you're curious, I'd say join the field. I agree with you in that I don't think for a second anyone can make an accurate predictio…

I’ve been doing research in DL field for the last 6 years (just presented my last paper at IJCNN last week), and I can say with confidence we have no clue how to get to AGI. We don’t even know how DL works on the fundamental level. More importantly, we don’t know how the brain works. So I agree with pron that your “relatively soon” is just as likely to be 10 as 100 years from now.

Re: Microsoft is investing $1B in OpenAI

#448

Post by TaylorAlexander »

dkersten wrote: post #82

> We think its impact should be to give everyone economic freedom to pursue what they find most fulfilling, creating new opportunities for all of our lives that are unimaginable today. The cynic in me thinks this will never happen, that instead it will make a small subset of the population super rich, while the rest are put to work somewhere to make them even more money. Microsoft will ultimately want a return on the…

Well, the super rich getting richer is the status quo, so I kind of feel like nothing much changes if this never happens. Now, riding that happy PR wave and failing to deliver would be lame, but perhaps they really believe this. I think it will depend entirely on how much really gets open sourced in the end. I want to believe they’ll really do it.

Re: Microsoft is investing $1B in OpenAI

#449

Post by marvin »

idlewords wrote: post #441
marvin wrote: post #410

Earlier quoted context omitted.

The Manhattan Project is a very apt analogy. Even if you believe that AGI is impossible, it should be possible to appreciate that many billions would quickly be invested in its development if somehow a viable pathway to it became clear. Even if just to a few well-connected experts. This is what happened when it became known nuclear weapons were a viable concept. The technology shifted power to such an extreme degree…

The Manhattan project happened when the entire conceptual road map to fission weapons was understood. This is manifestly not the case with AI, which can be charitably described as "add computers until magic".

I didn’t compare OpenAI to the Manhattan Project. I was pointing out that if a small number of people discover a plausible conceptual pathway to AGI, a similar project will happen.

Re: Microsoft is investing $1B in OpenAI

#450

Post by applecrazy »

chrshawkes wrote: post #425
applecrazy wrote: post #51

Earlier quoted context omitted.

The research that OpenAI’s doing is groundbreaking and the results are often beyond state-of-the-art. I aim to work in one of your research teams sometime!

Watch the Kool-Aid intake and you'll be just fine. Dreams are great and an absolute necessity for success but create your own. Don't buy into everything you hear, especially Elon Musk talking about Artificial General Intelligence.

Oh, I'm well aware of the hype around AGI. My personal view is that AGI is kind of an asymptotic goal, something we'll get kind of close to but never actually reach. Nevertheless, I would like to work on more pragmatic goals, like improving the current state-of-the-art language models and text generation networks. I'm actually starting by reimplementing Seq2Seq as described by Quoc Le et al.[1] for text summarization[2] (this code is extremely messy but it'll get better soon). It's been interesting to learn about word embeddings, RNNs and LSTMs, and data processing within the field of Natural Language Processing. Any tips on how to get up to speed within this field would be helpful, as I'm trying to get into research labs doing similar work at my university.

[1]: https://papers.nips.cc/paper/5346-sequence-to-sequence-learn... [2]: https://github.com/applecrazy/reportik/

Post reply on HN