51–60 of 128 posts
Earlier quoted context omitted.
They smell the urgency in the air, and they're close enough to the center to get a good and accurate whiff
How on earth would you know if a whiff was accurate, when we're talking about something which has never before been created? I think even if you have intuitions about an approach, and have promising results, if you're trying to arrive at something new, it's really hard to know how far away you are.
The cynical-but-likely-accurate take is that researching language modeling has a higher ROI and lower risk than researching robotics.
Earlier quoted context omitted.
I am a researcher on the AI/Systems side and I wanted to chime in. Transformers are amazing for language, and have broken all the SOTA is many areas (at the start of the year, some people may have wondered if CNNs are dead [they are not as I see it]). The issue with Transformer models is the insane amount of data they need. There is some amazing progress on using unsupervised methods to help, but that just saves you…
Great comment thank you. Yes unless the industry sees value in a step change in the scale on offer to regular devs, progress on massive nets will be slow. Hooking things together is pretty much my job. I have used ROS extensively in the past but now I just hook things together using python. But I consider what Tesla is doing to be pretty promising, and they are layering neural nets together where the output of three…
Thx for the pointers on Tesla. Had not seen the Hydranet stuff. There was a Karpathy talk about 2 weeks back at a CVPR workshop .. he revealed the scale of Tesla's current generation deep learning cluster [1]. It is insane! Despite being in industrial research, I don't foresee ever being able to touch a cluster like that.
A lot of our current research involves end-to-end training (some complex stuff with transformers and other networks stitched together). There was a CVPR tutorial on autonomous driving [2], where they pretty much said autonomy 2.0 is all about end-to-end. I've spoken to a few people who actually do commercial autonomy, and they seemed more skeptical on whether end2end is the answer in the near-term.
One idea we toy with is to use existing frozen architectures (OpenAI releases some and so do other big players) and do a small bit of fine-tuning.
[1] https://www.youtube.com/watch?v=NSDTZQdo6H8 [2] https://www.self-driving-cars.org/
Is the prevailing opinion that progress in reinforcement learning is dependent on algorithmic advances, as opposed to simply scaling existing algorithms? If that is the case, I could see this decision as an acknowledgement that they are not well positioned to push the frontier of reinforcement learning - at least not compared to any other academic or industry lab. Where they have seen success, and the direction it se…
Reinforcement learning itself is a dead-end on a road to AI. They seem to slowly starting to realize it, probably ahead of academia.
Earlier quoted context omitted.
Why do you believe this to be the case?
In a nutshell it’s too wasteful in energy spent and it doesn’t even try to mimic natural cognition. As physicists say about theories hopelessly detached from reality - “it’s not even wrong”. The achievements of RL are so dramatically oversold that it can probably be called the new snake oil.
The cynical-but-likely-accurate take is that researching language modeling has a higher ROI and lower risk than researching robotics.
Wojciech stated this pretty explicitly on his Gradient Dissent podcast a few months back.
The cynical-but-likely-accurate take is that researching language modeling has a higher ROI and lower risk than researching robotics.
I dont think this is cynical and I don't think it's a bad thing. OpenAI is not a huge org. The truth in 2021 is that not only is robotics 'just not there yet' in terms of being a useful vehicle for general intelligence research (obviously robotics research itself is still valuable), there is also nothing really pointing at this going to be the case in the next 5-10 years. Given that, unless they want to commercialise…
How successful do you think attempts to monetize this will be? Apart from Kiva at Amazon, I'm not even sure most shelf-moving robots are profitable enterprises (GreyOrange, Berkshire Grey, etcetera). I'm very skeptical of more general purpose warehouse robots such as you see from Covariance, Fetch, etcetera. I don't really know too much about fruit-picking other than grokking how hard it would be and how little it would pay.
To be clear, I'm not saying these companies make no money or have no customers. But it's not clear to me that any of them are profitable or likely will be soon, and robots are very expensive. I'm happy to learn why I'm wrong and these companies/technologies are further ahead than I realize.
Earlier quoted context omitted.
Wojciech stated this pretty explicitly on his Gradient Dissent podcast a few months back.
After a bit of Googling are you referring to Wojciech, the head of YouTube?
Is the prevailing opinion that progress in reinforcement learning is dependent on algorithmic advances, as opposed to simply scaling existing algorithms? If that is the case, I could see this decision as an acknowledgement that they are not well positioned to push the frontier of reinforcement learning - at least not compared to any other academic or industry lab. Where they have seen success, and the direction it se…
Reinforcement learning itself is a dead-end on a road to AI. They seem to slowly starting to realize it, probably ahead of academia.