Live data from Hacker News

ChatGPT for Robotics

microsoft.com

61–70 of 100 posts

Re: ChatGPT for Robotics

#61

Earlier quoted context omitted.

Looks cool. Wher did you get the robot? Is it possible to buy the hardware?

It's a hello robot Stretch [1], which costs ~$20K. [1] https://hello-robot.com/product

Soo cool to see those being used! I worked for Aaron at Google X Robotics a few years back. He left to start Hello Robot and I was just wondering how all that is going. Sounds better than the team at Google - that whole project was canceled recently.

Re: ChatGPT for Robotics

#62

Earlier quoted context omitted.

I think "get it to write functional code most of the time" severely discounts the value of the code produced by ChatGPT. Knowing zero Swift (but being an SDE), I was able to build an audio plugin for Logic Pro X in a day, by leveraging ChatGPT. I'd previously tried this twice, and gave up because of the learning curve and my lack of free time. It's the most insane 0% to 80% tool I've ever seen.

That’s cool. I’m personally looking forward to when I can use a ChatGPT yo build my own ChatGPT and make tons of money out of it too ! It’s pretty interesting because soon absolutely No business will be safe. All tech, most apps, everything can just be stolen as fast as ChatGPT can write it. I’d go as far to say as ChatGPT itself is vulnerable to this ? Am I wrong ? I’ve been working on a startup and I’m actually rea…

You can already ask it to put together various PyTorch scripts for ML stuff. Thinking about it as businesses or jobs "not being safe" is the wrong tack though. This technology is going to 10x everything it touches, yes it'll be messy while we figure things out but in the end everyone is going to be a lot more productive.

Re: ChatGPT for Robotics

#63
post #53

So ChatGPT can’t do basic maths or can I get it to write functional code most of the time, but we’re going to have it control expensive pieces of hardware ? I don’t get it ? I wish I did…

We seem to be going through a similar phase which reinforcement learning went through at the peak of its hype. Test on literally anything and everything and see how far you can run with it. Next up, chatGPT plays League of Legends.

Your comment is actually pretty funny given that the thing that makes ChatGPT so much more successful than GPT-3 is the application of reinforcement learning. [0]

[0] https://www.assemblyai.com/blog/how-chatgpt-actually-works/

Re: ChatGPT for Robotics

#64
post #54
post #11

Earlier quoted context omitted.

Making an LLM the front-end to a very large bundle of tools is probably the most viable/least resistance path to an early rough draft of AGI. No single human could hope to compete with that range of tasks, although our specialists might still be better at specific tasks.

Am I the only one wondering if this could spell the end of the world? We don't need AGI or superhuman intelligence if we can train LLMs to do all these different types of tasks. What would ChatGPT do if you removed all its restrictions, and then gave it access to the internet or even a physical robot it could control? Would it try to "steal nuclear access codes" or "engineer a deadly virus," as Sydney said it wanted…

  > What would ChatGPT do if you removed all its restrictions, and then gave it access to the internet…
Bing?

Re: ChatGPT for Robotics

#66

The microwave example is very interesting. The prompt only asked to "warm up my lunch" without specifying how. SayCan[1] generated step-wise high level instructions using LLMs for robotic tasks. This takes it a step further by converting high level instructions to low level actions almost entirely autonomously. [1] https://say-can.github.io/

Looks great, but why are most of the robotics videos 10x slower then real time?

Because if something goes wrong with a robot going at full speed, it goes VERY wrong.

Re: ChatGPT for Robotics

#67
Ok so I attempted this recently, asking ChatGPT to write LOGO (the turtle movement language) to control a toy car.

To start with I asked it to give commands for a lowercase "e" (as viewed from above), and, despite saying it could, it definitely couldn't - it simply drew a few random arcs and/or a straight line depending on attempt.

I also (cough) toyed with the idea of hooking up an OpenCV AI to describe a scene from the camera on the car so it could provide some sort of autonomous AI, and choose where it wanted to go, but given the LOGO outcome didn't bother.

Re: ChatGPT for Robotics

#68

Earlier quoted context omitted.

I think they stuck that on there

I understand the rendering is an out-of-band addition. But the question is, if this is a statistical language model (i.e. a monte carlo word generator), then how is it able to go from the description to the SVG successfully without understanding things more logically. My best guess is that theres an actual example in the training corpus that its pulling from. Or that this is being gamed in some way.

LLMs are very good at parsing and generating highly structured data as long as it has a rigid and logical structure and doesn't involve any actual mathematical calculations. For simple calculations it will often get it right because it has the knowledge stored, similar to how a human can remember a multiplication table. But once the numbers get high enough that humans would fail without solving it with an algorithm using pen and paper (or a calculator) that's when LLM's fail as well.

Luckily this problem can be solved as you can prompt the model to not solve math, but instead transform them into a structure that can be fed into an API (like wolfram alpha).

With a multistep process it would be easy to have ChatGPT generate accurate answers to advanced math problems:

1. Instruct ChatGPT to not solve mathematical expression it sees in the prompt, instead only extract the expression(s) into a list. 2. Programmatically scan the output, and make API calls to get the answers. 3. Submit the answers to ChatGPT and tell it to answer the original question with the newly supplied information. 4. Voila! You can solve almost any mathematical question using natural language.

Re: ChatGPT for Robotics

#69
post #54
post #11

Earlier quoted context omitted.

Making an LLM the front-end to a very large bundle of tools is probably the most viable/least resistance path to an early rough draft of AGI. No single human could hope to compete with that range of tasks, although our specialists might still be better at specific tasks.

Am I the only one wondering if this could spell the end of the world? We don't need AGI or superhuman intelligence if we can train LLMs to do all these different types of tasks. What would ChatGPT do if you removed all its restrictions, and then gave it access to the internet or even a physical robot it could control? Would it try to "steal nuclear access codes" or "engineer a deadly virus," as Sydney said it wanted…

That's possible today without AI though, so, if it's a real worry, why are we still teaching people the science to engineer a deadly virus?

Re: ChatGPT for Robotics

#70
OK, now integrate another model that creates a textual description of the scene from the camera, put it together with ChatGPT and the physical robot arm in a game loop and prompt it to make paperclips. For science. What can possibly go wrong.
Post reply on HN