ChatGPT for Robotics

PromptCraft Open Source Tool and AirSim-chatGPT integration

Chatgpt for drones. “Fly to [address], make sure to take a discrete route with low traffic, take a few pictures on each side of the house. Scan for people near by, make sure people are no closer than 300ft”.

Now this is the future. We're living in the androids timeline.

This is another indication that LLMs are becoming able to function as general AIs (the term AGI has a lot of baggage). Especially at the end of the real-time feedback video[1] The LLM seems to be acting as the high-level planner, based on the outputs of all the computer vision and object recognition happening at lower levels.

[1] https://youtu.be/p0fDH9zZm_c

The microwave example is very interesting.

The prompt only asked to "warm up my lunch" without specifying how.

SayCan[1] generated step-wise high level instructions using LLMs for robotic tasks. This takes it a step further by converting high level instructions to low level actions almost entirely autonomously.

[1] https://say-can.github.io/