LLMs are trained on text. Why would we expect them to understand a visual and tactile 3D world?
Show HN: Only 1 LLM can fly a drone
61–70 of 100 posts
Re: Show HN: Only 1 LLM can fly a drone
#62This sounds like a good way to get your drone shot down by a Concerned Citizen or the military.
Re: Show HN: Only 1 LLM can fly a drone
#63LLMs flying weaponized drones is exactly how it starts.
It's pretty entertaining seeing the plot lines and ficticious history in The Terminator movies actually happening in real time.
Re: Show HN: Only 1 LLM can fly a drone
#64LLMs are trained on text. Why would we expect them to understand a visual and tactile 3D world?
Because they’re also multimodal vLLMs.
Re: Show HN: Only 1 LLM can fly a drone
#65This is what VLA models are for. They would work much better. Would need a bit of fine tuning but probably not much. Lots of literature out there on using VLAs to control drones.
Did some research, found a model that is exactly that. https://cognitivedrone.github.io/
The Black Mirror speedrun continues
Re: Show HN: Only 1 LLM can fly a drone
#66Why would you want an LLM to fly a drone? Seems like the wrong tool for the job -- it's like saying "Only one power drill can pound roofing nails". Maybe that's true, but just get a hammer
There are almost endless reasons why. It's like asking why would you want a self-driving car. Having a drone to transport things would be amazing, or to patrol an area. LLMs can be helpful with object identification, reacting to different events, and taking commands from users. The first thought I had was those security guard robots that are popping up all over the place. if they were drones instead, and LLM talked t…
Both of those proposed uses are bad things that are worse than what they would replace.
Re: Show HN: Only 1 LLM can fly a drone
#67Earlier quoted context omitted.
I don’t think trendy is really the right word and maybe it’s not state of the art but a lot of us in the industry are seeing emerging capabilities that might make it SOTA. Hope that makes sense.
LLMs are indeed the definition of trendy (I've found using Google Trends to dive in is a good entry point to get a broad sense of whether something is "trendy")! Basically the right way to think about it is that something can be promising, and demonstrate emerging capabilities, but but those things don't make something SOTA, nor do they make it trendy. They can be related though (I expect everything SOTA was once pro…
I think the point may have flown over your head. I am suggesting you are being dismissive with a distinct lack of thought on your reply. Like said I don’t think state of the art is the right way to describe it but I think trendy is equally wrong from the other side of the spectrum. Models that can deal with vision have some really interesting use cases and ones that can be valuable, in a lot of ways I would say state of the art could describe it but I know to folks that are hopelessly negative, it’s a hard reach so I was trying to balance it for you. Hope that makes sense.
Re: Show HN: Only 1 LLM can fly a drone
#68Why would you want an LLM to fly a drone? Seems like the wrong tool for the job -- it's like saying "Only one power drill can pound roofing nails". Maybe that's true, but just get a hammer
What’s the right tool then? This looks like a pretty fun project and in my rough estimation a fun hacker project.
The right tool would likely be some conventional autopilot software; if you want AI cred you could train a Neural Network which maps some kind of path to the control features of the drone. LLMs are language models -- good for language, but not good for spacial reasoning or navigation or many of the other things you need to pilot a drone.
Re: Show HN: Only 1 LLM can fly a drone
#69Earlier quoted context omitted.
There are almost endless reasons why. It's like asking why would you want a self-driving car. Having a drone to transport things would be amazing, or to patrol an area. LLMs can be helpful with object identification, reacting to different events, and taking commands from users. The first thought I had was those security guard robots that are popping up all over the place. if they were drones instead, and LLM talked t…
You could have a program, not LLM-based but could be ANN, for flying and an LLM for overseeing; the LLM could give the program instructions to the pilot program as a (x,y,z) directions. I mean currently autopilots are typically not LLMs, right? You describe why it would be useful to have an LLM in a drone to interact with it but do not explain why it is the very same LLM that should be doing the flying.
I'm not OP, I don't know what specific roles the LLM should be using, but LLMs are great with object recognition, and using both text (street signs,notices,etc..) and visual cues to predict the correct response. The actual motor control i'm sure needs no LLMs, but the decision making could use any number of solutions, I agree that an LLM-only solution sounds bad, but I didn't do the testing and comparison to be confident in that assessment.
Re: Show HN: Only 1 LLM can fly a drone
#70Earlier quoted context omitted.
We don't need a lot of things, but new tech should also address what people want, not just needs. I don't know how to pilot drones, nor do I care to learn how to, but I want to do things with drones, does that qualify as a need? Tech is there to do things for us we're too lazy to do.
There are two different things: 1. a drone that you can talk to and fly on its own 2. a drone where the flying is controlled by an LLM (2) is a specific instance of the larger concept of (1). You make an argument that 1 should be addressed, which no one is denying in this thread - people are arguing that (2) is a bad way to do (1).
You're considering "talking to" a separate thing, I consider it the same as reading street signs or using object recognition. My voice or text input is just one type of input. Can other ML solutions or algorithms detect a tree (same as me telling it there is a tree,yaw to the right), yes, can LLMs detect a tree and determine what course of action to take? also true. Which is better? I don't know, but I won't be quick to dismiss anyone attempting to use LLMs.