Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Only 1 LLM can fly a drone

github.com

Post reply on HN
Print view View original

31–40 of 100 posts

Re: Show HN: Only 1 LLM can fly a drone

#31

Post by volkercraig »

notepad0x90 wrote: post #13
lewispollard wrote: post #11

Earlier quoted context omitted.

The point is that you don't need an LLM to pilot the thing, even if you want to integrate an LLM interface to take a request in natural language.

We don't need a lot of things, but new tech should also address what people want, not just needs. I don't know how to pilot drones, nor do I care to learn how to, but I want to do things with drones, does that qualify as a need? Tech is there to do things for us we're too lazy to do.

I don't think you understand what an "LLM" is. They're text generators. We've had autopilot since the 1930s that relies on measurable things... like PID loops, direct sensor input. You don't need the "language model" part to run an autopilot, that's just silly.

Re: Show HN: Only 1 LLM can fly a drone

#32

Post by volkercraig »

smw1218 wrote: post #26
bigfishrunning wrote: post #2

Why would you want an LLM to fly a drone? Seems like the wrong tool for the job -- it's like saying "Only one power drill can pound roofing nails". Maybe that's true, but just get a hammer

It's a great feature to tell my drone to do a task in English. Like "a child is lost in the woods around here. Fly a search pattern to find her" or "film a cool panorama of this property. Be sure to get shots of the water feature by the pool." While LLMs are bad at flying, better navigation models likely can't be prompted in natural language yet.

What you're describing is still ultimately the "view" layer of a larger autopilot system, that's not what OP is doing. He's getting the text generator to drive the drone. An LLM can handle parsing input, but the wayfinding and driving would (in the real world) be delegated to modern autopilot.

Re: Show HN: Only 1 LLM can fly a drone

#34

Post by pixl97 »

volkercraig wrote: post #31
notepad0x90 wrote: post #13

Earlier quoted context omitted.

We don't need a lot of things, but new tech should also address what people want, not just needs. I don't know how to pilot drones, nor do I care to learn how to, but I want to do things with drones, does that qualify as a need? Tech is there to do things for us we're too lazy to do.

I don't think you understand what an "LLM" is. They're text generators. We've had autopilot since the 1930s that relies on measurable things... like PID loops, direct sensor input. You don't need the "language model" part to run an autopilot, that's just silly.

You see to be talking past him and ignoring what they are actually saying.

LLMs are a higher level construct than PID loops. With things like autopilot I can give the controller a command like 'Go from A to B', and chain constructs like this to accomplish a task.

With an LLM I can give the drone/LLM system complex command that I'd never be able to encode to a controller alone. "Fly a grid over my neighborhood, document the location of and take pictures of every flower garden".

And if an LLM is just a 'text generator' then it's a pretty damned spectacular one as it can take free formed input and turn it into a set of useful commands.

Re: Show HN: Only 1 LLM can fly a drone

#35

Post by njhnjhnjhnjh »

volkercraig wrote: post #31
notepad0x90 wrote: post #13

Earlier quoted context omitted.

We don't need a lot of things, but new tech should also address what people want, not just needs. I don't know how to pilot drones, nor do I care to learn how to, but I want to do things with drones, does that qualify as a need? Tech is there to do things for us we're too lazy to do.

I don't think you understand what an "LLM" is. They're text generators. We've had autopilot since the 1930s that relies on measurable things... like PID loops, direct sensor input. You don't need the "language model" part to run an autopilot, that's just silly.

[dead]

Re: Show HN: Only 1 LLM can fly a drone

#36

Post by laffOr »

notepad0x90 wrote: post #13
lewispollard wrote: post #11

Earlier quoted context omitted.

The point is that you don't need an LLM to pilot the thing, even if you want to integrate an LLM interface to take a request in natural language.

We don't need a lot of things, but new tech should also address what people want, not just needs. I don't know how to pilot drones, nor do I care to learn how to, but I want to do things with drones, does that qualify as a need? Tech is there to do things for us we're too lazy to do.

There are two different things:

1. a drone that you can talk to and fly on its own

2. a drone where the flying is controlled by an LLM

(2) is a specific instance of the larger concept of (1).

You make an argument that 1 should be addressed, which no one is denying in this thread - people are arguing that (2) is a bad way to do (1).

Re: Show HN: Only 1 LLM can fly a drone

#37

Post by encrux »

volkercraig wrote: post #30

I don't understand. Surely training an LSTM with sensor input is more practical and reasonable way than trying to get a text generator to speak commands to a drone.

Very much depends on what you want to do.

The fact that a language model can „reason“ (in the LLM-slang meaning of the term) about 3D space is an interesting property.

If you give a text description of a scene and ask a robot to perform a peg in hole task, modern models are able to solve them fairly easily based on movement primitives. I implemented this on a UR robot arm back in 2023

The next logical step is, instead of having the model output text (code representing movement primitives), outputting tokens in action space. This is what models like pi0 are doing.

Re: Show HN: Only 1 LLM can fly a drone

#38

Post by dr-detroit »

pavlov wrote: post #4
bigfishrunning wrote: post #2

Why would you want an LLM to fly a drone? Seems like the wrong tool for the job -- it's like saying "Only one power drill can pound roofing nails". Maybe that's true, but just get a hammer

Yeah, it feels a bit like asking "which typewriter model is the best for swimming".

[dead]

Re: Show HN: Only 1 LLM can fly a drone

#39

Post by irl_zebra »

avaer wrote: post #25
bigfishrunning wrote: post #2

Why would you want an LLM to fly a drone? Seems like the wrong tool for the job -- it's like saying "Only one power drill can pound roofing nails". Maybe that's true, but just get a hammer

Using an LLM is the SOTA way to turn plain text instructions into embodied world behavior. Charitably, I guess you can question why you would ever want to use text to command a machine in the world (simulated or not). But I don't see how it's the wrong tool given the goal.

SOTA typically refers to achieving the best performance, not using the trendiest thing regardless of performance. There is some subtlety here. At some point an LLM might give the best performance in this task, but that day is not today, so an LLM is not SOTA, just trendy. It's kinda like rewriting something in Rust and calling it SOTA because that's the trend right now. Hope that makes sense.
Post reply on HN