Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Flywheel (YC S25) – Waymo for Excavators

news.ycombinator.com

Post reply on HN
Print view View original

71–80 of 82 posts

Re: Launch HN: Flywheel (YC S25) – Waymo for Excavators

#71

Post by shepardrtc »

jashmota wrote: post #20
shepardrtc wrote: post #16

How much safety training have you done with the models? i.e. does it know to stop if it's about to drive over a human?

We have some autonomous safety layers (comprised of object/human detection like you mentioned and others) even for remote teleop. But we haven't deployed autonomy on real sites yet. We still have some work to do before we can reliably deploy/test it out on the actual sites!

Safety is a huge concern with construction and really any operation that uses heavy equipment. I promise you they all know about Tesla autopilot running into people and will absolutely bring it up ad nauseum.

And I only say this because I think this is an interesting area to pursue so I hope you're successful with it. But a focus on safety will be of paramount importance.

Re: Launch HN: Flywheel (YC S25) – Waymo for Excavators

#72

Post by jashmota »

mitthrowaway2 wrote: post #59

What do you think of Komatsu's excavator teleoperation system[1], and what are you doing differently than they are? [1] https://youtu.be/QKMsqFMcmL0&t=962

1. We're OEM/machine-agnostic 2. Full compatibility with any existing model and not just specific recent models

Re: Launch HN: Flywheel (YC S25) – Waymo for Excavators

#73

Post by fragmede »

jashmota wrote: post #29
opwieurposiu wrote: post #21

Earlier quoted context omitted.

Yes, and you also get feedback from your butt as the machine tips and wobbles, particularly on smaller machines. Hearing the engine straining helps also. Often you can not clearly see what you are digging, this feedback lets you know if you are running into a rock or something. One big advantage would be cameras mounted on the boom and rear view cameras, as many machines have obstructed views.

We're indeed streaming the audio and have haptic feedback. My hypothesis is the seat vibration isn't as helpful. It's sub optimal and operators would be far more productive without it. We would do a paper when we have enough data on this. We're also putting more cameras but streaming a lot of them at once is tricky.

The problem is how do you quantify it and metricize it and get some numbers to compare? Because it's absolutely invaluable after you get to a level of mastery with a machine that you're sitting in that the seat moves in. You just feel it, you can't explain it in words, you're just one with the machine. Your butt really needs to feel that in order for that to really happen. Can you do it without it? Absolutely. Does it make it worse to not have that? Also true. It's extra effort, it's extra cost, and it makes things better for the actual users of the product, but it'll never show up as something you can measure.

Do you want to sell Microsoft Teams to the executives or do you want to give joy to the people who actually use the product?

Re: Launch HN: Flywheel (YC S25) – Waymo for Excavators

#74

Post by jashmota »

mschuster91 wrote: post #65
jashmota wrote: post #56

Earlier quoted context omitted.

I think swinging (which is about 40% of dig and dump workflow by time spent) should not be manual. That's lowest levels of autonomy which requires roughly centering to the pit/truck which we have already achieved. Hence operator only has to look in front! Those workflow numbers come from multiple observations at different sites, one of the examples is this: https://www.youtube.com/watch?v=orEsvu1CS64 I wish to talk t…

> I think swinging (which is about 40% of dig and dump workflow by time spent) should not be manual. It's been over a decade since I last operated an excavator, so grains of salt as usual - but I'd say it should be manual, or at least semi-automated. You need to take care where you unload the bucket on a truck, to avoid its weight distribution being off-center, or to keep various kinds of soil separated on the bed (e…

I agree - the dumping and digging itself (where you move boom, arm, bucket much more than swing/tracks) should be manual. But swinging to the truck and back to pit (pure swinging motion to center around these areas of interest) do not have to be manual. I agree with your and other comments that the transition has to be smooth and that's something we're working on.

Re: Launch HN: Flywheel (YC S25) – Waymo for Excavators

#75

Post by jashmota »

fragmede wrote: post #73
jashmota wrote: post #29

Earlier quoted context omitted.

We're indeed streaming the audio and have haptic feedback. My hypothesis is the seat vibration isn't as helpful. It's sub optimal and operators would be far more productive without it. We would do a paper when we have enough data on this. We're also putting more cameras but streaming a lot of them at once is tricky.

The problem is how do you quantify it and metricize it and get some numbers to compare? Because it's absolutely invaluable after you get to a level of mastery with a machine that you're sitting in that the seat moves in. You just feel it, you can't explain it in words, you're just one with the machine. Your butt really needs to feel that in order for that to really happen. Can you do it without it? Absolutely. Does i…

You've touched one of my favorite things to think about. I think about this a lot. There will always be people who enjoy fully analog cars/planes/watches over alternatives. I think that doesn't define the whole market. How many of new joinees are we seeing in this industry? Would you rather have a functioning back because you operate these remotely or go to site and sit in one of these and have a broken back by 50s? This is leaving out the fact that remote operation could save lots of lives. In our launch video we did put together some of the news clips on it: https://www.ycombinator.com/launches/O8i-flywheel-ai-waymo-f...

Re: Launch HN: Flywheel (YC S25) – Waymo for Excavators

#76

Post by jashmota »

jasongill wrote: post #69
jashmota wrote: post #31

Earlier quoted context omitted.

Thanks! Yes we're testing both CAN spoofing and retrofits. CAN spoofing really limits compatibility to a few machines/brands. Would love to get in touch with you: my email is contact ai useflywheel dot ai

I disagree - your AI model should be decoupled from your control logic; so if the model determines that you need to move the boom, your control logic takes that action - either via hydraulic retrofit, canbus control, or just faking it in a simulator, and the AI model observes and reacts to the change as a result of that action. So in reality, CANbus control should be no different than hydraulic control to your system…

I totally agree, what I meant was if we only do CANbus retrofit, we would really limit ourselves in how compatible we are. So we have to focus on doing the right kind of retrofit really well. CANbus retrofit is easier and we've done that many times before. Especially because my cofounder Mahimana used to work in self-driving company and worked on this and single-handedly built out their current ML data and eval pipeline.

Re: Launch HN: Flywheel (YC S25) – Waymo for Excavators

#77

Post by jashmota »

aorloff wrote: post #60

I would create a bespoke training set from the very best operator you can find, after significant interviewing, and then only use that to train your model (eg. partner with a local firm that rents at Herc or United Rentals a lot and has a skilled operator and train on their jobs while you rent them free equipment). The range of skillset on these things is large, there are 2 dominant (fairly swappable - meaning most m…

This is a great idea - I would dig into this. I disagree on AI not winning here. Support crew mostly work on eliminating blindspots for operator. Yes you can eliminate blindspots with cameras and a 7inch TFT screen, but those are small, don't work great in direct sunlight. The current solution not being 10x better is why they still fallback to having those crew members. I genuinely believe we will change that.

Re: Launch HN: Flywheel (YC S25) – Waymo for Excavators

#78

Post by jashmota »

shepardrtc wrote: post #71
jashmota wrote: post #20

Earlier quoted context omitted.

We have some autonomous safety layers (comprised of object/human detection like you mentioned and others) even for remote teleop. But we haven't deployed autonomy on real sites yet. We still have some work to do before we can reliably deploy/test it out on the actual sites!

Safety is a huge concern with construction and really any operation that uses heavy equipment. I promise you they all know about Tesla autopilot running into people and will absolutely bring it up ad nauseum. And I only say this because I think this is an interesting area to pursue so I hope you're successful with it. But a focus on safety will be of paramount importance.

I completely agree. If you have worked in the industry, I would like to talk to you and get your feedback on what I might be missing on my understanding in requirements in safety layers. Would be great if we can chat on email: contact at useflywheel dot ai

Re: Launch HN: Flywheel (YC S25) – Waymo for Excavators

#79

Post by jashmota »

IgorPartola wrote: post #70
jashmota wrote: post #28

Earlier quoted context omitted.

You're right! This is exactly why we like to do mechanical actuation - we are able to achieve bilateral telepresence, which essentially gives the torque (haptic) feedback over the internet! So on small excavators, you can absolutely feel the resistance. We also stream the engine audio, which tells you how hard the hydraulic pump is working. Operators like our system for these reasons :) I'd like to get a chance to ta…

Not my industry at all. I am curious if something like this is an opportunity for a whole new type of controls and feedback. Since the operator doesn’t have to be in the excavator physically they could take on any position: standing, sitting, lying down, etc. Instead of sending haptic feedback to the joystick it could be sent to a vibrating wrist band. You could hook up the equivalent of a Nintendo Power Glove to hav…

These are really really interesting thoughts around the remote teleop interface. A few things you mentioned have been on my mind. Teleop UX is underdeveloped and its affects is underestimated, and it could turn out to be a huge thing for us and humanoid companies if autonomy is harder than it seems now. I don't believe construction equipment operation today is optimal. We just go with it as that's what we have. It's far from intuitive and easy for a newbie to start operating to give some context and requires hours of practice. There would be a bit of training to be done for remote teleop as well. Might as well make it easy to interface and improve on the existing experience, which hasn't really evolved for years and decades!

I'd like to have a chat with you if you're up for it: contact at useflywheel dot ai

Re: Launch HN: Flywheel (YC S25) – Waymo for Excavators

#80

Post by rapjr9 »

One potential application might be fighting wildfires. Seems like it would be really useful if one firefighter could remotely monitor a dozen autonomous bulldozers that were given general instructions. Would have to acquire a different training data set I'd guess, but the same approach seems applicable, get the data from teleoperated bulldozers used to fight fires. Getting the bulldozers into the right area would require some transport, like a truck or heavy lift helicopter, though maybe mini bulldozers would also work.

You might be interested in the work of Peter Corke also, he's automated horizontal mine shaft loaders and huge drag line shovels in his research:

https://www.youtube.com/watch?v=YUb9_Ysd2Hw

I think he used a different approach than you do, using visual servoing to get feedback and data from a camera. Maybe there's some value in combining both approaches, learn to control a machine from an operator, and also keep track of what is being moved with a camera to add another layer of control.

Post reply on HN