Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a robotics research setup that lives next to my desk

dfdxlabs.com

Post reply on HN
Print view View original

11–20 of 64 posts

Re: Building a robotics research setup that lives next to my desk

#11

Post by mplappert »

dlt713705 wrote: post #8

As impressive as this setup may be, I'm still amazed at how slow this type of robot is, whether amateur or professional grade. I have no expertise in this field, but as an observer, the apparent progress in this area seems very limited. I guess my expectations are too high and my understanding of the problems to solve is too low.

It’s partially my fault I currently clip the max speed _and_ I only input soft control changes when teleoperating to avoid crashing into things. The robot itself could definitely move more quickly than what you see in the video.

It would be interesting to explore how RL can be applied on top of my (flawed) human demos to optimize beyond what I’m able to do.

Re: Building a robotics research setup that lives next to my desk

#12

Post by mplappert »

avilay wrote: post #5

Hey this is cool! I am doing something similar myself with the SO101 arm robot from Robot Studio using a patchwork of my own code and LeRobot. Would love to collaborate with you if you are open to it. You can find me on Discord as `.avilay`. https://www.linkedin.com/posts/avilay_lerobot-huggingface-ro... Would like to know your reasoning on not going with LeRobot.

Looks very cool! I’m not a huge discord user but how about you shoot me an email and we can figure out how to share notes? (I don’t want to post it directly here but it’s easy to find on my personal website, just google my name)

Re why not SO-101: the article has a footnote about this; I actually bought the SO-101 as well! I want to integrate it into the same setup so I can switch depending on task.

Somewhat surprisingly the xarm was actually much faster to arrive; I got it within 2 days of ordering. I don’t have a 3D printer and getting the SO-101 from the vendor I ordered it at took almost 4 weeks. So partially it just came down to what I had access to more quickly.

Second point is reliability: I think the SO-101 is cool but I’d be surprised if it doesn’t break more quickly than the xarm. I wanted something that’s going to last a long time without headaches. And these industrial arms are really mature hardware wise now.

Hope this helps!

Re: Building a robotics research setup that lives next to my desk

#15

Post by mplappert »

wxw wrote: post #4

Great article. I'll be following along. Would like to learn more about the robotics space. - I've heard the advantage of ROS besides the architecture is the ecosystem (driver integrations, etc). Is that not an issue because the arm supports a Python SDK OOTB? - Any issues you've been running into with this setup? - How do you determine if a session recording is good enough for training? Is 50/100 samples really all y…

Glad you like it!

Re your questions:

- The driver situation turned out totally fine; I intentionally picked HW with good python sdk support so that was very painless.

- The static camera (the C920) is not super great; it drops frames and sometimes cuts out. We’ll see how that goes but it’s probably the clostest thing I want to swap right now. Another issue is reach of the arm when forcing the worst to be axis parallel with the table; you cannot get very far away. The chess setup demo in the video gives an example: I can just reach the row of pawns and beyond that it’s out of reach.

- I don’t know yet! The 50-100 figure comes from the ACT and diffusion policy papers but it depends on the type of task. For fine tuning my sense is that you only need a few hours worth of demos to get good results with pi0.5 etc. a big reason I’m doing this project is that I want to try all of this myself, so the next posts definitely will talk about that

Re: Building a robotics research setup that lives next to my desk

#16

Post by mplappert »

whiplash451 wrote: post #14

How does Lerobot prevent « full control » and « understanding »? I thought this was an open source library. I am not an official supporter of the library but am asking out of curiosity.

For understanding: I think the level is much deeper if I wrote the code vs reading someone else’s. Same applies to coding agents of course which is why I wrote most of it myself and only delegate some tasks (for example codex was great help at setting up telemetry dashboards or writing the custom glfw renderer).

On control: LeRobot will change all the time and I’ll be unaware of what changed. If something suddenly doesn’t work anymore, it’s a pain to find out. I can of course fork or pin but that defeats the purpose a bit.

At the end it’s also partially just preference: I wanted to write this layer myself, and I have opinions about how it should be architected, so I did.

Re: Building a robotics research setup that lives next to my desk

#17

Post by avilay »

nl wrote: post #9
avilay wrote: post #5

Hey this is cool! I am doing something similar myself with the SO101 arm robot from Robot Studio using a patchwork of my own code and LeRobot. Would love to collaborate with you if you are open to it. You can find me on Discord as `.avilay`. https://www.linkedin.com/posts/avilay_lerobot-huggingface-ro... Would like to know your reasoning on not going with LeRobot.

I'm very interested in the SO101. I've never done any robotics and that seems a palatable entry level thing to try things out. How have you found it? (The author does explain his reasons for not using LeRobot in the post - although "I also use LeRobot for training and running baseline policies, and the vendor SDKs for the hardware.")

> The author does explain his reasons for not using LeRobot in the post - although "I also use LeRobot for training and running baseline policies, and the vendor SDKs for the hardware.")

Ah! That is exactly what I use it for as well :-)

I am liking the SO101 - teleop and robot both work great. For sure, it is very easy to get started with. I was able to collect around 50 demos with them and train my first ACT policy within days of setting up the robots. Happy to share more detailed learnings from this if/when you get started with it. https://github.com/avilay/learn-robotics

Re: Building a robotics research setup that lives next to my desk

#19

Post by sails »

Love this, I’m playing around with the cheapo esp32+servos version of this, super fun.

Something I’m working on is a hardware CLI for agents to run experiments, with a “CICD” pipeline that validates everything and means I can delegate more of the experiments to the agents. I wonder if you have any thoughts on this?

The idea is to allow the coding agent to run the full loop of experiments and validations, with vision, audio, button pressing, speaking etc to interact in place of the human

Re: Building a robotics research setup that lives next to my desk

#20

Post by avilay »

mplappert wrote: post #12
avilay wrote: post #5

Hey this is cool! I am doing something similar myself with the SO101 arm robot from Robot Studio using a patchwork of my own code and LeRobot. Would love to collaborate with you if you are open to it. You can find me on Discord as `.avilay`. https://www.linkedin.com/posts/avilay_lerobot-huggingface-ro... Would like to know your reasoning on not going with LeRobot.

Looks very cool! I’m not a huge discord user but how about you shoot me an email and we can figure out how to share notes? (I don’t want to post it directly here but it’s easy to find on my personal website, just google my name) Re why not SO-101: the article has a footnote about this; I actually bought the SO-101 as well! I want to integrate it into the same setup so I can switch depending on task. Somewhat surprisi…

Thanks for your response! I totally get your point about the delay in getting the robot, I ordered mine from PartaBot and they did take a couple of weeks to get here. But when they did, they worked great out-of-the-box :-)

Will email you to compare notes.

Post reply on HN