Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a robotics research setup that lives next to my desk

dfdxlabs.com

Post reply on HN
Print view View original

31–40 of 64 posts

Re: Building a robotics research setup that lives next to my desk

#31

Post by mplappert »

andned wrote: post #30

I had a very similar setup. Really happy with the xarm 6 lite. I played around with the diffusion policy paper experiments and was thinking to buy a webcam as a top camera as well but I ended up buying two intel realsense ones because of the timestamp drift issues. How did you solve that? Or is camera feed syncing not necessary for your intended projects?

I timestamp everything twice: once with the hardware clock (if available, like for the realsense camera) and once within my robot stack once it gets read from the device (using `time.monotonic_ns()`). Both are stored and alignment can happen with either timestamp. I think the 2nd timestamp is actually more meaningful since ultimately I want to reconstruct the state that the policy would've seen; so if one modality is delayed I should actually include that effect during training.

That being said, I might switch to a realsense for the static tabletop camera as well; the realsense wrist is clearly much more reliable than the cheap Logitech C920 that I currently use.

Re: Building a robotics research setup that lives next to my desk

#32

Post by mplappert »

thomasikzelf wrote: post #28

Nice, I will be following your posts! I just bought a robot arm myself, the seeed studio B601DM (€1500 6+1 axis), it works great and is open source hardware as well and a bit more solid then the so101. I also opted to not use ros, I don't want to give up control by putting another framework in between. Is your plan to see whats possible right now or do you also have ideas on how to improve sota?

Oh very cool! Looks a bit like the TRLC-DK1 (I was looking at this one for a bit).

I think pushing the sota is quite hard to do solo but we'll see. Mostly I want to get back up to speed after having not done much robotics during the last 6 years. Best way for me to learn is to just do it, so here we are. We'll see how far I get (I suspect at some point compute will be the main bottleneck)

Re: Building a robotics research setup that lives next to my desk

#33

Post by colinator »

Good move getting a nice robot! I'm doing something similar to you, but I went with a cheap robot, the "HIWONDER 6DOF Robotic Arm Kit". It was only $600, but wow is it bad. The precision and repeatability are both "are you drunk?" level. I can hear the gears grind when it moves. I suppose I should upgrade. But if my system can work with a terrible robot, I assume it would work even better with a nice one!

My project is https://github.com/colinator/Ariel - basically, no VLAs - instead, "just write code". Or have the agents do it.

I don't have a writeup yet about applying Ariel to _this_ robot, but this is for a previous one: https://colinator.github.io/Ariel/post1.html.

Excited to follow your progress!

Re: Building a robotics research setup that lives next to my desk

#34

Post by mplappert »

colinator wrote: post #33

Good move getting a nice robot! I'm doing something similar to you, but I went with a cheap robot, the "HIWONDER 6DOF Robotic Arm Kit". It was only $600, but wow is it bad. The precision and repeatability are both "are you drunk?" level. I can hear the gears grind when it moves. I suppose I should upgrade. But if my system can work with a terrible robot, I assume it would work even better with a nice one! My project…

Wow very cool project and thanks for the Ariel pointer! Added to my reading list.

And yes, having a nice robot makes life so much easier. Very happy with my choice so far!

Re: Building a robotics research setup that lives next to my desk

#35

Post by sails »

mplappert wrote: post #23
sails wrote: post #19

Love this, I’m playing around with the cheapo esp32+servos version of this, super fun. Something I’m working on is a hardware CLI for agents to run experiments, with a “CICD” pipeline that validates everything and means I can delegate more of the experiments to the agents. I wonder if you have any thoughts on this? The idea is to allow the coding agent to run the full loop of experiments and validations, with vision,…

Very cool! Have you seen the recent nvidia thing? They do this at scale for robotics manipulation: https://research.nvidia.com/labs/gear/enpire/

Ah no I hadn't seen that, very interesting.

I'm finding a gap just before running those experiments.

The process of updating firmware, doing basic smoke tests on each device and validating it is live, and can function correctly.

Basically the pre-deployment green light that you get on github, but for hardware.

Have you seen or thought about that at all?

Re: Building a robotics research setup that lives next to my desk

#36

Post by thomasikzelf »

mplappert wrote: post #32
thomasikzelf wrote: post #28

Nice, I will be following your posts! I just bought a robot arm myself, the seeed studio B601DM (€1500 6+1 axis), it works great and is open source hardware as well and a bit more solid then the so101. I also opted to not use ros, I don't want to give up control by putting another framework in between. Is your plan to see whats possible right now or do you also have ideas on how to improve sota?

Oh very cool! Looks a bit like the TRLC-DK1 (I was looking at this one for a bit). I think pushing the sota is quite hard to do solo but we'll see. Mostly I want to get back up to speed after having not done much robotics during the last 6 years. Best way for me to learn is to just do it, so here we are. We'll see how far I get (I suspect at some point compute will be the main bottleneck)

It looks like one stole the design from the other, I don't know which one, haha.

Re: Building a robotics research setup that lives next to my desk

#37

Post by mplappert »

sails wrote: post #35
mplappert wrote: post #23

Earlier quoted context omitted.

Very cool! Have you seen the recent nvidia thing? They do this at scale for robotics manipulation: https://research.nvidia.com/labs/gear/enpire/

Ah no I hadn't seen that, very interesting. I'm finding a gap just before running those experiments. The process of updating firmware, doing basic smoke tests on each device and validating it is live, and can function correctly. Basically the pre-deployment green light that you get on github, but for hardware. Have you seen or thought about that at all?

This seems indeed useful. I haven't seen this for robotics but I'm sure people need this for larger deployments (either for a distributed fleet or for an "arm farm"-like setup where there's many robots in one location for data collection / eval). Interesting idea!

Re: Building a robotics research setup that lives next to my desk

#40

Post by robotresearcher »

mplappert wrote: post #31
andned wrote: post #30

I had a very similar setup. Really happy with the xarm 6 lite. I played around with the diffusion policy paper experiments and was thinking to buy a webcam as a top camera as well but I ended up buying two intel realsense ones because of the timestamp drift issues. How did you solve that? Or is camera feed syncing not necessary for your intended projects?

I timestamp everything twice: once with the hardware clock (if available, like for the realsense camera) and once within my robot stack once it gets read from the device (using `time.monotonic_ns()`). Both are stored and alignment can happen with either timestamp. I think the 2nd timestamp is actually more meaningful since ultimately I want to reconstruct the state that the policy would've seen; so if one modality is…

Both timestamps are useful in different ways. The early-as-possible hardware stamp is best for reasoning about causality, while the later-and-full-o-jitter middleware stamps are good for compensating for that inevitable jitter.

Time is one of the hard problems in robots, because they are inevitably but non-obviously distributed systems.

Robots are annoyingly, wonderfully difficult.

Post reply on HN