Jetson AGX Xavier

I am designing a four wheel drive robot using the NVIDIA AGX Xavier [1] that will follow trails on its own or follow the operator on trails. You don't want your robot to lose cellular coverage and become useless. Even if you had coverage, there would be significant data usage as Rover uses four 4k cameras, which is about 30 megapixels (actually they max out at 13mp each or 52mp total). Constantly streaming that to the cloud would be very expensive on a metered internet connection. Even on a direct line the machine would saturate many broadband connections. Of course you can selectively stream but this makes things more complicated.

Latency is an issue. Imagine a self driving car that required a cloud connection. It's approaching an intersection and someone on a bicycle falls over near its path. Better send that sensor data to the cloud fast to determine how to act!

On my Rover robot it streams the cameras directly in to the GPU memory where it can be processed using ML without ever being copied through the CPU. It's super low latency and allows for robots that respond rapidly to their environment. Imagine trying to make a ping-pong playing robot with a cloud connection.

I am also designing a farming robot. [2] We don't expect any internet connection on farms!

[1] https://reboot.love/t/new-cameras-on-rover/ [2] https://www.twistedfields.com/technology

Edit: Don’t forget security! Streaming high resolution sensors over the cloud is a security nightmare.

But 'edge,' as used in context of AI, is also a wink-and-a-nod that the device is inference-only (no learning, no training). The term "inference only" doesn't sound very marketing-friendly.

I've started doing machine learning experiments with it finally. (See [1] for details)

There's a few tricks to getting the best performance. You want to convert your neural network to run with NVIDIA's TensorRT library instead of just tensorflow or torch. TensorRT does all the optimized goodness that gets you the most out of the hardware. Not all possible network operations can run in TensorRT (though nvidia updates the framework regularly). This means some networks can't be easily converted to something fully optimized for this platform. Facebook's detectron2 for example uses some operations that don't readily convert. [2]

But then if you're new like me you've got to both find some code that will ultimately produce something you can convert to TensorRT, and you also need something that you can easily train. I've learned that training using your own dataset is often non-obvious. A lot of example code shows how to use an existing dataset but they totally gloss over the specific label format those datasets use. That means you've got to do some digging to figure out how to make your own dataset load properly in to the training code.

After trying a few different things, I've gotten some good results training using Bonnet(al) [3]. I was able to make enough sense of its training code to use my own dataset, and it looks like it will readily convert to TensorRT. Then you load the converted network using NVIDIA's Deepstream library for maximum pipeline efficiency [4].

The performance numbers for the AGX Xavier are very good, and I am hopeful I will get my application fully operation soon enough.

[1] https://reboot.love/t/new-cameras-on-rover/

[2] https://github.com/facebookresearch/detectron2/issues/192

[3] https://github.com/PRBonn/bonnetal

[4] https://developer.nvidia.com/deepstream-sdk