Looks very cool, but it feels like there's a big gap between Evaluation (online only, low API call limit) and Enterprise ("contact us") pricing models. Not sure what market you're ultimately going for, but right now it seems to defeat the point of providing a nice simple API if it's only usable for either throwaway projects or by very large customers.
Show HN: 3D sensing SDK for iOS – Produces point cloud and 6DOF device motion
11–20 of 21 posts
Re: Show HN: 3D sensing SDK for iOS – Produces point cloud and 6DOF device motion
#12Very cool! Can you give a brief overview of the underlying technology you use to extract the distance data back out from a 2D image?
For the vision part, you start by extracting interest points in all images (Harris keypoints, or SIFT, or similar), then you match them up by using local patch descriptors, (a reasonable implementation in OpenCV for example is the Lucas Kanade Optical Flow tracker) and once you have the correspondences you can estimate a relative 3D camera transformation that explains the motion. In this case the problem is hard because the depth of every point is unknown in addition to the camera transform.
For the IMU stream you can use the accelerometer and gyro in the camera which gives you an estimate for both linear and rotational acceleration. These can be integrated over time to get a reasonable guess for the camera transformations from one time point to another as well.
You combine the two guesses (from vision and from the phone inertial measurement units) into a best guess, and then combine that in addition with the best guess from 30 milliseconds ago to arrive at an evolving probability distribution of this best guess over time. Standard way would be something like a Kalman Filter.
Another issue is dealing with drift over time, as errors in estimation build up and if you're scanning the same area your model will start to drift. This requires something called "Loop Closure" which optimizes the camera matrices across the entire duration of scan and not only frame to frame. This is very computational intensive and hard to do online and without it scans for longer than few seconds will get progressively uglier and misaligned.
This stuff is super tricky to get right. Also, be skeptical of these demos because they are easy to can. It's fairly easy to get that one shot where it looks like it works, but in practice these are exceptionally fragile and very very difficult to get working. Though I'm impressed it seemed to work okay inside the mall -- with all the specular reflections from the floor. Though I'd guess that if anyone placed a foot into the field of vision (and made the environment geometry nonstatic) it would all break :) Good luck to the team though!
Re: Show HN: 3D sensing SDK for iOS – Produces point cloud and 6DOF device motion
#13Looks very cool, but it feels like there's a big gap between Evaluation (online only, low API call limit) and Enterprise ("contact us") pricing models. Not sure what market you're ultimately going for, but right now it seems to defeat the point of providing a nice simple API if it's only usable for either throwaway projects or by very large customers.
You make a great point, and we certainly don't want to exclude the in-between cases. We're still figuring out what those pricing tiers might look like, so if you've got an application that you're excited about, just let us know and we'll figure out a way to make it work!
Re: Show HN: 3D sensing SDK for iOS – Produces point cloud and 6DOF device motion
#14Very cool! Can you give a brief overview of the underlying technology you use to extract the distance data back out from a 2D image?
These are usually implemented using Structure from Motion techniques, but more specifically in this case SLAM (Simultaneous Localization and Mapping). There are two sources of information: from vision and from the Inertial Measurement Units (IMU) on the phone. For the vision part, you start by extracting interest points in all images (Harris keypoints, or SIFT, or similar), then you match them up by using local patch…
Re: Show HN: 3D sensing SDK for iOS – Produces point cloud and 6DOF device motion
#15Earlier quoted context omitted.
You make a great point, and we certainly don't want to exclude the in-between cases. We're still figuring out what those pricing tiers might look like, so if you've got an application that you're excited about, just let us know and we'll figure out a way to make it work!
Really amazing product! Is the form on your site the most expedient way to get in touch/get access to the SDK? Also, is the error accumulation small enough so that you could use the product continuously for extended periods of time?
Re: Show HN: 3D sensing SDK for iOS – Produces point cloud and 6DOF device motion
#16Re: Show HN: 3D sensing SDK for iOS – Produces point cloud and 6DOF device motion
#17Very cool! Can you give a brief overview of the underlying technology you use to extract the distance data back out from a 2D image?
These are usually implemented using Structure from Motion techniques, but more specifically in this case SLAM (Simultaneous Localization and Mapping). There are two sources of information: from vision and from the Inertial Measurement Units (IMU) on the phone. For the vision part, you start by extracting interest points in all images (Harris keypoints, or SIFT, or similar), then you match them up by using local patch…
Apple Developer Videos on Sensor Fusion specifically mention NOT to do this even though their tech uses the Gyroscope which is orders of magnitude more precise than the accelerometer.
I believe it's "Understanding Core Motion." (Developer account required)
Re: Show HN: 3D sensing SDK for iOS – Produces point cloud and 6DOF device motion
#18Very cool! Can you give a brief overview of the underlying technology you use to extract the distance data back out from a 2D image?
I'm fairly sure that the tech is based around "structure from motion". The API simultaneously estimates the position of the camera at each point in time, and the location of some reference points (blue circles in the vid). Because the device has an accelerometer, it is even able to extract distances, not just relative distances. I'm actually surprised by this as I always assumed the accelerometer was too noisy to be…
Re: Show HN: 3D sensing SDK for iOS – Produces point cloud and 6DOF device motion
#19Earlier quoted context omitted.
I'm fairly sure that the tech is based around "structure from motion". The API simultaneously estimates the position of the camera at each point in time, and the location of some reference points (blue circles in the vid). Because the device has an accelerometer, it is even able to extract distances, not just relative distances. I'm actually surprised by this as I always assumed the accelerometer was too noisy to be…
Accelerators are worthless to measure distances even if they were super precise. This is because you have to do double integration to get distance and the errors only accumulates. My guess is that they have used algorithms like SIFT to track points in space and estimate only small relative distances from sensors (gyro + accelerators). This is however is very cool. They should make this as an app that can construct 3D…
I never said that a distance scale was obtained by applying double integration to the accelerometer output. I only said that in order to measure absolute distances, as opposed to relative distances, it is necessary to have an accelerometer, since no other data provides an absolute scale. See the other reply by one of the founders for the details.
Re: Show HN: 3D sensing SDK for iOS – Produces point cloud and 6DOF device motion
#20Really impressive! Nice name, video, website! However I would strongly recommend picking only one of your "features", the indoor navigation. If I were you, I'd definitely try to build a business by concentrating only on indoor navigation! Indoor navigation is a huge new area where all the big players are looking for possible partners/acquisitions right now! Overlay-based AR, and the measuring tape demo is a joke comp…