Earlier quoted context omitted.
Here's an example from this morning. At 10:00 am, a colleague created a ticket with an idea for the music plugin I'm working on: wouldn't it be cool if we could use nod detection (head tracking) to trigger recording? That way, musicians who use our app wouldn't need a foot switch (as a musician, you often have your hands occupied). Yes, that would be cool. An hour later, I shipped a release build with that feature fu…
I appreciate this example. This does seem like a pretty difficult feature to build de novo. Did you already have some machine vision work integrated into your app? How are you handling machine vision? Is it just a call to an LLM API? Or are you doing it with a local model?
Despite this being "easier" than some of the alternatives, it is nonetheless an API I have zero experience with, and the implementation was built with code that I would have no idea how to write, although once written, I can get the gist. Here is the "detectNodWithPitch" function as an example (that's how a "nod" is detected - the pitch of the face is determined, and then the change of pitch is what is considered a nod, of course, this is not entirely straightforward).
```
- (void)detectNodWithPitch:(float)pitch { // Get sensitivity-adjusted threshold // At sensitivity 0: threshold = kMaxThreshold degrees (requires strong nod) // At sensitivity 1: threshold = kMaxThreshold - kThresholdRange degrees (very sensitive) float sens = _cppOwner->getSensitivity(); float threshold = NodDetectionConstants::kMaxThreshold - (sens * NodDetectionConstants::kThresholdRange);
// Debounce check
NSTimeInterval now = [NSDate timeIntervalSinceReferenceDate];
if (now - _lastNodTime 0.0f) ? std::min(delta / kMaxDisplayDelta, 1.0f) : 0.0f;
_cppOwner->setNodProgress(progress);
if (!_nodStarted)
{
_cppOwner->setNodInProgress(false);
// Check if nod is starting (head tilting down past nod start threshold)
if (delta > threshold * NodDetectionConstants::kNodStartFactor)
{
_nodStarted = YES;
_maxPitchDelta = delta;
_cppOwner->setNodInProgress(true);
DBG("HeadNodDetector: Nod started, delta=" threshold)
{
DBG("HeadNodDetector: Nod detected! maxDelta=" handleNodDetected();
}
else
{
DBG("HeadNodDetector: Nod too weak, maxDelta=" setNodInProgress(false);
_cppOwner->setNodProgress(0.0f);
}
}@end
```