51–60 of 150 posts
Really cool idea. Unfortunately I can't try it because app crashes for me. I'm guessing it's because of the HN crowd
> The rooms are hosted on a Raspberry Pi 3 lying on the floor of my office.
Hah, probably :D
> Because it uses face rec, you can ONLY show your face, and if you disappear from view your audio will stop after a while, to avoid situations like when you need to go the the restroom but forget to mute. Of course, the real killer app for Zoom calls is the opposite of this: some kind of deep fakery that makes it seem we're there when we're not. Yet as it is, this is a fantastic idea. It's surprising video codecs de…
Then it could look like you are at home in the meeting, but you are out in the forrest walking.
Looking forward to Linux port!
Awesome! A thought: now that neural compression is becoming widespread, it could be a good idea to put some kind of indicator or watermark stating the compression is neural (learned/function approximation in general). I think this would avoid liabilities and criticism around the fact that some weird things may appear (incorrect detail generation), maybe giving a wrong semantic idea. It may also be a good idea to put…
Absolutely. Reminds me of Xerox number mangling:
https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres...
But with this you have the downside of more milliseconds spent compressing and decompressing the frames.
Do you have any indication which effects dominate in typical 4G/wifi environments?
Eventually, a neural-net approach to video codecs is inevitable, as including high-level semantics is much more dense. I wonder about a few things though: • How much of the 8.7MB of the app are the weights? • Did you measure the energy consumption difference between this and H265? Especially considering Apple has hardware acceleration for this. • Do you plan for a Web port as well? • Is the performance envelope tied…
* I did not measure it methodically, but am always careful to not overheat the device when testing (XCode allows you to track this). My main testing device is an iPhone 11, and battery drain does not seem to be an issue compared with e.g. Zoom or Facetime. Where H265 currently wins is when you want to run in higher resolutions, but H265 is not available everywhere, say on slightly older iPads and there is no license included on Windows unless the customer pays separately.
* A WebGPU port would be nice, but I am currently waiting for the APIs to stabilize. If I can find some funding this will be a priority.
* I am not using CoreML but writing my own Metal compute shaders, but am using the "neural" parts of the Apple GPU through other APIs (MPS). I also have support for D3D and OpenCL, but have only tested the latter on teensy Mali GPUs, which at the time did not show impressive performance. On Android my approach would be to target Vulkan now that OpenCL is deprecated, I believe I have most of the plumbing in place, and speculate that things would work on modern mid-to-high end devices.
* When not cutting code, I am working on a plan for enterprise markets. Personally I have found the MacOS version really useful for pair-programming style scenarios, so that could be what I will be going after.
(The reason the MacOS version is still only in beta is because I hit a bug in AVFoundation where capturing from the web camera seems to be burning lots of CPU for absolutely no reason, and I don't want people to come away with the impression that it is my app doing that.)