This sub and blog post are all talking about ChatGPT which is no doubt amazing and far ahead of the curve. However I would also point to Metas new vision model:
SAM (Segment Anything): it is so far beyond any other vision model, I actually believe vision will be solved in a few years now. People don’t realize that there was an industry of publishing paper with incremental improvements in small datasets in CVPR that has been completely invalidated by this paper. I’ve seen engineers in Cruise segmentation team, say Metas new model seems to work better than the in house models they developed and that they should build on top of this. I’ve worked in Tesla Autopilot before and saw it hit a mannequin because we never had mannequin in our dataset before ( we might have had it in the data but it was not a part of our ontology for the network to predict). One approach to mitigate this was OpenAI’s clip that used the English Language as classification labels but Meta’s SAM is so much better where it detects objects without need to specify language. It just understands scenes and objects, at a fundamental level, it can detect anything in a picture if you prompt it right. Honestly it feels a lot like GPT1 which was also ignored by most. If you prompt it right you can get it to segment anything in an image, but prompting it right requires human input. However I can imagine the third or fourth version, with some RL sprinkled in just working zero-shot on complete pixel understanding of any image in the world. This was one of the holy grail of computer vision, that we are seeing solved right in front of our eyes