11–20 of 45 posts
Offtopic sort of, but does anyone know if folks are working on combining vision and natural language in one model? I think that could wield some interesting results.
The results are quite interesting:
https://www.reddit.com/r/Art/comments/p866wv/deep_dive_meai_...
Does tesla use transformers for the auto pilot?
For example, if it is easy to fool them with optical illusions, such as innocent images that look racy at the first glance: https://medium.com/@marekkcichy/does-ai-have-a-dirty-mind-to... CW: Even though it does not contain a single explicit picture, it might be considered NSFW (literally - as at the first glance it looks like nudity); full disclosure: I mentored the project.
Does tesla use transformers for the auto pilot?
See: https://youtu.be/j0z4FweCy4M (timestamp 54.40 onwards)
Does tesla use transformers for the auto pilot?
Does tesla use transformers for the auto pilot?
Doubtful. The biggest downside of transformers for vision is how ungodly long they take to produce results. Tesla has to operate in realtime.
But not on the raw camera input — they use regnets for that. The transformers come higher up the stack:
Transformers mentioned on the slide at timestamp 1:00:18.
Does tesla use transformers for the auto pilot?
They do. Karpathy spoke about it in on Tesla AI day. They use it for transforming image space to a vector space. See: https://youtu.be/j0z4FweCy4M (timestamp 54.40 onwards)
I'd be really surprised if they use transformers due to how computationally expensive they are for anything involving vision.
EDIT: Found it. 1h: https://www.youtube.com/watch?v=j0z4FweCy4M?t=1h
Fascinating. I guess transformers are efficient.
I am much more interested if they fall for the same tricks. For example, if it is easy to fool them with optical illusions, such as innocent images that look racy at the first glance: https://medium.com/@marekkcichy/does-ai-have-a-dirty-mind-to... CW: Even though it does not contain a single explicit picture, it might be considered NSFW (literally - as at the first glance it looks like nudity); full disclosure: I men…
In contrast, attention mechanisms in transformers can be seen as taking into consideration dense graphs of the whole input (at least in text, I haven't really worked with vision transformers but if an attention mechanism exists then it should be similar), along with some positional encoding and a neighborhood summary.
If they indeed can be thought as stacking neighborhood summaries along with attention mechanisms, then they shouldn't fall for the same tricks since they have access to "global" information instead of disconnected components.
But take this reply with a grain of salt as I am still learning about Geometric DL. If I misunderstood something, please correct me.
A useful HN feature would be small space to put in a summary, like the abstract: Convolutional neural networks (CNNs) have so far been the de-facto model for visual data. Recent work has shown that (Vision) Transformer models (ViT) can achieve comparable or even superior performance on image classification tasks. This raises a central question: how are Vision Transformers solving these tasks? Are they acting like con…
I am much more interested if they fall for the same tricks. For example, if it is easy to fool them with optical illusions, such as innocent images that look racy at the first glance: https://medium.com/@marekkcichy/does-ai-have-a-dirty-mind-to... CW: Even though it does not contain a single explicit picture, it might be considered NSFW (literally - as at the first glance it looks like nudity); full disclosure: I men…
I suggest you take a look at Geometric Deep Learning, but the gist here is that convolutions can be thought of as translation equivariant functions, and pooling operations as permutation invariant combinations, all operating on a graph of as many components as the number of times the operation will be output, each component is composed of the pixels that the operation will act on, so there is local information that i…