Original article was published by /u/Yuqing7 on Deep Learning
A new research paper, An Image Is Worth 16×16 Words: Transformers for Image Recognition at Scale, has the machine learning community both excited and curious. With Transformer architectures now being extended to the computer vision (CV) field, the paper suggests the direct application of Transformers to image recognition can outperform even the best convolutional neural networks when scaled appropriately. Unlike prior works using self-attention in CV, the scalable design does not introduce any image-specific inductive biases into the architecture.
The paper An Image Is Worth 16×16 Words: Transformers for Image Recognition at Scale is available on OpenReview.