RECLIP-RO-ViT matches the SOTA performance of RO-ViT on mask APr and outperforms RO-ViT by 0.7 on all-category average precision for open vocabulary detection, showing that its representation is also suitable for standard detection on the base categories.

RECLIP with small images saves 3 ~ 59x compute resources in cores x hours, and achieves competitive performance on zero-shot ImageNet classification and image-text retrieval when compared to SOTA methods.

RECLIP with small images shows attractive trade-offs between the resource use and zero-shot retrieval and image classification performance when compared to the baseline method.

In the first phase, we leverage small images which contain sufficient visual concepts with paired texts as the input to the image and text encoders. In the second phase, we perform high resolution fine-tuning. We improve representation of the images and texts further.

RECLIP utilizes small images during the major training to reduce computation and leverage a brief fine-tuning stage at the end of training to adapt for high-resolution inference.

Our paper RECLIP has been accepted by the TMLR. We introduce a simple method designed to make CLIP more affordable and reproducible for the community. Authors: @runzeli047, Dahun Kim, @weichengkuo, Bir Bhanu. @GoogleDeepMind