Lorenzo Agnolucci

2025

1. ICLR

   Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion

   M. Mistretta*, A. Baldrati*, L. Agnolucci*, M. Bertini, and A. D. Bagdanov

   In The Thirteenth International Conference on Learning Representations, 2025

   Abs arXiv Bib PDF Code Website

   Pre-trained multi-modal Vision-Language Models like CLIP are widely used off-the-shelf for a variety of applications. In this paper, we show that the common practice of individually exploiting the text or image encoders of these powerful multi-modal models is highly suboptimal for intra-modal tasks like image-to-image retrieval. We argue that this is inherently due to the CLIP-style intermodal contrastive loss that does not enforce any intra-modal constraints, leading to what we call intra-modal misalignment. To demonstrate this, we leverage two optimization-based modality inversion techniques that map representations from their input modality to the complementary one without any need for auxiliary data or additional trained adapters. We empirically show that, in the intra-modal tasks of image-to-image and text-to-text retrieval, approaching these tasks inter-modally significantly improves performance with respect to intramodal baselines on more than fifteen datasets. Additionally, we demonstrate that approaching a native inter-modal task (e.g. zero-shot image classification) intra-modally decreases performance, further validating our findings. Finally, we show that incorporating an intra-modal term in the pre-training objective or narrowing the modality gap between the text and image feature embedding spaces helps reduce the intra-modal misalignment. The code is publicly available at: https://github.com/miccunifi/Cross-the-Gap.

   @inproceedings{mistretta2025cross,
     title = {{Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion}},
     author = {Mistretta*, M. and Baldrati*, A. and Agnolucci*, L. and Bertini, M. and Bagdanov, A. D.},
     booktitle = {The Thirteenth International Conference on Learning Representations},
     year = {2025},
   }

2024

1. WACV Oral

   ARNIQA: Learning Distortion Manifold for Image Quality Assessment

   L. Agnolucci, L. Galteri, M. Bertini, and A. Del Bimbo

   In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024

   Abs arXiv Bib PDF Code

   No-Reference Image Quality Assessment (NR-IQA) aims to develop methods to measure image quality in alignment with human perception without the need for a high-quality reference image. In this work, we propose a self-supervised approach named ARNIQA (leArning distoRtion maNifold for Image Quality Assessment) for modeling the image distortion manifold to obtain quality representations in an intrinsic manner. First, we introduce an image degradation model that randomly composes ordered sequences of consecutively applied distortions. In this way, we can synthetically degrade images with a large variety of degradation patterns. Second, we propose to train our model by maximizing the similarity between the representations of patches of different images distorted equally, despite varying content. Therefore, images degraded in the same manner correspond to neighboring positions within the distortion manifold. Finally, we map the image representations to the quality scores with a simple linear regressor, thus without fine-tuning the encoder weights. The experiments show that our approach achieves state-of-the-art performance on several datasets. In addition, ARNIQA demonstrates improved data efficiency, generalization capabilities, and robustness compared to competing methods. The code and the model are publicly available at https://github.com/miccunifi/ARNIQA.

   @inproceedings{agnolucci2024arniqa,
     title = {ARNIQA: Learning Distortion Manifold for Image Quality Assessment},
     author = {Agnolucci, L. and Galteri, L. and Bertini, M. and Del Bimbo, A.},
     booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
     year = {2024},
   }

2023

1. ICCV

   Zero-Shot Composed Image Retrieval with Textual Inversion

   A. Baldrati*, L. Agnolucci*, M. Bertini, and A. Del Bimbo

   In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023

   Abs arXiv Bib PDF Code Website

   Composed Image Retrieval (CIR) aims to retrieve a target image based on a query composed of a reference image and a relative caption that describes the difference between the two images. The high effort and cost required for labeling datasets for CIR hamper the widespread usage of existing methods, as they rely on supervised learning. In this work, we propose a new task, Zero-Shot CIR (ZS-CIR), that aims to address CIR without requiring a labeled training dataset. Our approach, named zero-Shot composEd imAge Retrieval with textuaL invErsion (SEARLE), maps the visual features of the reference image into a pseudo-word token in CLIP token embedding space and integrates it with the relative caption. To support research on ZS-CIR, we introduce an open-domain benchmarking dataset named Composed Image Retrieval on Common Objects in context (CIRCO), which is the first dataset for CIR containing multiple ground truths for each query. The experiments show that SEARLE exhibits better performance than the baselines on the two main datasets for CIR tasks, FashionIQ and CIRR, and on the proposed CIRCO. The dataset, the code and the model are publicly available at https://github.com/miccunifi/SEARLE.

   @inproceedings{baldrati2023zero,
     title = {Zero-Shot Composed Image Retrieval with Textual Inversion},
     author = {Baldrati*, A. and Agnolucci*, L. and Bertini, M. and Del Bimbo, A.},
     booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
     pages = {15338--15347},
     year = {2023},
   }