Adhiraj Ghosh

I am a second-year MSc student in Machine Learning at the University of Tübingen. I focus my research on multimodal deep learning, especially in the context of Vision-Language Representation Learning. Currently, I am at Bethge Lab, working on the holistic understanding of Vision-Language models. Previously, I worked on visualising figurative speech at the Computer Graphics Group, which led to an Outstanding Paper award at EMNLP 2023.

Before starting my master's, I used to be a Computer Vision Researcher at the Center of Artificial Intelligence,ZHAW, working on domain adaptation in Optical Music Recognition. I have also worked with Dr. Daniel Lin Wen-Yan at SMU on feature correspondence-based object tracking. I completed my Bachelor's in Electrical and Electronics Engineering from Manipal, India.

I am very eager to collaborate on relevant projects, so please reach out if you are interested!

Email  /  CV  /  Google Scholar  /  Github  /  LinkedIn  /  Twitter  /  YouTube

profile photo

Recent News

Apr 2023 : New paper on arXiv. Check out coverage by Computerphile and AI 'N Stuff!
Dec 2023 : ViPE awarded outstanding paper at EMNLP!
Oct 2023 : 1 paper accepted at EMNLP 2023(main conference).
Sep 2023 : Work on Real World Music Object Recognition published in TISMIR.
Mar 2023 : Started working in the Tubingen AI Centre in Dr. Hendrik Lensch's group.
Oct 2022 : Moved to Germany! Started my MSc at the University of Tübingen.
Aug 2022 : RPTM accepted for oral presentation at WACV 2023. Check out the paper and SOTA comparisons!

Work Experience

Mar 2023 - Sep 2023: Research Assistant at the Computer Graphics group, Tübingen AI Center.
May 2021 - Aug 2022: Scientific Research Assistant, Zürich University of Applied Sciences.
Jan 2020 - Dec 2020: Visiting Researcher, Singapore Management University
Jun 2018 - Aug 2019 : Undergraduate Research Intern, Jadavpur University.

No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance
Vishaal Udandarao*, Ameya Prabhu*, Adhiraj Ghosh, Yash Sharma, Philip H.S. Torr, Adel Bibi, Samuel Albanie, Matthias Bethge.
arXiv:2404.04125, 2024
Paper / Code / Let It Wag! Benchmark

The impressive empirical performance of VLMs is attributed to test concepts within their pretraining datasets, thus not showcasing "zero-shot" generalization. Instead, they need exponentially more data on a concept to linearly improve performance.

ViPE: Visualise Pretty-much Everything
Hassan Shahmohammadi, Adhiraj Ghosh, Hendrik Lensch.
EMNLP 2023 (Outstanding Paper Award)
Paper / Code / Dataset / HuggingFace / Music Videos

ViPE is the first automated model for translating any arbitrary piece of text into a visualisable prompt. It helps any text-to-image model in figurative or non-lexical language visualisations.

Real World Music Object Recognition
Lukas Tuggener*, Raphael Emberger*, Adhiraj Ghosh*, Pascal Sager*, et al.
Paper / Code

We present solutions to improve recognition accuracy in Music Object Recognition on low-quality, real-world music sheet data and provide confidence-rated model outputs to enable efficient human post-processing.

Relation Preserving Triplet Mining for Stabilising the Triplet Loss in Re-identification Systems
Adhiraj Ghosh, Kuruparan Shanmugalingam, Wen-Yan Lin
WACV 2023
Paper / Code / Video / Poster

We propose a new, feature-guided triplet mining scheme for understanding intrinsic pose to solve the intra-class variance problem in re-identification datasets.

Irony Detection in Bengali Tweets: A New Dataset, Experimentation and Results
Adhiraj Ghosh, Kamal Sarkar
Paper / Dataset

This paper presents the description of the Bengali irony detection dataset developed by us and reports results obtained on our Bengali irony dataset using SOTA machine learning methodologies.