Text-Only vs Vision-Language Models: Idiom Detection Explained

Aspect Text-Only Models Vision-Language Models Idiom Detection Effective for textual idioms using contextual embeddings Struggles with idioms requiring visual interpretation Figurative Meaning Captures contextual nuances in text Often aligns images with literal meanings, missing figurative context Cultural Understanding Limited understanding of cultural idioms May capture cultural variations if trained on diverse datasets Multimodal Context Lacks visual grounding Utilizes images to enhance idiomatic meaning comprehension Image Ranking Not applicable Challenging due to multiple valid visual representations of idioms