When Is a Task Vector Enough? An Empirical Theory of Implicit Multimodal ICL

Implicit multimodal in-context learning compresses demonstrations into internal interventions, ranging from static task vectors to query-conditioned transformations and attention routing. Despite their common goal, these methods differ substantially in how the intervention depends on the query and w...