Queries come from one sequence, keys/values from another - how a decoder reads an encoder's output.
In cross-attention the queries belong to the current stream (say, generated text or noisy image latents) while keys and values come from a different one (encoder output, a text prompt, audio features). It is the bridge in encoder-decoder models like Whisper and the conditioning mechanism in diffusion U-Nets, where image latents cross-attend into the text prompt's embeddings.
Open any of these on hfviewer to find this block in the interactive architecture graph.
hfviewer renders the full architecture of 2,300+ Hugging Face models as interactive graphs - hover any block to see what it does, with this model's real numbers.
Browse all model graphs →