Projects the final hidden state onto vocabulary logits - often reusing the input embedding matrix ('tied weights').
The LM head is a single linear map from the model's hidden width to one logit per vocabulary token; softmax over those logits gives next-token probabilities. Many models tie it to the input embedding matrix, saving hidden×vocab parameters (often several hundred million) at negligible quality cost - common in small and mid-size models, less so at the largest scales.
Open any of these on hfviewer to find this block in the interactive architecture graph.
hfviewer renders the full architecture of 2,300+ Hugging Face models as interactive graphs - hover any block to see what it does, with this model's real numbers.
Browse all model graphs →