GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Paper
•
2305.13245
•
Published
•
6
Inference-Optimization
-
-
Weighted Grouped Query Attention in Transformers
Paper • 2407.10855 • Published -
Fast Transformer Decoding: One Write-Head is All You Need
Paper • 1911.02150 • Published • 9 -
MLKV: Multi-Layer Key-Value Heads for Memory Efficient Transformer Decoding
Paper • 2406.09297 • Published • 6 -
Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
Paper • 2405.12981 • Published • 33 -
Effectively Compress KV Heads for LLM
Paper • 2406.07056 • Published • 1 -
The Nature of Mathematical Modeling and Probabilistic Optimization Engineering in Generative AI
Paper • 2410.18441 • Published • 7 -
TransMLA: Multi-head Latent Attention Is All You Need
Paper • 2502.07864 • Published • 57