Megatron LM
Megatron LM has a captured commit titled batch-invariant Mamba decode without a z replay buffer.
Megatron LM: Ongoing research training transformer models at scale.

Source-linked text edition ·
The written, source-linked counterpart to this video.
Published edition1 story