Transformers Model Integration Contracts
As a side effect, UMT5's decoder could attend to future tokens.
During training, it could peek at the answer it was supposed to predict.

Source-linked text edition ·
The written, source-linked counterpart to this video.
Published edition1 story