Transformers
Transformers has a captured commit titled use torch.accelerator device APIs so TP loading works again on TPU.
Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
