pytorch
Counter-based random numbers: why the same seed gives different weights on different GPUs Lesson
A counter-based generator such as Philox computes each random number from a key and a position, which is why parallel GPUs can generate random weights fast, and why the chip's layout can leave a fingerprint on a model's starting values.
Google’s TorchTPU offers a new PyTorch route to its accelerators News
Google’s TorchTPU provides a PyTorch backend for TPUs as PyTorch/XLA moves toward archival, without establishing a universal cost or performance win.
neuraloperator Tool
The open-source PyTorch library for Fourier neural operators and related architectures -- the toolkit behind FourCastNet and the plasma and lithography surrogates. If you want to try learning a solution operator for a PDE instead of solving it step by step, this is the reference implementation.
TorchTPU Tool
Google’s Apache-licensed PyTorch backend for TPUs includes compilation integration and distributed-training support; migration suitability depends on the workload.