Eleanor Saitta on Nostr: Ted M. Young Floating point math is lossy and thus non-associative. Modern GPUs have ...

Ted M. Young (npub1l0q…7r2w) Floating point math is lossy and thus non-associative. Modern GPUs have complex optimizers running and will change the order in which they do e.g. matrix multiplication depending on the layout of data, what else is happening on the GPU, etc. When you iterate repeatedly, results diverge.