Internal references as a separate type

Would there be a practical difference between storing an offset (as suggested in this thread) vs. storing a reference, that gets recalculated whenever the struct is moved?

In my mind, the latter approach perfoms theoretically better, if the number of struct moves times the number of self-referential fields in the struct is less than the number of self-referential field accesses. Practically, I don't know if the impact of the additional addition involved when calculating the address has any measurable impact on performance.

If not, the only difference might be when it comes to implementing such a feature and one might be easier to do than the other, but my knowledge about the internals of Rust is nonexistent.