Ah, I misunderstood. I was looking at swapping the sub-buffer, but the issue was when swapping self. However, it seems to me that this could be fixed by checking whether self has been swapped for a different buffer. playground
self
Yes, padding causes the same issue.