While this is true, having a const vs mut distinction is helpful for tracking provenance information around mutability, particularly in cases where there's a low-level (e.g. -sys) API that takes pointers and you're writing a higher-level binding that exposes a safe API and are converting & vs &mut references to pointers in the process.
And as others have mentioned, they differ in variance. You might want to peruse this thread: