I mean, someone has to care. If we were to say that that someone should be a third party, we run into huge problems where you and I can both act as that third party, splitting the ecosystem, because our libraries are totally incompatible (you can't even build a crate relying on both of them, with no code!) This is even worse than the situation we have right now.
Like always, I think modifying the coherence rules to enable more blanket impls is the solution here. We should have a base library (std presumably) defining the trait DateTime, and serde should define how to Serialize any T: DateTime, operating against that abstract interface. The most effective strategy for this I know of is mutually exclusive traits.