https://github.com/opendatacube/odc-geo is a prototype library for geometry-related operations on xarray objects, I believe including CRS management.
#27 is making xstac a bit more flexible, but I really don't want to have to worry about this.