When using the Stage Manager's “One at a Time” feature, only the currently focused window is displayed. While I don't understand the underlying principle behind it, I've experienced performance benefits when using apps like Photoshop.
If an option enabling such functionality were added to yabai, I believe it would allow for more diverse customization without needing to enable Stage Manager.