Optimal transport with cost-free transformations for image co-registration

Abstract. An extension of the optimal transport problem is proposed, which includes a family of transformations incurring no transportation costs. This extension improves the co-registration among imagery datasets where transformations such as rotations, displacements and changes of perspective are a natural component of data acquisition. More generally, it provides a strategy for co-registration that blends the robustness of optimal transport with the interpretability of models. The extended optimal transport problem pairs two distributions with minimal additional distortion, while identifying a cost-free, explainable component of the map. A data-driven formulation is developed, as well as a methodology for its numerical solution. The latter complements gradient descent with a game-theory inspired approach, favoring collaborative moves between the cost-free and the unrestricted transformations. Sample validations are provided. The methodology is illustrated through its successful application to matching pairs of both synthetic and real images, which are conceptualized as weighted samples from underlying distributions, and through the determination of treatment effects by co-registering treated and untreated populations in a synthetic example.