A Relaxed Wasserstein Distance Formulation for Mixtures of Radially Contoured Distributions

Abstract

Recent research proposed a Wasserstein-type distance specifically for Gaussian mixture models (GMMs). To extend this framework to identifiable mixture models with more general elliptically contoured distributions, it is essential that the marginal mixtures’ components are derived from the same distribution family while maintaining the marginal consistency property. In this paper, we introduce a simplified relaxed Wasserstein distance for identifiable mixtures of radially contoured distributions, where components may originate from different families. We demonstrate some properties of this distance and show that its definition does not require marginal consistency. We apply this distance in color transfer tasks and compare its performance with the Wasserstein-type distance for GMMs in an experiment. Our method requires fewer parameters than GMMs and the error of our method is comparable. The color distribution of our output image is more desirable.

Keywords

Optimal transport (OT) / Wasserstein distance / Mixture models / Radial basis functions (RBFs) / Barycenter / Image processing application / 49Q22 / 62H30 / 65K05 / 65K10 / 68U10 / 65D12

References

[1] Agueh M, Carlier G. Barycenters in the Wasserstein space. SIAM J. Math. Anal., 2011, 43(2): 904-924.

[2] Altschuler, J., Weed, J., Rigollet, P.: Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration. In: Advances in Neural Information Processing Systems, vol. 2017, pp. 1965–1975 (2017)

[3] Bauer M, Joshi S, Modin K. Diffeomorphic density matching by optimal information transport. SIAM J. Imag. Sci., 2015, 8(3): 1718-1751.

[4] Boissard E. Simple bounds for convergence of empirical and occupation measures in 1-Wasserstein distance. Electron. J. Probab., 2011, 16: 2296-2333.

[5] Bolley F, Guillin A, Villani C. Quantitative concentration inequalities for empirical measures on non-compact spaces. Probab. Theory Relat. Fields, 2007, 137(3/4): 541-593.

[6] Bonneel N, Rabin J, Peyré G, Pfister H. Sliced and Radon Wasserstein barycenters of measures. J. Math. Imaging Vis., 2015, 51(1): 22-45.

[7] Bottou L, Curtis FE, Nocedal J. Optimization methods for large-scale machine learning. SIAM Rev., 2018, 60(2): 223-311.

[8] Boyd SP. Convex Optimization, 2004. Cambridge, Cambridge University Press.

[9] Chen K, Zhang Y. Optimal transport and Wasserstein barycenter for radially contoured distributions. Anal. Math., 2025, 51: 775-792.

[10] Cuturi, M.: Sinkhorn distances: lightspeed computation of optimal transport. In: Proceedings of the 27th International Conference on Neural Information Processing Systems—Volume 2. NIPS’13, pp. 2292–2300. Curran Associates Inc., Red Hook, NY, USA (2013)

[11] Delon J, Desolneux A. A Wasserstein-type distance in the space of Gaussian mixture models. SIAM J. Imag. Sci., 2020, 13(2): 936-970.

[12] Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B Methodol. 39(1), 1–38 (1977)

[13] Dudley RM. The speed of mean Glivenko-Cantelli convergence. Ann. Math. Stat., 1969, 40(1): 40-50.

[14] Dvurechensky, P., Gasnikov, A., Kroshnin, A.: Computational optimal transport: complexity by accelerated gradient descent is better than by Sinkhorn’s algorithm. In: 35th International Conference on Machine Learning, ICML 2018, vol. 3, pp. 2196–2220 (2018)

[15] Evans, L.C., Gangbo, W.: Differential Equations Methods for the Monge-Kantorovich Mass Transfer Problem, vol. 137. American Mathematical Society, Providence (1999)

[16] Galerne, B., Leclaire, A., Rabin, J.: Semi-discrete optimal transport in patch space for enriching Gaussian textures. In: Geometric Science of Information, vol. 10589, pp. 100–108. Springer, Cham (2017)

[17] Gelbrich M. On a formula for the $L^2$ Wasserstein metric between measures on Euclidean and Hilbert spaces. Math. Nachr., 1990, 147(1): 185-203.

[18] Gómez E, Gómez-Villegas MA, Marín JM. A survey on continuous elliptical vector distributions. Revista Matemática Complutense, 2003, 16(1): 345-361.

[19] Guo, W., Ho, N., Jordan, M.I.: Fast algorithms for computational optimal transport and Wasserstein barycenter. In: Proceedings of Machine Learning Research, vol. 108, pp. 2088–2097 (2020)

[20] Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed. Springer, New York (2009)

[21] Holzmann H, Munk A, Gneiting T. Identifiability of finite mixtures of elliptical distributions. Scand. J. Stat., 2006, 33(4): 753-763.

[22] Kantorovich LV. On the translocation of masses. J. Math. Sci., 2006, 133(4): 1381-1382.

[23] Karlsson, J., Ringh, A.: Generalized Sinkhorn iterations for regularizing inverse problems using optimal mass transport. SIAM J. Imag. Sci. 10(4), 1935–1962 (2017)

[24] Korotin, A., Selikhanovych, D., Burnaev, E.: Neural Optimal Transport (2023). arXiv:2201.12220

[25] Light, W.A., Singh, S.P. Some aspects of radial basis function approximation. In: Approximation Theory, Spline Functions and Applications. pp. 163–190. Springer, Dordrecht (1992)

[26] Lin, T., Ho, N., Jordan, M.I.: On efficient optimal transport: an analysis of greedy and accelerated mirror descent algorithms. In: 36th International Conference on Machine Learning, ICML 2019, vol. 2019, pp. 7024–7033 (2019)

[27] McCann, R.J.: A convexity principle for interacting gases. Advances in Mathematics 128(1), 153–179 (1997)

[28] Monge, G.: Mémoire sur la Théorie des Déblais et des Remblais. De l’Imprimerie Royale, Paris (1781)

[29] Mueller M, Karasev P, Kolesov I, Tannenbaum A. Optical flow estimation for flame detection in videos. IEEE Trans. Image Process., 2013, 22(7): 2786-2797.

[30] Nemirovski A, Juditsky A, Lan G, Shapiro A. Robust stochastic approximation approach to stochastic programming. SIAM J. Optim., 2009, 19(4): 1574-1609.

[31] Rabin, J., Peyré, G., Delon, J., Bernot, M.: Wasserstein barycenter and its application to texture mixing. In: Scale Space and Variational Methods in Computer Vision. Lecture Notes in Computer Science, vol. 6667, pp. 435–446. Springer, Berlin (2012)

[32] Teodoro, A.M.A.M., Almeida, M.S.C., Figueiredo, M.A.T.: Single-frame image denoising and inpainting using Gaussian mixtures. In: ICPRAM 2015—4th International Conference on Pattern Recognition Applications and Methods, Proceedings, vol. 2, pp. 283–288 (2015)

[33] Weed J, Bach F. Sharp asymptotic and finite-sample rates of convergence of empirical measures in Wasserstein distance. Bernoulli, 2019, 25(4A): 2620-2648.

[34] Wu Z, Schaback R. Local error estimates for radial basis function interpolation of scattered data. IMA J. Numer. Anal., 1993, 13(1): 13-27.

[35] Yakowitz SJ, Spragins JD. On the identifiability of finite mixtures. Ann. Math. Stat., 1968, 39(1): 209-214.

[36] Yu G, Sapiro G, Mallat S. Solving inverse problems with piecewise linear estimators: from Gaussian mixture models to structured sparsity. IEEE Trans. Image Process., 2012, 21(5): 2481-2499.

[37] Zhang Y. Reconstruct multiscale functions using different RBFs in different subdomains. Appl. Math. Comput., 2007, 189: 893-901.

[38] Zoran, D., Weiss, Y.: From learning models of natural image patches to whole image restoration. In: 2011 International Conference on Computer Vision, pp. 479–486. IEEE, Barcelona, Spain (2011)

Funding

National Key R&D Program of China(2024YFA1012401)

Science and Technology Commission of Shanghai Municipality(23JC1400501)

Natural Science Foundation of China(12241103)

RIGHTS & PERMISSIONS

Shanghai University