Optimization methods for regularization-based ill-posed problems: a survey and a multi-objective framework

Abstract

Ill-posed problems are widely existed in signal processing. In this paper, we review popular regularization models such as truncated singular value decomposition regularization, iterative regularization, variational regularization. Meanwhile, we also retrospect popular optimization approaches and regularization parameter choice methods. In fact, the regularization problem is inherently a multiobjective problem. The traditional methods usually combine the fidelity term and the regularization term into a singleobjective with regularization parameters, which are difficult to tune. Therefore, we propose a multi-objective framework for ill-posed problems, which can handle complex features of problem such as non-convexity, discontinuity. In this framework, the fidelity term and regularization term are optimized simultaneously to gain more insights into the ill-posed problems. A case study on signal recovery shows the effectiveness of the multi-objective framework for ill-posed problems.

Keywords

ill-posed problem / regularization / multiobjective optimization / evolutionary algorithm / signal processing

References

[1] HadamardJ. Sur les Problemes aux Derivees Partielles et Leur Signification Physique. Princeton University Bulletin, 1902, 13: 49–52

[2] KabanikhinS I. Inverse and Ill-Posed Problems: Theory and Applications. Berlin: Water De Gruyter, 2011

[3] ZhangB Y, XuD H, LiuT W. Stabilized algorithms for ill-posed problems in signal processing. In: Proceedings of the IEEE International Conferences on Info-tech and Info-net. 2001, 1: 375–380

[4] ScherzerO. Handbook of Mathematical Methods in Imaging. Springer Science & Business Media, 2011

[5] GroetschC W. Inverse problems in the mathematical sciences. Mathematics of Computation, 1993, 63(5): 799–811

[6] RudinL I, OsherS, FatemiE. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 1992, 60(1): 259–268

[7] TikhonovA N. Solution of incorrectly formulated problems and the regularization method. Soviet Math, 1963, 4: 1035–1038

[8] TikhonovA N, Arsenin V I. Solutions of Ill-posed Problems. Washington, DC: V. H. Winston & Sons, 1977

[9] LandweberL. An iteration formula for Fredholm integral equations of the first kind. American Journal of Mathematics, 1951, 73(3): 615–624

[10] HestenesM R, Stiefel E. Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 1952, 49(6): 409–436

[11] VogelC R. Computational Methods for Inverse Problems. Society for Industrial and Applied Mathematics, 2002, 23

[12] HansenP C. The truncated SVD as a method for regularization. Bit Numerical Mathematics, 1987, 27(4): 534–553

[13] HonerkampJ, WeeseJ. Tikhonovs regularization method for ill-posed problems. Continuum Mechanics and Thermodynamics, 1990,2(1): 17–30

[14] ZhangX Q, BurgerM, BressonX, Osher S. Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM Journal on Imaging Sciences, 2010, 3(3): 253–276

[15] DebK. Multi-Objective Optimization Using Evolutionary Algorithms. New York: John Wiley & Sons, 2001, 16

[16] FonsecaC M, Fleming P J. An overview of evolutionary algorithms in multiobjective optimization. Evolutionary Computation, 1995, 3(1): 1–16

[17] CoelloC A C, Van Veldhuizen D A, LamontG B . Evolutionary Algorithms for Solving Multi-objective Problems. New York: Kluwer Academic, 2002

[18] TanK C, KhorE F, LeeT H. Multiobjective Evolutionary Algorithms and Applications. Springer Science & Business Media, 2005

[19] KnowlesJ, CorneD, DebK. Multiobjective Problem Solving from Nature: from Concepts to Applications. Springer Science & Business Media, 2008

[20] RaquelC, YaoX. Dynamic multi-objective optimization: a survey of the state-of-the-art. In: YangS X , YaoX, eds. Evolutionary Computation for Dynamic Optimization Problems. Springer Berlin Heidelberg, 2013, 85–106

[21] LückenC V, Barán B, BrizuelaC . A survey on multi-objective evolutionary algorithms for many-objective problems. Computational Optimization and Applications, 2014, 58(3): 707–756

[22] HwangC L, MasudA S M.Multiple Objective Decision Making- Methods and Applications. Springer Science & Business Media, 1979, 164

[23] GirosiF, JonesM B, PoggioT. Regularization theory and neural networks architectures. Neural Computation, 1995, 7(2): 219–269

[24] BelgeM, KilmerM E, MillerE L. Efficient determination of multiple regularization parameters in a generalized L-curve framework. Inverse Problems, 2002, 18(4): 1161

[25] HansenP C. Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion. American Mathematical Monthly, 1997, 4(5): 491

[26] ErikssonP, Jiménez C, BuehlerS A . Qpack, a general tool for instrument simulation and retrieval work.Journal of Quantitative Spectroscopy and Radiative Transfer, 2005, 91(1): 47–64

[27] GiustiE. Minimal Surfaces and Functions of Bounded Variation. Springer Science & Business Media, 1984, 80

[28] CattéF, CollT.Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis, 1992, 29(1): 182–193

[29] BjörckA. Numerical Methods for Least Squares Problems. Society for Industrial and Applied Mathematics, 1996

[30] GroetschC W. The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind. Pitman Advanced Publishing Program, 1984

[31] HansonR J. A numerical method for solving Fredholm integral equationstific and Statistical Computing, 1992, 13(5): 1142–1150

[32] StewartG W. Rank degeneracy. SIAM Journal on Scientific and Statistical Computing, 1984, 5(2): 403–413

[33] HansenP C, SekiiT, ShibahashiH . The modified truncated SVD method for regularization in general form. SIAM Journal on Sciendependent Component Analysis and Blind Source Separation. 2006, 206–213

[34] Van LoanC F. Generalizing the singular value decomposition. SIAM Journal on Numerical Analysis, 1976, 13(1): 76–83

[35] HansenP C.Regularization, GSVD and truncated GSVD. BIT Numerical Mathematics, 1989, 29(3): 491–504

[36] PaigeC C. Computing the generalized singular value decomposition. SIAM Journal on Scientific and Statistical Computing, 1986, 7(4): 1126–1146

[37] MorigiS, Reichel L, SgallariF . A truncated projected SVD method for linear discrete ill-posed problems. Numerical Algorithms, 2006, 43(3): 197–213

[38] FernandoK V, Hammarling S. A product induced singular value decomposition (ΠSVD) for two matrices and balanced realization. In: Proceedings of SIAM Conference on Linear Algebra in Signals, Systems and Control. 1988, 128–140

[39] ZhaH Y. The restricted singular value decomposition of matrix triplets. SIAM Journal on Matrix Analysis and Applications, 1991, 12(1): 172–194

[40] De MoorB, GolubG H. The restricted singular value decomposition: properties and applications. SIAM Journal on Matrix Analysis and Applications, 1991, 12(3): 401–425

[41] De MoorB, ZhaH Y. A tree of generalizations of the ordinary singular value decomposition. Linear Algebra and Its Applications, 1991, 147: 469–500

[42] De MoorB. Generalizations of the OSVD: structure, properties and applications. In: VaccaroR J, ed.SVD & Signal Processing, II: Algorithms, Analysis & Applications. 1991, 83–98

[43] NoscheseS, Reichel L. A modified TSVD method for discrete illposed problems. Numerical Linear Algebra with Applications, (in press)

[44] DykesL, Noschese S, ReichelL . Rescaling the GSVD with application to ill-posed problems. Numerical Algorithms, 2015, 68(3): 531–545

[45] EdoL, FrancoW, MartinssonP G , RokhlinV, TygertM. Randomized algorithms for the low-rank approximation of matrices. Proceedings of the National Academy of Sciences, 2007, 104(51): 20167–20172

[46] WoolfeF, Liberty E, RokhlinV , TygertM. A fast randomized algorithm for the approximation of matrices. Applied & Computational Harmonic Analysis, 2008, 25(3): 335–366

[47] SifuentesJ, Gimbutas Z, GreengardL . Randomized methods for rankdeficient linear systems. Electronic Transactions on Numerical Analysis, 2015, 44: 177–188

[48] LiuY G, LeiY J, LiC G, Xu W Z, PuY F . A random algorithm for low-rank decomposition of large-scale matrices with missing entries. IEEE Transactions on Image Processing, 2015, 24(11): 4502–4511

[49] SekiiT. Two-dimensional inversion for solar internal rotation. Publications of the Astronomical Society of Japan, 1991, 43: 381–411

[50] ScalesJ A. Uncertainties in seismic inverse calculations. In: Jacobsen B H, MosegaardK , SibaniP, eds. Inverse Methods. Berlin: Springer- Verlag, 1996, 79–97

[51] LawlessJ F, WangP. A simulation study of ridge and other regression estimators. Communications in Statistics-Theory and Methods, 1976, 5(4): 307–323

[52] DempsterA P, Schatzoff M, WermuthN . A simulation study of alternatives to ordinary least squares. Journal of the American Statistical Association, 1977, 72(357): 77–91

[53] HansenP C, O’Leary D P. The use of the L-curve in the regularization of discrete ill-posed problems. SIAM Journal on Scientific Computing, 1993, 14(6): 1487–1503

[54] HansenP C. Analysis of discrete ill-posed problems by means of the L-curve. SIAM Review, 1992, 34(4): 561–580

[55] XuP L. Truncated SVD methods for discrete linear ill-posed problems. Geophysical Journal International, 1998, 135(2): 505–514

[56] WuZ M, BianS F, XiangC B, Tong Y D. A new method for TSVD regularization truncated parameter selection. Mathematical Problems in Engineering, 2013

[57] ChiccoD, Masseroli M. A discrete optimization approach for SVD best truncation choice based on ROC curves. In: Proceedings of the 13th IEEE International Conference on Bioinformatics and Bioengineering. 2013: 1–4

[58] GolubG H, HeathM, WahbaG. Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics, 1979, 21(2): 215–223

[59] JbilouK, Reichel L, SadokH . Vector extrapolation enhanced TSVD for linear discrete ill-posed problems. Numerical Algorithms, 2009, 51(2): 195–208

[60] BouhamidiA, JbilouK, ReichelL, Sadok H, WangZ . Vector extrapolation applied to truncated singular value decomposition and truncated iteration. Journal of Engineering Mathematics, 2015, 93(1): 99–112

[61] VogelC R. Computational Methods for Inverse Problems. Society for Industrial and Applied Mathematics, 2002

[62] DoicuA, Trautmann T, SchreierF . Numerical Regularization for Atmospheric Inverse Problems. Springer Science & Business Media, 2010

[63] BakushinskyA B, Goncharsky A V. Iterative Methods for the Solution of Incorrect Problems. Moscow: Nauka, 1989

[64] RiederA. Keine Probleme mit Inversen Problemen: Eine Einführung in ihre stabile Lösung. Berlin: Springer-Verlag, 2013

[65] NemirovskiyA S, PolyakB T.Iterative methods for solving linear illposed problems under precise information. Engineering Cybernetics, 1984, 22(4): 50–56

[66] BrakhageH. On ill-posed problems and the method of conjugate gradients. Inverse and Ill-posed Problems, 1987, 4: 165–175

[67] HankeM. Accelerated Landweber iterations for the solution of illposed equations. Numerische Mathematik, 1991, 60(1): 341–373

[68] BarzilaiJ, Borwein J M. Two-point step size gradient methods. IMA Journal of Numerical Analysis, 1988, 8(1): 141–148

[69] AxelssonO. Iterative Solution Methods. Cambridge:Cambridge University Press, 1996

[70] Van der SluisA, Van der Vorst H A. The rate of convergence of conjugate gradients. Numerische Mathematik, 1986, 48(5): 543–560

[71] ScalesJ A, Gersztenkorn A. Robust methods in inverse theory. Inverse Problems, 1988, 4(4): 1071–1091

[72] BjörckÅ, Eldén L. Methods in numerical algebra for ill-posed problems. Technical Report LiTH-MAT-R-33-1979. 1979

[73] TrefethenL N, BauD. Numerical Linear Algebra. Society for Industrial and Applied Mathematics, 1997

[74] CalvettiD, LewisB, ReichelL. On the regularizing properties of the GMRES method. Numerische Mathematik, 2002, 91(4): 605–625

[75] CalvettiD, LewisB, ReichelL. Alternating Krylov subspace image restoration methods. Journal of Computational and Applied Mathematics, 2012, 236(8): 2049–2062

[76] BrianziP, FavatiP, MenchiO, Romani F. A framework for studying the regularizing properties of Krylov subspace methods.Inverse Problems, 2006, 22(3): 1007–1021

[77] SonneveldP, Van Gijzen M B. IDR(s): a family of simple and fast algorithms for solving large nonsymmetric systems of linear equations. SIAM Journal on Scientific Computing, 2008, 31(2): 1035–1062

[78] FongD C L, Saunders M. LSMR: an iterative algorithm for sparse least-squares problems. SIAM Journal on Scientific Computing, 2011, 33(5): 2950–2971

[79] ZhaoC, HuangT Z, ZhaoX L, Deng L J. Two new efficient iterative regularization methods for image restoration problems. Abstract & Applied Analysis, 2013

[80] PerezA, Gonzalez R C. An iterative thresholding algorithm for image segmentation. IEEE Transactions on Pattern Analysis & Machine Intelligence, 1987, 9(6): 742–751

[81] BeckA, Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2009, 2(1): 183–202

[82] Bioucas-DiasJ M, Figueiredo M A T. Two-step algorithms for linear inverse problems with non-quadratic regularization. In: Proceedings of the IEEE International Conference on Image Processing. 2007, 105–108

[83] Bioucas-DiasJ M, Figueiredo M A T. A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Transactions on Image Processing. 2007, 16(12): 2992–3004

[84] BayramI, Selesnick I W. A subband adaptive iterative shrinkage/ thresholding algorithm. IEEE Transactions on Signal Processing, 2010, 58(3): 1131–1143

[85] YamagishiM, YamadaI. Over-relaxation of the fast iterative shrinkage-thresholding algorithm with variable stepsize. Inverse Problems, 2011, 27(10): 105008–105022

[86] BhottoM Z A, AhmadM O, SwamyM N S. An improved fast iterative shrinkage thresholding algorithm for image deblurring. SIAM Journal on Imaging Sciences, 2015, 8(3): 1640–1657

[87] ZhangY D, DongZ C, PhillipsP, Wang S H, JiG L , YangJ Q. Exponential wavelet iterative shrinkage thresholding algorithm for compressed sensing magnetic resonance imaging. Information Sciences, 2015, 322: 115–132

[88] ZhangY D, WangS H, JiG L, Dong Z C. Exponential wavelet iterative shrinkage thresholding algorithm with random shift for compressed sensing magnetic resonance imaging. IEEJ Transactions on Electrical and Electronic Engineering, 2015, 10(1): 116–117

[89] WuG M, LuoS Q. Adaptive fixed-point iterative shrinkage/ thresholding algorithm for MR imaging reconstruction using compressed sensing. Magnetic Resonance Imaging, 2014, 32(4): 372–378

[90] FangE X, WangJ J, HuD F, Zhang J Y, ZouW , ZhouY. Adaptive monotone fast iterative shrinkage thresholding algorithm for fluorescence molecular tomography. IET Science Measurement Technology, 2015, 9(5): 587–595

[91] ZuoW M, MengD Y, ZhangL, Feng X C, ZhangD . A generalized iterated shrinkage algorithm for non-convex sparse coding. In: Proceedings of the IEEE International Conference on Computer Vision. 2013, 217–224

[92] KrishnanD, FergusR. Fast image deconvolution using hyperlaplacian priors. In: BengioY , SchuurmansD, Lafferty J D, et al., eds. Advances in Neural Information Processing Systems 22. 2009, 1033–1041

[93] ChartrandR, YinW. Iteratively reweighted algorithms for compressive sensing. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2008, 3869–3872

[94] SheY Y. An iterative algorithm for fitting nonconvex penalized generalized linear models with grouped predictors. Computational Statistics & Data Analysis, 2012, 56(10): 2976–2990

[95] GongP H, ZhangC S, LuZ S, Huang J Z, YeJ P . A general iterative shrinkage and thresholding algorithm for non-convex regularized optimization problems. In: Proceedings of International Conference on Machine Learning. 2013, 37–45

[96] BrediesK, LorenzD A. Linear convergence of iterative softthresholding.Journal of Fourier Analysis and Applications, 2008, 14(5–6): 813–837

[97] KowalskiM. Thresholding rules and iterative shrinkage/thresholding algorithm: a convergence study. In: Proceedings of the IEEE International Conference on Image Processing. 2014, 4151–4155

[98] ChambolleA, DossalC. On the convergence of the iterates of the “fast iterative shrinkage/thresholding algorithm”. Journal of Optimization Theory & Applications, 2015, 166(3): 968–982

[99] JohnstoneP R, MoulinP. Local and global convergence of an inertial version of forward-backward splitting. Advances in Neural Information Processing Systems, 2014, 1970–1978

[100] MorozovV A. On the solution of functional equations by the method of regularization. Soviet Mathematics Doklady, 1966, 7(11): 414–417

[101] VainikkoG M. The discrepancy principle for a class of regularization methods. USSR Computational Mathematics and Mathematical Physics, 1982, 22(3): 1–19

[102] VainikkoG M. The critical level of discrepancy in regularization methods. USSR Computational Mathematics and Mathematical Physics, 1983, 23(6): 1–9

[103] PlatoR. On the discrepancy principle for iterative and parametric methods to solve linear ill-posed equations. Numerische Mathematik, 1996, 75(1): 99–120

[104] BorgesL S, Bazán F S V, CunhaM C C . Automatic stopping rule for iterative methods in discrete ill-posed problems. Computational & Applied Mathematics, 2015, 34(3): 1175–1197

[105] DziwokiG, Izydorczyk J. Stopping criteria analysis of the OMP algorithm for sparse channels estimation. In: Proceedings of the International Conference on Computer Networks. 2015, 250–259

[106] FavatiP, LottiG, MenchiO, Romani F.Stopping rules for iterative methods in nonnegatively constrained deconvolution. Applied Numerical Mathematics, 2014, 75: 154–166

[107] EnglH W, HankeM, NeubauerA. Regularization of Inverse Problems. Springer Science & Business Media, 1996

[108] AmsterP. Iterative Methods. Universitext, 2014, 53–82

[109] WaseemM. On some iterative methods for solving system of nonlinear equations. Dissertation for the Doctoral Degree.Islamabad: COMSATS Institute of Information Technology, 2012

[110] BurgerM, OsherS. A guide to the TV zoo. In: BurgerM, Mennucci A C G, OsherS , et al., eds. Level Set and PDE Based Reconstruction Methods in Imaging. Springer International Publishing, 2013, 1–70

[111] TikhonovA N. Regularization of incorrectly posed problems. Soviet Mathematics Doklady, 1963, 4(1): 1624–1627

[112] NikolovaM. Analysis of the recovery of edges in images and signals by minimizing nonconvex regularized least-squares. Multiscale Modeling & Simulation, 2005, 4(3): 960–991

[113] BurgerM, OsherS. Convergence rates of convex variational regularization. Inverse Problems, 2004, 20(5): 1411–1421

[114] HofmannB, Kaltenbacher B, PöschlC , ScherzerO. A convergence rates result for Tikhonov regularization in Banach spaces with nonsmooth operators. Inverse Problems, 2007, 23(3): 987–1010

[115] ResmeritaE. Regularization of ill-posed problems in Banach spaces:convergence rates. Inverse Problems, 2005, 21(4): 1303–1314

[116] ResmeritaE, Scherzer O. Error estimates for non-quadratic regularization and the relation to enhancement. Inverse Problems, 2006, 22(3): 801–814

[117] EnglH W. Discrepancy principles for Tikhonov regularization of illposed problems leading to optimal convergence rates. Journal of Optimization Theory and Applications, 1987, 52(2): 209–215

[118] GfrererH. An a posteriori parameter choice for ordinary and iterated Tikhonov regularization of ill-posed problems leading to optimal convergence rates. Mathematics of Computation, 1987, 49(180): 507–522

[119] NattererF. Error bounds for Tikhonov regularization in Hilbert scales. Applicable Analysis, 1984, 18(1–2): 29–37

[120] NeubauerA. An a posteriori parameter choice for Tikhonov regularization in the presence of modeling error. Applied Numerical Mathematics, 1988, 4(6): 507–519

[121] EnglH W, Kunisch K, NeubauerA . Convergence rates for Tikhonov regularisation of non-linear ill-posed problems. Inverse Problems, 1989, 5(4): 523–540

[122] ScherzerO, EnglH W, KunischK. Optimal a posteriori parameter choice for Tikhonov regularization for solving nonlinear ill-posed problems. SIAM Journal on Numerical Analysis, 1993, 30(6): 1796–1838

[123] VarahJ M. On the numerical solution of ill-conditioned linear systems with applications to ill-posed problems. SIAM Journal on Numerical Analysis, 1973, 10(2): 257–267

[124] VinodH D, UllahA. Recent Advances in Regression Methods. Danbury: Marcel Dekker Incorporated, 1981

[125] O’SullivanF. A statistical perspective on ill-posed inverse problems. Statistical Science, 1986, 1(4): 502–518

[126] GrafarendE W, Schaffrin B. Ausgleichungsrechnung in linearen modellen. BI Wissenschaftsverlag Mannheim, 1993

[127] RodgersC D. Inverse Methods for Atmospheric Sounding: Theory and Practice. Singapore: World Scientific, 2000

[128] CeccheriniS. Analytical determination of the regularization parameter in the retrieval of atmospheric vertical profiles. Optics Letters, 2005, 30(19): 2554–2556

[129] MallowsC L. Some comments on C_p. Technometrics, 1973, 15(4): 661–675

[130] RiceJ. Choice of smoothing parameter in deconvolution problems. Contemporary Mathematics, 1986, 59: 137–151

[131] HankeM, RausT. A general heuristic for choosing the regularization parameter in ill-posed problems. SIAM Journal on Scientific Computing, 1996, 17(4): 956–972

[132] WuL M. A parameter choice method for Tikhonov regularization. Electronic Transactions on Numerical Analysis, 2003, 16: 107–128

[133] GaoW, YuK P. A new method for determining the Tikhonov regularization parameter of load identification. In: Proceedings of the International Symposium on Precision Engineering Measurement and Instrumentation. 2015

[134] ItoK, JinB, TakeuchiT. Multi-parameter Tikhonov regularizationan augmented approach. Chinese Annals of Mathematics, Series B, 2014, 35(03): 383–398

[135] JinB, LorenzD A. Heuristic parameter-choice rules for convex variational regularization based on error estimates. SIAM Journal on Numerical Analysis, 2010, 48(3): 1208–1229

[136] PazosF, BhayaA. Adaptive choice of the Tikhonov regularization parameter to solve ill-posed linear algebraic equations via Liapunov optimizing control. Journal of Computational and Applied Mathematics, 2015, 279: 123–132

[137] HämarikU, PalmR, RausT. A family of rules for parameter choice in Tikhonov regularization of ill-posed problems with inexact noise level. Journal of Computational and Applied Mathematics, 2012, 236(8): 2146–2157

[138] ReichelL, Rodriguez G. Old and new parameter choice rules for discrete ill-posed problems. Numerical Algorithms, 2013, 63(1): 65–87

[139] KryanevA V. An iterative method for solving incorrectly posed problems. USSR Computational Mathematics and Mathematical Physics, 1974, 14(1): 24–35

[140] KingJ T, Chillingworth D. Approximation of generalized inverses by iterated regularization. Numerical Functional Analysis & Optimization, 1979, 1(5): 499–513

[141] FakeevA G. A class of iterative processes for solving degenerate systems of linear algebraic equations. USSR Computational Mathematics and Mathematical Physics, 1981, 21(3): 15–22

[142] BrillM, SchockE. Iterative solution of ill-posed problems: a survey. In: Proceedings of the 4th International Mathematical Geophysics Seminar. 1987

[143] HankeM, Groetsch C W. Nonstationary iterated Tikhonov regularization. Journal of Optimization Theory and Applications, 1998, 98(1): 37–53

[144] LampeJ, Reichel L, VossH . Large-scale Tikhonov regularization via reduction by orthogonal projection. Linear Algebra and Its Applications, 2012, 436(8): 2845–2865

[145] ReichelL, YuX B. Tikhonov regularization via flexible Arnoldi reduction. BIT Numerical Mathematics, 2015, 55(4): 1145–1168

[146] HuangG, Reichel L, YinF . Projected nonstationary iterated Tikhonov regularization. BIT Numerical Mathematics, 2016, 56(2): 467–487

[147] AmbrosioL, FuscoN, PallaraD. Functions of Bounded Variation and Free Discontinuity Problems. Oxford: Oxford University Press, 2000

[148] AcarR, VogelC R. Analysis of bounded variation penalty methods for ill-posed problems. Inverse Problems, 1997, 10(6): 1217–1229

[149] HuntB R. The application of constrained least squares estimation to image restoration by digital computer. IEEE Transactions on Computers, 1973, 100(9): 805–812

[150] DemomentG. Image reconstruction and restoration: overview of common estimation structures and problems. IEEE Transactions on Acoustics, Speech and Signal Processing, 1989, 37(12): 2024–2036

[151] KatsaggelosA K. Iterative image restoration algorithms. Optical Engineering, 1989, 28(7): 735–748

[152] KatsaggelosA K, Biemond J, SchaferR W , MersereauR M. A regularized iterative image restoration algorithm. IEEE Transactions on Signal Processing, 1991, 39(4): 914–929

[153] BabacanS D, MolinaR, KatsaggelosA K . Parameter estimation in TV image restoration using variational distribution approximation. IEEE Transactions on Image Processing, 2008, 17(3): 326–339

[154] WenY W, ChanR H. Parameter selection for total-variation-based image restoration using discrepancy principle. IEEE Transactions on Image Processing, 2012, 21(4): 1770–1781

[155] ChenA, HuoB M, WenC W. Adaptive regularization for color image restoration using discrepancy principle. In: Proceedings of the IEEE International Conference on Signal processing, Comminications and Computing. 2013, 1–6

[156] LinY, Wohlberg B, GuoH . UPRE method for total variation parameter selection. Signal Processing, 2010, 90(8): 2546–2551

[157] SteinC M. Estimation of the mean of a multivariate normal distribution. Annals of Statistics, 1981, 9(6): 1135–1151

[158] RamaniS, BluT, UnserM. Monte-Carlo SURE: a black-box optimization of regularization parameters for general denoising algorithms. IEEE Transactions on Image Processing, 2008, 17(9): 1540–1554

[159] PalssonF, Sveinsson J R, UlfarssonM O , BenediktssonJ A. SAR image denoising using total variation based regularization with surebased optimization of regularization parameter. In: Proceedings of the IEEE International Conference on Geoscience and Remote Sensing Symposium. 2012, 2160–2163

[160] LiaoH Y, LiF, NgM K. Selection of regularization parameter in total variation image restoration. Journal of the Optical Society of America A, 2009, 26(11): 2311–2320

[161] BertalmíoM, Caselles V, RougéB , SoléA. TV based image restoration with local constraints. Journal of Scientific Computing, 2003, 19(1–3): 95–122

[162] AlmansaA, Ballester C, CasellesV , HaroG. A TV based restoration model with local constraints. Journal of Scientific Computing, 2008, 34(3): 209–236

[163] VogelC R, OmanM E. Iterative methods for total variation denoising. SIAM Journal on Scientific Computing, 1997, 17(1): 227–238

[164] ChanT F, GolubG H, MuletP. A nonlinear primal-dual method for total variation-based image restoration. Lecture Notes in Control & Information Sciences, 1995, 20(6): 1964–1977

[165] ChambolleA. An algorithm for total variation minimization and applications. Journal ofMathematical Imaging & Vision, 2004, 20(1–2): 89–97

[166] HuangY M, NgM K, WenY W. A fast total variation minimization method for image restoration. SIAM Journal on Multiscale Modeling & Simulation, 2008, 7(2): 774–795

[167] BressonX, ChanT F. Fast dual minimization of the vectorial total variation norm and applications to color image processing. Inverse Problems & Imaging, 2008, 2(4): 455–484

[168] NgM K, QiL Q, YangY F, Huang Y M. On semismooth Newton’s methods for total variation minimization. Journal of Mathematical Imaging & Vision, 2007, 27(3): 265–276

[169] ZhuM Q, ChanT F. An efficient primal-dual hybrid gradient algorithm for total variation image restoration. UCLA CAM Report. 2008, 8–34

[170] ZhuM Q, WrightS J, ChanT F. Duality-based algorithms for totalvariation- regularized image restoration. Computational Optimization and Applications, 2010, 47(3): 377–400

[171] KrishnanD, LinP, YipA M. A primal-dual active-set method for non-negativity constrained total variation deblurring problems. IEEE Transactions on Image Processing, 2007, 16(11): 2766–2777

[172] KrishnanD, PhamQ V, YipA M. A primal-dual active-set algorithm for bilaterally constrained total variation deblurring and piecewise constant Mumford-Shah segmentation problems. Advances in Computational Mathematics, 2009, 31(1–3): 237–266

[173] OsherS, BurgerM, GoldfarbD, Xu J J, YinW T . An iterative regularization method for total variation-based image restoration. Multiscale Modeling & Simulation, 2005, 4(2): 460–489

[174] GoldsteinT, OsherS. The split Bregman method forl₁-regularized problems. SIAM Journal on Imaging Sciences, 2009, 2(2): 323–343

[175] GlowinskiR, Le Tallec P. Augmented Lagrangian and Operator- Splitting Methods in Nonlinear Mechanics. Society for Industrial and Applied Mathematics, 1989

[176] WuC C, TaiX C. Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM Journal on Imaging Sciences, 2010, 3(3): 300–339

[177] DarbonJ, Sigelle M. Image restoration with discrete constrained total variation part I: fast and exact optimization. Journal of Mathematical Imaging & Vision, 2006, 26(3): 261–276

[178] DuanY P, TaiX C. Domain decomposition methods with graph cuts algorithms for total variation minimization. Advances in Computational Mathematics, 2012, 36(2): 175–199

[179] FuH Y, NgM K, NikolovaM, Barlow J L. Efficient minimization methods of mixed l₂-l₁ and l₁-l₁ norms for image restoration. SIAM Journal on Scientific Computing, 2005, 27(6): 1881–1902

[180] GoldfarbD, YinW T. Second-order cone programming methods for total variation-based image restoration. SIAM Journal on Scientific Computing, 2005, 27(2): 622–645

[181] OliveiraJ P, Bioucas-Dias J M, FigueiredoM A T . Adaptive total variation image deblurring: a majorization-minimization approach. Signal Processing, 2009, 89(9): 1683–1693

[182] Bioucas-DiasJ M, Figueiredo M A T, OliveiraJ P . Total variationbased image deconvolution: a majorization-minimization approach, In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 2006, 861–864

[183] ChanT F, Esedoglu S. Aspects of total variation regularized l₁ function approximation. SIAM Journal on Applied Mathematics, 2004, 65(5): 1817–1837

[184] HeL, BurgerM, OsherS. Iterative total variation regularization with non-quadratic fidelity. Journal of Mathematical Imaging & Vision, 2006, 26(1–2): 167–184

[185] JonssonE, HuangS C, ChanT F. Total variation regularization in positron emission tomography. CAM Report. 1998

[186] PaninV Y, ZengG L, GullbergG T . Total variation regulated EM algorithm. IEEE Transactions on Nuclear Science, 1999, 46(6): 2202–2210

[187] LeT, Chartrand R, AsakiT J . A variational approach to reconstructing images corrupted by Poisson noise. Journal of Mathematical Imaging & Vision, 2007, 27(3): 257–263

[188] RudinL, LionsP L, OsherS. Multiplicative denoising and deblurring: theory and algorithms. In: OsherS, Paragios N, eds. Geometric Level Set Methods in Imaging, Vision, and Graphics. New York: Springer, 2003, 103–119

[189] HuangY M, NgM K, WenY W. A new total variation method for multiplicative noise removal. SIAM Journal on Imaging Sciences, 2009, 2(1): 20–40

[190] BoneskyT, Kazimierski K S, MaassP , SchöpferF, Schuster T. Minimization of Tikhonov functionals in Banach spaces. Abstract & Applied Analysis, 2008, 2008(1): 1563–1569

[191] MeyerY. Oscillating Patterns in Image Processing and Nonlinear Evolution Equations, University Lecture Series. Rhode Island: American Mathematical Society, 2002

[192] BlomgrenP, ChenT F. Color TV: total variation methods for restoration of vector valued images. IEEE Transactions on Image Processing, 1970, 7(3): 304–309

[193] SetzerS, SteidlG, PopilkaB, Burgeth B. Variational methods for denoising matrix fields. In: LaidlawD , WeickertJ, eds. Visualization and Processing of Tensor Fields. Berlin: Springer Berlin Heidelberg, 2009, 341–360

[194] EsedogluS, OsherS. Decomposition of images by the anisotropic Rudin-Osher-Fatemi model. Communications on Pure and Applied Mathematics, 2004, 57(12): 1609–1626

[195] ShiY Y, ChangQ S. Efficient algorithm for isotropic and anisotropic total variation deblurring and denoising. Journal of Applied Mathematics, 2013

[196] MarquinaA, OsherS. Explicit algorithms for a new time dependent model based on level set motion for nonlinear deblurring and noise removal. SIAM Journal on Scientific Computing, 2000, 22(2): 387–405

[197] ChanT F, Marquina A, MuletP . High-order total variation-based image restoration. SIAM Journal on Scientific Computing, 2000, 22(2): 503–516

[198] GilboaG, OsherS. Nonlocal operators with applications to image processing. SIAM Journal on Multiscale Modeling & Simulation, 2008, 7(3): 1005–1028

[199] KindermannS, OsherS, JonesP W. Deblurring and denoising of images by nonlocal functionals. SIAM Journal on Multiscale Modeling & Simulation, 2005, 4(4): 1091–1115

[200] HuY, JacobM. Higher degree total variation (HDTV) regularization for image recovery. IEEE Transactions on Image Processing, 2012, 21(5): 2559–2571

[201] YangJ S, YuH Y, JiangM, Wang G. High-order total variation minimization for interior SPECT. Inverse Problems, 2012, 28(1): 15001–15024

[202] LiuX W, HuangL H. A new nonlocal total variation regularization algorithm for image denoising. Mathematics and Computers in Simulation, 2014, 97: 224–233

[203] RenZ M, HeC J, ZhangQ F. Fractional order total variation regularization for image super-resolution. Signal Processing, 2013, 93(9): 2408–2421

[204] OhS, WooH, YunS, Kang M. Non-convex hybrid total variation for image denoising. Journal of Visual Communication & Image Representation, 2013, 24(3): 332–344

[205] DonohoD L. Compressed sensing. IEEE Transactions on Information Theory, 2006, 52(4): 1289–1306

[206] CandèE J, Wakin M B. An introduction to compressive sampling. IEEE Signal Processing Magazine, 2008, 25(2): 21–30

[207] TsaigY, DonohoD L. Extensions of compressed sensing. Signal Processing, 2006, 86(3): 549–571

[208] CandèsE J, Romberg J, TaoT . Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory, 2006, 52(2): 489–509

[209] CandèsE J, Tao T. Near-optimal signal recovery from random projections: Universal encoding strategies?. IEEE Transactions on Information Theory, 2006, 52(12): 5406–5425

[210] DonohoD L, EladM. Optimally sparse representation in general (nonorthogonal) dictionaries via l₁ minimization. Proceedings of National Academy of Sciences, 2003, 100(5): 2197–2202

[211] WrightJ, MaY. Dense error correction via l₁-minimization. IEEE Transactions on Information Theory, 2010, 56(7): 3540–3560

[212] YangJ F, ZhangY. Alternating direction algorithms for l₁-problems in compressive sensing. SIAM Journal on Scientific Computing, 2011, 33(1): 250–278.

[213] NatarajanB K. Sparse approximate solutions to linear systems. SIAM Journal on Computing, 1995, 24(2): 227–234

[214] MallatS G, ZhangZ. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing, 1993, 41(12): 3397–3415

[215] TroppJ, Gilbert A C. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 2007, 53(12): 4655–4666

[216] BlumensathT, DaviesM E. Iterative thresholding for sparse approximations. Journal of Fourier Analysis and Applications, 2008, 14(5–6): 629–654

[217] GorodnitskyI F, RaoB D. Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE Transactions on Signal Processing, 1997, 45(3): 600–616

[218] BaoC L, JiH, QuanY H, Shen Z W. l₀ norm based dictionary learning by proximal methods with global convergence. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 3858–3865

[219] FoucartS, LaiM J. Sparsest solutions of underdetermined linear systems via lq-minimization for 0<q≤1. Applied and Computational Harmonic Analysis, 2009, 26(3): 395–407

[220] CaiT T, WangL, XuG. Shifting inequality and recovery of sparse signals. IEEE Transactions on Signal Processing, 2010, 58(3): 1300–1308

[221] CaiT T, WangL, XuG. New bounds for restricted isometry constants. IEEE Transactions on Information Theory, 2010, 56(9): 4388–4394

[222] ChenS S, DonohoD L, SaundersM A . Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing, 1998, 20(1): 33–61

[223] EfronB, HastieT, JohnstoneI, Tibshirani R. Least angle regression. The Annals of Statistics, 2004, 32(2): 407–499

[224] FigueiredoM A T, Nowak R D. An EM algorithm for wavelet-based image restoration. IEEE Transactions on Image Processing, 2002, 12(8): 906–916

[225] StarckJ L, MaiK N, MurtaghF. Wavelets and curvelets for image deconvolution: a combined approach. Signal Processing, 2003, 83(10): 2279–2283

[226] HerrholzE, Teschke G. Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems. Inverse Problems, 2010, 26(12): 125012–125035

[227] JinB, LorenzD, SchifflerS. Elastic-net regularization: error estimates and active set methods. Inverse Problems, 2009, 25(11): 1595–1610

[228] FigueiredoM A T, Nowak R D, WrightS J . Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE Journal of Selected Topics in Signal Processing, 2007, 1(4): 586–597

[229] KimS J, KohK, LustigM, Boyd S, GorinevskyD . An interior-point method for large-scale l₁-regularized least squares. IEEE Journal of Selected Topics in Signal Processing, 2007, 1(4): 606–617

[230] DonohoD L, TsaigY. Fast solution of l₁-norm minimization problems when the solution may be sparse. IEEE Transactions on Information Theory, 2008, 54(11): 4789–4812

[231] CombettesP L, WajsE R. Signal recovery by proximal forwardbackward splitting. SIAM Journal on Multiscale Modeling & Simulation, 2005, 4(4): 1168–1200

[232] BeckerS, BobinJ, CandésE J . NESTA: a fast and accurate firstorder method for sparse recovery. SIAM Journal on Imaging Sciences, 2011, 4(1): 1–39

[233] OsborneM R, Presnell B, TurlachB A . A new approach to variable selection in least squares problems. IMA Journal of Numerical Analysis, 1999, 20(3): 389–403

[234] LiL, YaoX, StolkinR, Gong M G, HeS . An evolutionary multiobjective approach to sparse reconstruction. IEEE Transactions on Evolutionary Computation,2014, 18(6): 827–845

[235] ChartrandR. Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Processing Letters, 2007, 14(10): 707–710

[236] CandesE J, TaoT. Decoding by linear programming. IEEE Transactions on Information Theory, 2005, 51(12): 4203–4215

[237] SaabR, Chartrand R, YilmazÖ . Stable sparse approximations via nonconvex optimization. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 2008, 3885–3888

[238] TibshiraniR. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 1996, 58(1): 267–288

[239] ZhangC H. Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 2010, 38(2): 894–942

[240] FanJ Q, LiR. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 2001, 96(456): 1348–1360

[241] NikolovaM, NgM K, ZhangS, Ching W K. Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization. SIAM Journal on Imaging Sciences, 2008, 1(1): 2–25

[242] FrankL E, Friedman J H. A statistical view of some chemometrics regression tools. Technometrics, 1993, 35(2): 109–135

[243] FuW J. Penalized regressions: the bridge versus the lasso. Journal of Computational and Graphical Statistics, 1998, 7(3): 397–416

[244] LyuQ, LinZ C, SheY Y, Zhang C. A comparison of typical l_p minimization algorithms. Neurocomputing, 2013, 119: 413–424

[245] CandesE J, WakinM B, BoydS P. Enhancing sparsity by reweighted l₁ minimization. Journal of Fourier Analysis and Applications, 2008, 14(5–6): 877–905

[246] RaoB D, Kreutz-Delgado K. An affine scaling methodology for best basis selection. IEEE Transactions on Signal Processing, 1999, 47(1): 187–200

[247] SheY Y. Thresholding-based iterative selection procedures for model selection and shrinkage. Electronic Journal of Statistics, 2009, 3: 384–415

[248] XuZ B, ZhangH, WangY, Chang X Y, LiangY .L_1/2 regularization. Science China Information Sciences, 2010, 53(6): 1159–1169

[249] XuZ B, GuoH L, WangY, Zhang H. Representative of L_1/2 regularization among l_q (0<q≤1) regularizations: an experimental study based on phase diagram. Acta Automatica Sinica, 2012, 38(7): 1225–1228

[250] CandesE J, PlanY. Matrix completion with noise. Proceedings of the IEEE, 2009, 98(6): 925–936

[251] CaiJ F, CandesE J, ShenZ. A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 2010, 20(4): 1956–1982

[252] BoydS, ParikhN, ChuE, Peleato B, EcksteinJ . Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations & Trends in Machine Learning, 2011, 3(1): 1–122

[253] QianJ J, YangJ, ZhangF L, Lin Z C. Robust low-rank regularized regression for face recognition with occlusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2014, 21–26

[254] LiuY J, SunD, TohK C. An implementable proximal point algorithmic framework for nuclear norm minimization. Mathematical Programming, 2012, 133(1–2): 399–436

[255] YangJ F, YuanX M. Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Mathematics of Computation, 2013, 82(281): 301–329

[256] LiT, WangW W, XuL, FengX C. Image denoising using lowrank dictionary and sparse representation. In: Proceedings of the 10th IEEE International Conference on Computational Intelligence and Security. 2014, 228–232

[257] WatersA E, Sankaranarayanan A C, BaraniukR G . SpaRCS: recovering low-rank and sparse matrices from compressive measurements. In: Proceedings of the Neural Information Processing Systems Conference. 2011, 1089–1097

[258] LiQ, LuZ B, LuQ B, Li H Q, LiW P . Noise reduction for hyperspectral images based on structural sparse and low-rank matrix decomposition. In: Proceedings of the IEEE International on Geoscience and Remote Sensing Symposium. 2013, 1075–1078

[259] ZhouT Y, TaoD C. Godec: randomized low-rank & sparse matrix decomposition in noisy case. In: Proceedings of the 28th International Conference on Machine Learning. 2011, 33–40

[260] ZhangH Y, HeW, ZhangL P, Shen H F, YuanQ Q . Hyperspectral image restoration using low-rank matrix recovery.IEEE Transactions on Geoscience & Remote Sensing, 2014, 52(8): 4729–4743

[261] ZhangZ, XuY, YangJ, Li X L, ZhangD . A survey of sparse representation: algorithms and applications. IEEE Access, 2015, 3: 490–530

[262] BurgerM, FranekM, SchÖnliebC B . Regularized regression and density estimation based on optimal transport. Applied Mathematics Research eXpress, 2012, 2012(2): 209–253

[263] OsherS, Solè A, VeseL . Image decomposition and restoration using total variation minimization and the H¹ norm. Multiscale Modeling & Simulation, 2003, 1(3): 349–370

[264] BarbaraK. Iterative regularization methods for nonlinear ill-posed problems. Algebraic Curves & Finite Fields Cryptography & Other Applications, 2008, 6

[265] MiettinenK. Nonlinear Multiobjective Optimization. Springer Science & Business Media, 2012

[266] MarlerR T, AroraJ S. Survey of multi-objective optimization methods for engineering. Structural and Multidisciplinary Optimization, 2004, 26(6): 369–395

[267] GongM G, JiaoL C, YangD D, Ma W P. Research on evolutionary multi-objective optimization algorithms. Journal of Software, 2009, 20(20): 271–289

[268] FonsecaC M, Fleming P J. Genetic algorithm for multiobjective optimization: formulation, discussion and generation. In: Proceedings of the International Conference on Genetic Algorithms. 1993, 416–423

[269] SrinivasN, DebK. Multiobjective optimization using non-dominated sorting in genetic algorithms. Evolutionary Computation, 1994, 2(3): 221–248

[270] HornJ, Nafpliotis N, GoldbergD E . A niched Pareto genetic algorithm for multiobjective optimization. In: Proceedings of the 1st IEEE Conference on Evolutionary Computation. 1994, 1: 82–87

[271] ZitzlerE, ThieleL. Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach. IEEE Transactions on Evolutionary Computation, 1999, 3(4): 257–271

[272] ZitzlerE, Laumanns M, ThieleL . SPEA2: improving the strength Pareto evolutionary algorithm. Eurogen, 2001, 3242(103): 95–100

[273] KimM, Hiroyasu T, MikiM , WatanabeS. SPEA2+: improving the performance of the strength Pareto evolutionary algorithm 2. In: Proceedings of the International Conference on Parallel Problem Solving from Nature. 2004, 742–751

[274] KnowlesJ D, CorneD W. Approximating the non-dominated front using the Pareto archived evolution strategy. Evolutionary Computation, 2000, 8(2): 149–172

[275] CorneD W, Knowles J D, OatesM J . The Pareto-envelope based selection algorithm for multi-objective optimization. In: Proceedings of the Internatioal Conference on Parallel Problem Solving from Nature. 2000, 869–878

[276] CorneD W, JerramN R, KnowlesJ D, Oates M J. PESA-II: regionbased selection in evolutionary multi-objective optimization. In: Proceedings of the Genetic and Evolutionary Computation Conference. 2001, 283–290

[277] DebK, Agrawal S, PratapA , MeyarivanT. A fast elitist nondominated sorting genetic algorithm for multi-objective optimization: NSGA-II. Lecture Notes in Computer Science, 2000, 1917: 849–858

[278] ZhangQ F, LiH. MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, 2007, 11(6): 712–731

[279] IshibuchiH, SakaneY, TsukamotoN, Nojima Y. Simultaneous use of different scalarizing functions in MOEA/D. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation. 2010, 519–526

[280] WangL P, ZhangQ F, ZhouA M, Gong M G, JiaoL C . Constrained subproblems in decomposition based multiobjective evolutionary algorithm. IEEE Transactions on Evolutionary Computation, 2016, 20(3): 475–480

[281] LiK, FialhoA, KwongS, Zhang Q F. Adaptive operator selection with bandits for a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, 2014, 18(1): 114–130

[282] KeL J, ZhangQ F, BattitiR. Hybridization of decomposition and local search for multiobjective optimization.IEEE Transactions on Cybernetics, 2014, 44(10): 1808–1820

[283] CaiX Y, WeiO. A hybrid of decomposition and domination based evolutionary algorithm for multi-objective software next release problem. In: Proceedings of the 10th IEEE International Conference on Control and Automation. 2013, 412–417

[284] DebK, JainH. An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: solving problems with box constraints. IEEE Transactions on Evolutionary Computation , 2014, 18(4): 577–601

[285] YuanY, XuH, WangB. An improved NSGA-III procedure for evolutionary many-objective optimization. In: Proceedings of ACM Annual Conference on Genetic & Evolutionary Computation. 2014, 661–668

[286] SeadaH, DebK. U-NSGA-III: a unified evolutionary optimization procedure for single, multiple, and many objectives: proof-ofprinciple results. In: Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization. 2015, 34–49

[287] ZhuZ X, XiaoJ, LiJ Q, Zhang Q F. Global path planning of wheeled robots using multi-objective memetic algorithms. Integrated Computer-Aided Engineering, 2015, 22(4): 387–404

[288] ZhuZ X, JiaS, HeS, SunY W, JiZ, ShenL L. Three-dimensional Gabor feature extraction for hyperspectral imagery classification using a memetic framework. Information Sciences, 2015, 298: 274–287

[289] ZhuZ X, XiaoJ, HeS, JiZ, SunY W. A multi-objective memetic algorithm based on locality-sensitive hashing for one-to-many-to-one dynamic pickup-and-delivery problem. Information Sciences, 2015, 329: 73–89

[290] LiH, GongM G, WangQ, Liu J, SuL Z . A multiobjective fuzzy clustering method for change detection in synthetic aperture radar images. Applied Soft Computing, 2016, 46: 767–777

[291] JinY, Sendhoff B. Pareto based approach to machine learning: an overview and case studies. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 2008, 38(3): 397–415

[292] PlumbleyM D. Recovery of sparse representations by polytope faces pursuit. In: Proceedings of the 6th International Conference on In of the first kind using singular values. SIAM Journal on Numerical Analysis, 1971, 8(3): 616–622

[293] WrightS J, NowakR D, FigueiredoM A T . Sparse reconstruction by separable approximation. IEEE Transactions on Signal Processing, 2009, 57(7): 2479–2493

[294] YangY, YaoX, ZhouZ H. On the approximation ability of evolutionary optimization with application to minimum set cover. Artificial Intelligence, 2012, 180(2): 20–33

[295] QianC, YuY, ZhouZ H. An analysis on recombination in multiobjective evolutionary optimization. Artificial Intelligence, 2013, 204(1): 99–119

[296] GongM G, ZhangM Y, YuanY. Unsupervised band selection based on evolutionary multiobjective optimization for hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(1): 544–557

[297] QianC, YuY, ZhouZ H.Pareto ensemble pruning. In: Proceedings of AAAI Conference on Artificial Intelligence. 2015, 2935–2941

[298] QianC, YuY, ZhouZ H. On constrained Boolean Pareto optimization. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence. 2015, 389–395

[299] QianC, YuY, ZhouZ H. Subset selection by Pareto optimization. In: Proceedings of the Neural Information Processing Systems Conference. 2015, 1765–1773

[300] GongM G, LiuJ, LiH, CaiQ, SuL Z. A multiobjective sparse feature learning model for deep neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(12): 3263–3277

RIGHTS & PERMISSIONS

2016 Higher Education Press and Springer-Verlag Berlin Heidelberg