In order to evaluate the performance of the deep space celestial autonomous navigation system, the utility function model is proposed. This paper adopts MATLAB as a simulation platform, deep space exploration as a simulation background, Earth-Mars transfer orbit as a model, and attempts to propose an effective assessment method in Celestial Autonomous Navigation. At the same time, several common non-linear filtering algorithms will be applied in this model, with the way of numerical and graphical display to show the navigation performance effectiveness of different filtering algorithm. The results show that the evaluation method can reflect and evaluate effectively the performance of different filtering algorithm.