On Applying the Prognostics Performance Metrics

Abhinav Saxena, Jose Celaya, Bhaskar Saha, Sankalita Saha, and Kai Goebel
Submission Type: 
Full Paper
Supporting Agencies (optional): 
NASA Ames Research Center
phmc_09_39.pdf635.01 KBSeptember 17, 2009 - 6:10am

Prognostics performance evaluation has gained significant attention in the past few years. As prognostics technology matures and more sophisticated methods for prognostic uncertainty management are developed, a standardized methodology for performance evaluation becomes extremely important to guide improvement efforts in a constructive manner. This paper is in continuation of previous efforts where several new evaluation metrics tailored for prognostics were introduced and were shown to effectively evaluate various algorithms as compared to other conventional metrics. Specifically, this paper presents a detailed discussion on how these metrics should be interpreted and used. Several shortcomings identified while applying these metrics to a variety of real applications are also summarized along with discussions that attempt to alleviate these problems. Further, these metrics have been enhanced to include the capability of incorporating probability distribution information from prediction algorithms as opposed to evaluation based on point estimates only. Several methods have been suggested and guidelines have been provided to help choose one method over another based on probability distribution characteristics. These approaches also offer a convenient and intuitive visualization of algorithm performance with respect to metrics like prediction horizon and α-λ performance, and also quantify the corresponding performance while incorporating the uncertainty information.

Publication Control Number: 
Submission Keywords: 
data driven prognostics
diagnostic performance
model based prognostics
performance metrics
PHM system design and engineering
prognostic performance
remaining useful life (RUL)
return on investment (ROI)
Submitted by: 

follow us

PHM Society on Facebook Follow PHM Society on Twitter PHM Society on LinkedIn PHM Society RSS News Feed