Empirical Evaluation of Diagnostic Algorithm Performance Using a Generic Framework

Alexander Feldman, Tolga Kurtoglu, Sriram Narasimhan, Scott Poll, David Garcia, Johan de Kleer, Lukas Kuhn, and Arjan van Gemund
Publication Target: 
Publication Issue: 
Submission Type: 
Full Paper
ijPHM_10_002.pdf977.61 KBJuly 6, 2010 - 8:24am

A variety of rule-based, model-based and datadriven techniques have been proposed for detection and isolation of faults in physical systems. However, there have been few efforts to comparatively analyze the performance of these approaches on the same system under identical conditions. One reason for this was the lack of a standard framework to perform this comparison. In this paper we introduce a framework, called DXF, that provides a common language to represent the system description, sensor data and the fault diagnosis results; a run-time architecture to execute the diagnosis algorithms under identical conditions and collect the diagnosis results; and an evaluation component that can compute performance metrics from the diagnosis results to compare the algorithms. We have used DXF to perform an empirical evaluation of 13 diagnostic algorithms on a hardware testbed (ADAPT) at NASA Ames Research Center and on a set of synthetic circuits typically used as benchmarks in the model-based diagnosis community. Based on these empirical data we analyze the performance of each algorithm and suggest directions for future development.

Publication Year: 
Publication Volume: 
Publication Control Number: 
Page Count: 
Submission Keywords: 
applications: aviation
diagnostic algorithm
diagnostic performance
fault diagnosis
Submission Topic Areas: 
Health management system design and engineering
Model-based methods for fault detection, diagnostics, and prognosis
Systems and platform applications
Technology maturation
Verification and validation
Submitted by: 

follow us

PHM Society on Facebook Follow PHM Society on Twitter PHM Society on LinkedIn PHM Society RSS News Feed