Abstract
Development of an automated system to enable the ongoing monitoring and control of the performance of HIS is a step towards increasing the confidence and security with which ordinary users will perceive AI-based Systems. The main objective of this research was to develop a prototype intelligent system to perform this monitoring. The proposed monitming system integrates performance metrics for the HIS's components with heuristics for evaluating data quality by checking data types, data ranges, percentage of missing data and the composition of data. The research demonstrated the possibility of developing such system and the feasibility of using it for a hybrid intelligent system comprised of an Expert System and a Neural Network. Data quality was expressed as blanks for missing data, a special character for data type mismatch and out of range data left as is. Random en-ors were not considered within the scope of this research. The monitoring process started by defining standards for each of the data quality, data composition and data characteristics metrics. Five test cases were randomly generated. Errors were introduced systematically to each of these cases. The drop in HIS accuracy was measured and the metrics c01Tesponding to these drops were calculated. The standard value of the metrics used corresponds to an allowable drop of accuracy of three times the standard deviation of the accuracy estimate based on a sample of size N. To test the monitoring system, 20 test cases were generated, errors were introduced randomly to them. The introduced errors had effects on quality, composition and characteristics of data. Those samples were used to run the original HIS and the accuracies of the components were measured. The monitoring system then was used to determine the HIS status. Out of the 20 test cases the monitoring system could correctly identify the errors and the reason of performance degradation in 18 cases. This proved the feasibility of monitoring the HIS used for this application. A similar approach could be readily adopted for monitoring the perfmmance of Hybrid Intelligent Systems developed for a wide range of business and industiial applications.
Recommended Citation
Baky, Nariman Abdel and Ercal, Fikret, "Performance Monitoring of Hybrid Intelligent Systems" (1994). Computer Science Technical Reports. 171.
https://scholarsmine.mst.edu/comsci_techreports/171
Department(s)
Computer Science
Report Number
CSc-94-22
Document Type
Technical Report
Document Version
Final Version
File Type
text
Language(s)
English
Rights
© 1994 University of Missouri - Rolla, All rights reserved
Publication Date
1 December, 1994
Comments
The first Author is a Graduate Student.
This report is substantially the M.S. thesis of the first author, completed December 1994.