Anomaly Detection and Diagnosis Algorithms for Discrete Symbols

We present a set of novel algorithms which we call sequenceMiner that detect and characterize anomalies in large sets of high-dimensional symbol sequences that arise from recordings of switch sensors in the cockpits of commercial airliners. While the algorithms we present are general and domain-independent, we focus on a specific problem that is critical to determining the system-wide health of a fleet of aircraft. The approach taken uses unsupervised clustering of sequences using the normalized length of the longest common subsequence (nLCS) as a similarity measure, followed by detailed outlier analysis to detect anomalies. In this method, an outlier sequence is defined as a sequence that is far away from the cluster centre. We present new algorithms for outlier analysis that provide comprehensible indicators as to why a particular sequence is deemed to be an outlier. The algorithms provide a coherent description to an analyst of the anomalies in the sequence when compared to more normal sequences. In the final section of the paper we demonstrate the effectiveness of sequenceMiner for anomaly detection on a real set of discrete sequence data from a fleet of commercial airliners. We show that sequenceMiner discovers actionable and operationally significant safety events. We also compare our innovations with standard HiddenMarkov Models, and show that our methods are superior.

Data and Resources

Additional Info

Field Value
Maintainer Ashok Srivastava
Last Updated March 31, 2025, 17:36 (UTC)
Created March 31, 2025, 17:36 (UTC)
Identifier DASHLINK_147
Issued 2010-09-22
Modified 2020-01-29
accessLevel public
accrualPeriodicity irregular
bureauCode {026:00}
catalog_@context https://project-open-data.cio.gov/v1.1/schema/catalog.jsonld
catalog_@id https://data.nasa.gov/data.json
catalog_conformsTo https://project-open-data.cio.gov/v1.1/schema
catalog_describedBy https://project-open-data.cio.gov/v1.1/schema/catalog.json
harvest_object_id 5fde5024-2469-4a3d-97c0-d9dd1339055f
harvest_source_id 61638e72-b36c-4866-9d28-551a3062f158
harvest_source_title DNG Legacy Data
landingPage https://c3.nasa.gov/dashlink/resources/147/
programCode {026:029}
publisher Dashlink
resource-type Dataset
source_datajson_identifier true
source_hash 8d5dab6ea967cc82368436405d68aad2bdb6c6c37fc2dd4bc8476e3ae6159aca
source_schema_version 1.1