How Data Science is Creating Safer Skies Through Flight Data Monitoring and Anomaly Detection

Steven Smith

Jan 15, 2021 | 5 MINUTE Read

How Data Science is creating safer skies through flight data monitoring and anomaly detection

In Data Science, we call deviations from expected behaviour “anomalies”, and they can be extremely challenging to detect on a large scale. It is made even more complicated when an anomaly occurs in a very short time frame or evolves over the course of multiple flights.

The L3Harris Commercial Aviation Solutions mission is “Creating Safer Skies”; our assignment was to develop an advanced tool, powered by machine learning, that will make the issue of identifying anomalies easier to approach. Our algorithm can learn the typical behaviour of aircraft using a dataset of thousands of flights. Using this model of aircraft behaviour, we can then predict what an aircraft is likely to do next — over the next few seconds or minutes. When we see an aircraft behaving significantly differently to our prediction, we then recognise that situation as an “anomaly”.

As this technology learns the inter-relationship between aircraft systems, this technology does not require domain knowledge to set thresholds for known issues and thus can be used to identify anomalies and predict failures not previously known to be an issue. We can also determine which parameters are behaving strangely to give an investigator or engineer a head-start in knowing which subsystem to investigate. This can save significant time in diagnosing a faulty component in the aircraft, and in many cases, performing the repair before a critical failure occurs.

We have identified many applications for this technology, including detecting unusual pilot behaviour upon take-off and landing, improper aircraft configuration, and localising mechanical faults inside the jet engine.

Incident investigation

A Boeing 737 encountered a serious un-commanded pitch-up incident during a routine flight. The pilots in command struggled to pitch the nose of the aircraft downward, as the elevator did not respond adequately to input on the control column — in fact, to regain control, the crew had to exert over 100 daN of force on the control column, several times the expected value during routine operations.

Fortunately, the pilots managed to eventually regain control, perform a go-around, and land safely on a second attempt. After the incident, the Accident Investigation Board Norway (AIBN) conducted an in-depth investigation. As a result, it was determined that de-icing fluid, or humidity from the de-icing, had leaked into the tail compartment and frozen on the input cranks for the elevator Power Control Units (PCUs) during the descent.

In understanding with the AIBN, the data science team at L3Harris Flight Data Services have conducted a further investigation into the flight, along with a series of other flights that also encountered unusually high control column forces.

Application of Machine Learning

One of the most effective ways of detecting unusual conditions on a flight is to build a model of what normal conditions look like. When the actual aircraft flight data deviates significantly from this learned model of “normal” behaviour, that deviation can be reported as an anomaly.

There are numerous approaches for training a machine learning algorithm to perform an anomaly detection task, this approach is derived from a NASA Jet Propulsion Laboratory research paper titled "Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding". We trained a bi-directional Long Short-Term Memory (LSTM) neural network to model the relationship between input and output time-series data. An LSTM is a specialized version of a neural network that is used to learn the structure inside time series data. The approach takes the aircraft’s control inputs as inputs to the algorithm, while trying to predict what the aircraft’s external state will be several seconds into the future.

In our case, we used various control inputs such as control column force, elevator pitch command, Stabiliser manual trim up/down, Autopilot trim up/down, Mach trim command, as well as external measurements, such as Airspeed and Pitch. These values were then used to predict the values that we would expect to see four seconds in the future for the Elevator (left and right), Elevator Actuator (left and right) and Stabilizer positions.

Training an LSTM takes significant resources. The model requires Graphics Processing Units (GPU) to perform millions of matrix calculations in a reasonable time frame. However, it still took more than twenty four hours to train our final model to detect these anomalies. Once the network was trained using a significant training dataset, it was necessary to determine the performance of our anomaly detector. We used a “test set” comprised of 3,122 flights that had not been seen by our algorithm.

We then calculated the point at which a flight transitions from being normal to becoming an anomaly using a log-normal probability distribution of the difference between the actual flight data and our predicted flight data. We determined that those flights which deviated more than 99% from the "normal" were considered to be anomalies.

Analyzing the Results

When looking at the anomalous flights generated by our approach, we found that the most anomalous of the flights in our test set by a considerable margin was the incident under investigation.

Flight Data Monitoring Data Science Chart

Figure 2: Analysis of actual vs. predicted flight data by time in seconds.

The top plot in Figure 2 shows the actual recorded flight data – parameters taken from the Flight Data Recorder of the incident flight. The centre plot shows the prediction of the aircraft’s behaviour based on our generated Machine Learning model. The bottom plot simply shows the difference between the first two plots, this difference between actual and prediction is the “reconstruction error”. There is a very large and significant jump in reconstruction error at approximately 6,000 seconds (the x-axis) into the flight data, and that the parameter that is causing the error, the orange spike, is “Elevator Actuator (L)”. This observation matches with the AIBN investigation report, the aircraft “came close to stalling as a result of a blocked elevator.”

In Conclusion

The application of this LSTM model has shown it can identify anomalies in aircraft systems that can predict failures enabling engineers to pro-actively take action to improve the safety of their aircraft. In the case of incident investigation, we were able to localise the anomaly to a particular moment in the flight, as well as being able to determine that the anomaly was in the left elevator actuator. However, it does not take an incident of this severity for the model to be effective.

This Machine Learning model is being applied to other applications across both Maintenance and Safety to ensure we play our part in “Creating safer skies” for our customers.