The Opportunity
- This company operates equipment in motion at high speed. Certain components develop defects or damage that could cause failures. These failures not only impede operations, resulting in lost time and unscheduled repairs, but also pose a potential safety risk.
- They have implemented imaging systems which take high-speed, high-resolution images of the components. The goal is to locate potential problems for a human to examine.
- The built-in detection algorithms, based on decades-old engineering methods, were not detecting enough events, and produced too many false positives.
- The client wanted to know if emerging technologies could help recoup their investment in the imaging system by making it more precise.
Our Approach
- We proposed testing whether new image recognition and classification technologies, such as deep learning (neural networks) could recognize eight kinds of defects or problems. We had hundreds of thousands of images totaling 4 Terabytes of data.
- Our challenge was that we had only a few hundred “labeled” examples – where humans made the classification for the algorithms to learn with – of “bad cases” to train our models on. We had to invent a “semi-supervised” method of training our models.
The Impact
- In a test dataset, the machine learning algorithms were able to detect anomalies at more than twice the rate of humans. Of course, a machine can process images many thousands of times faster than a human.
- Machine learning algorithms were able to correctly detect anomolies at a rate 4.2 times better than a legacy, automated system.
- The capture rate of true positives ranged from 82% to 95%, and the false positive rate from 6% to 14%. This reflects a tradeoff in the tuning of these models: accuracy of positives vs flagging too many “good” cases as “bad.”