AI may perform well with predictable operating conditions, but what happens when sensors provide faulty readings, and the system encounters an unfamiliar problem?
This article features three engineering methods that can help identify potential risks and evaluate whether AI systems can handle situations safely.
An AI system that monitors an industrial pump may accurately detect signs of wear under expected operating conditions. However, if a sensor starts sending incorrect readings, the AI system could miss a developing fault because it relies on information that no longer reflects the pump’s actual condition.
Stress-testing helps identify weaknesses that may remain hidden during routine evaluations. A 2026 study on “Bridging the Robustness Gap: Stress-Testing Artificial Intelligence Algorithms,” found that changes to data, labels, and model parameters affected AI algorithms differently where some unsupervised models experienced near-total structural failure, while supervised models retained some functionality.
It shows that passing an AI evaluation does not guarantee reliable performance in real-world conditions. Stress-testing helps expose these gaps before the system is used in practice by examining how the AI system responds to disruptions and whether its outputs remain reliable enough for its intended use.
When stress-testing reveals weaknesses, safety controls define how the system responds when its reliability is compromised. Data validation checks can flag unreliable inputs, while warning mechanisms can alert operators to unexpected outputs. Human oversight can also provide a further safeguard by allowing questionable recommendations to be reviewed before they inform critical decisions.
The effectiveness of these safeguards depends on how the system responds when a problem is detected. An alert offers limited protection if the system continues issuing recommendations that may be acted upon without further checks. Therefore, an AI system should only be relied upon to the extent that its reliability has been demonstrated, with safeguards in place to prevent unreliable outputs from contributing to unsafe decisions.
Stress-testing examines how an AI system responds to disruptions, while simulation-based testing evaluates its behavior across scenarios that may be difficult or unsafe to reproduce in real operating environments.
For example, an AI system designed to monitor industrial equipment could be tested in a simulated environment where a pump motor overheats while a faulty pressure sensor reports inaccurate readings. A 2020 study, “Formal Scenario-Based Testing of Autonomous Vehicles: From Simulation to the Real World,” says that simulated scenarios help identify safety-critical situations for AI-based systems and select cases for further real-world testing. Similarly, the U.S. National Institute of Standards and Technology (NIST) explains that testing combinations of input conditions reveal rare failure scenarios that conventional testing may overlook. These highlight the value of testing multiple interacting conditions to identify weaknesses that individual tests may miss by allowing its responses to be assessed without exposing actual equipment to unnecessary risks.
If a simulated test reveals that an AI system fails to recognize a pressure drop when its temperature sensor malfunctions, developers can investigate the cause and use representative examples of these combined faults to refine the system.
They can then repeat the simulation to assess whether its ability to identify the pressure drop has improved. To determine whether this improvement extends beyond the original scenario, developers can test combinations of sensor errors and pressure changes that were not included in the training data.
Comparing fault detection rates across these scenarios helps establish whether the AI system can identify pressure drops consistently under different operating conditions, rather than improving only in the specific scenario used during testing.
Stress-testing challenges an AI system to reveal weaknesses, while simulation-based testing evaluates its behavior across controlled scenarios. Runtime monitoring, on the other hand, identifies when an AI system’s predictions become unreliable during operations.
For example, an AI system monitoring an industrial pump may report normal conditions despite sensor readings that suggest a developing blockage. A runtime monitor can compare the system’s output with physical constraints and flag inconsistencies for investigation. A 2026 study on “COMET: A Coordinated Multi-Layer Runtime Monitoring Framework for Human-Supervised AI-Enabled Safety Systems” found that checking sensor signal quality and comparing AI predictions with expected physical behavior can detect potentially unreliable predictions that a model’s output alone may not reveal.
A mismatch between an AI prediction and the actual condition of the equipment does not necessarily mean that the AI model is faulty. Comparing sensor readings with independent measurements or physics-based estimates can help determine whether the discrepancy originates from faulty instrumentation or the AI’s interpretation of valid data.
This distinction ensures that corrective actions address the actual source of the problem rather than treating every failure as an AI model issue.
As AI becomes more involved in engineering decisions, the ability to question its outputs and verify its performance becomes increasingly important. This requires an understanding of both AI methods and the physical systems in which they are applied.
The Engineering Institute of Technology (EIT’s) Bachelor of Science (Industrial Automation Engineering) and Master of Engineering – Applied Research (Industrial Automation) represent two pathways in a field where automation, system monitoring, and AI technologies can intersect. These areas of study reflect the growing connection between established engineering principles and increasingly intelligent systems.
References
International AI experts warn of potentially catastrophic risks from AI
Bridging the Robustness Gap: Stress-Testing Artificial Intelligence Algorithms
Formal Scenario-Based Testing of Autonomous Vehicles: From Simulation to the Real World
This article was published November 4th, 2026 and the content is current as at the date of publication.
Back to news