The Missing Layer in Robot Safety Assurance

NVIDIA Corp. — why robot safety assurance must account for attacks that can change how a machine sees, decides, and acts. Robot safety has traditionally asked: Can a machine remain safe when something goes wrong?

Physical AI raises a harder question: Can it remain safe when an attacker changes what it sees, decides, or does even when nothing appears to have failed? Modern robots perceive through multimodal sensors, interpret context using AI models, and translate those interpretations into physical action. As they move into dynamic environments, their safety increasingly depends on the integrity of the data guiding their decisions. That dependence creates risks conventional safety assessments may not fully capture. Recent research has demonstrated that manipulating what a robot sees, hears, or interprets can redirect its behavior without requiring direct control. Such manipulation can occur across the robotic brain — a layered attack surface encompassing training pipelines, system infrastructure, and runtime perception. Layer One: Corrupting intelligence at its source In 2017, BadNets demonstrated that a model could behave normally under most conditions, yet fail in the presence of a specific hidden trigger. In one example, a subtle pattern caused a stop sign to be misclassified as a speed limit sign without affecting the model’s behavior on other inputs. What began as a classification vulnerability has since evolved into action manipulation. At NeurIPS 2025, researchers introduced BadVLA a backdoor attack targeting Vision-Language-Action (VLA) models that allow robots to see, interpret instructions, and produce coordinated physical movement. Rather than altering a single label, the attack caused conditional deviations in the robot’s action trajectory when a trigger was present. Without the trigger, the model largely preserved normal task performance, while the backdoor remained effective under task transfers and model fine-tuning. A related study in 2025, GoBA, showed that ordinary objects such as a coffee mug could serve as a reliable trigger. The researchers reported a 97% attack success rate without degrading performance on clean inputs. These studies expose a blind spot in model validation: a model may pass testing yet produce corrupted behavior when a hidden trigger appears in operation.