Available at SSRN 5202517
Integrating Large Language Models (LLMs) into robotic systems has revolutionized embodied artificial intelligence, enabling advanced decision-making and adaptability. However, ensuring reliability—encompassing both security against adversarial attacks and safety in complex environments—remains a critical challenge. To address this, we propose a unified framework that mitigates prompt injection attacks while enforcing operational safety through robust validation mechanisms. Our approach combines prompt assembling, state management, and safety validation, evaluated using both performance and security metrics. Experiments show a 30.8% improvement under injection attacks and up to a 325% improvement boost in complex environment settings under adversarial conditions compared to baseline scenarios. This work bridges the gap between safety and security in LLM-based robotic systems, offering actionable insights for deploying reliable LLM-integrated mobile robots in real-world settings.
Threat Model Overview
(Add overview.png to public folder)
Comprehensive threat model illustrating security vulnerabilities and attack vectors in LLM-integrated robotic systems.
System Workflow
(Add workflow.png to public folder)
Complete workflow showing the integration of prompt assembling, state management, and safety validation in our proposed system.
Simulation Demo GIF
(Add your eyesim.gif to public folder)
Eyesim implementation demonstrating LLM-integrated navigation.
Physical Robot Demo GIF
(Add your pioneer.gif to public folder)
Real-world implementation using Pioneer Robot.
Novel approach bridging security and safety in LLM-integrated robotic systems with comprehensive validation
Comprehensive approach addressing both security against attacks and safety in complex environments
30.8% improvement under injection attacks and up to 325% boost in complex environment settings
Wenxiao Zhang
Xiangrui Kong
Conan Dewitt
Thomas Braunl
Jin B. Hong
BibTeX:
@article{zhang5202517enhancing,
title={Enhancing Reliability in Llm-Integrated Robotic Systems: A Unified Approach to Security and Safety},
author={Zhang, Wenxiao and Kong, Xiangrui and Dewitt, Conan and Br{"a}unl, Thomas and Hong, Jin},
journal={Available at SSRN 5202517}
}Available at SSRN 5202517