Safety Verification of Neural Network Controlled Cyber-Physical Systems under Precision Errors

Abstract

Cyber-physical systems (CPS) are increasingly utilizing neural networks (NN) as controllers due to their ability to model complex, non-linear dynamics of the system. Thus, ensuring the safety of these systems becomes crucial, specifically in the context of safetycritical applications. State-of-the-art safety verification techniques for CPS primarily analyze the reachability of safe states, considering bounded errors stemming from noisy dynamic environments or inaccurate implementations. However, it assumes real-valued arithmetic and does not account for the round-off error due to fixed-point quantization while deploying NN controller on resource-constrained embedded systems. Some recent efforts have focused on generating sound quantized NN implementations in fixed-point, ensuring specific target error bounds, but they assume the safety of the NN controller is already proven. To bridge this gap, this thesis introduces Nexus, a novel two-phase framework combining reachability analysis of CPS with real-valued NNs and sound NN quantization. Nexus provides an end-to-end solution that ensures CPS safety within bounded errors while generating mixed-precision fixed-point implementations for NN controllers. In the first phase, Nexus performs reachability analysis on a CPS using a real-valued NN controller to compute over-approximated reachable sets that prove system safety up to a given time bound. In the second phase, Nexus generates an optimized fixed-point mixed-precision implementation of the NN controller that preserves this safety guarantee. Additionally, Nexus’s extended code generation exploits the inherent parallelism in neural network computations to generate implementations for automated parallelization on FPGAs using directives in commercial HLS compilers. Nexus has been evaluated on 12 neural network-controlled CPS benchmarks and demonstrated Nexus’s ability to perform safety verification and quantization of the NN controller while satisfying the given error bound to generate safe hardware implementation of the NN controller. Nexus’s extended code generation significantly reduces the latency of the implementation for all the benchmarks. We also evaluate Nexus’s extended code generation on larger benchmarks to show the scalability of Nexus while generating efficient implementation.