Hierarchical Assessment of Autonomous Navigation Algorithms

Project began April 2019.

While much effort has been devoted to the development of autonomous navigation algorithms in recent years, techniques and procedures for quantifying the performance of these algorithms have been developed on an ad-hoc, case-by-case basis. In particular, the use of system-level tests to compare the performance of autonomy subsystems (such as path planning or navigation) leads to a high degree of uncertainty in algorithmic performance. In this work, we propose to develop a numerical model for quantifying the propagation of error through the subsystems of autonomous navigation software.

The performance of autonomous ground systems has typically been measured at the system level, and subsystem-level information is often obscured to the tester. An analog in traditional manned-vehicle testing might be developing a model of powertrain influence on maximum speed. For example, the maximum speed of a vehicle depends on many factors other than the powertrain (aerodynamics of the vehicle, tire shape and inflation, etc.). If we wish to isolate the influence of the powertrain, we must isolate the powertrain in a system-level test such as a dynamometer. Similarly, in an autonomous vehicle, if we would like to understand the performance of the path planning algorithm, we must devise a test to isolate this algorithm (from perception, global planning, etc.) at a system level. Therefore, the objective of this research is to study test methods for autonomous vehicles in simulation to determine how to design tests that isolate each component of the autonomous architecture.

Recent Publications from Prior Related Work:

• Hudson, C. R., Goodin, C., Doude, M., & Carruth, D. W. (2018, August). Analysis of Dual LIDAR Placement for Off-Road Autonomy Using MAVS. In 2018 World Symposium on Digital Intelligence for Systems and Machines (DISA) (pp. 137-142). IEEE..
• Goodin, C., Doude, M., Hudson, C., & Carruth, D. (2018). Enabling off-road autonomous navigation simulation of LIDAR in dense vegetation. Electronics, 7(9), 154.
• Durst, P. J., Goodin, C. T., Bethel, C. L., Anderson, D. T., Carruth, D. W., & Lim, H. (2018). A Perception- Based Fuzzy Route Planning Algorithm for Autonomous Unmanned Ground Vehicles. Unmanned Systems, 6(04), 251-266.
• Goodin, C., Carruth, D., Doude, M., Hudson, C., Dabbiru, L., & Sharma, S. (2019). Training of Neural Networks with Automated Labeling of Simulated Sensor Data (No. 2019-01-0120). SAE Technical Paper.
• Goodin, C., Carrillo, J. T., McInnis, D. P., Cummins, C. L., Durst, P. J., Gates, B. Q., & Newell, B. S. (2017, May). Unmanned ground vehicle simulation with the Virtual Autonomous Navigation Environment. In Military Technologies (ICMT), 2017 International Conference on (pp. 160-165). IEEE.
• Durst, P. J., Monroe, G., Bethel, C. L., Anderson, D. T., & Carruth, D. W. (2018, May). A history and overview of mobility modeling for autonomous unmanned ground vehicles. In Autonomous Systems: Sensors, Vehicles, Security, and the Internet of Everything (Vol. 10643, p. 106430G). International Society for Optics and Photonics.
• Carruth, D. W. (2018, August). Simulation for Training and Testing Intelligent Systems. In 2018 World Symposium on Digital Intelligence for Systems and Machines (DISA) (pp. 101-106). IEEE.
• Carruth, D. W., & Bethel, C. L. (2017, January). Challenges with the integration of robotics into tactical team operations. In Applied Machine Intelligence and Informatics (SAMI), 2017 IEEE 15th International Symposium on (pp. 000027-000032). IEEE.