Project Team
Lead Organization: Siemens Technology
Team Composition: Siemens Technology, Allem Business Ventures (ABV)
Background
The ARM Institute’s strategic collaboration with the Joint Robotics Organization for Building Organic Technologies (JROBOT) Working Group resulted in the identification, funding and execution of several robotic for manufacturing programs addressing pervasive Department of Defense (DoD) sustainment needs.
The Autonomous Swarm Inspection and Interactive 3D Modelling with Orchestrated Visualization (ASIIMOV) project, funded with support from the Deputy Assistant Secretary of the Navy for Sustainment, has developed autonomous swarm capabilities to significantly enhance the inspection quality and visualization of critical Navy assets.
The program was led by Siemens Technologies, with the support of Allem Business Ventures, successfully developed multi-drone navigation software, asset survey capability without prior knowledge, high-resolution image acquisition, high resolution 3D model creation, and the use of a mixed reality (MR) interface to operate and interact with a swarm of drones.
Problem Statement:
It takes significant time to inspect and monitor maintenance activities of large naval ships, inhibiting the availability of assets critical to sustaining the security of the Unites States and its allies. State of the art large structure inspection is manual, non-repeatable, time-consuming, costly and unsafe. Emerging Unmanned Air Vehicle (UAV)-based inspection methods need extensive preparation, skilled pilot control and data processing capabilities that take days to complete.
In addition, current inspection methods do not readily permit the utilization of modern manufacturing techniques, making it increasing more challenging for the DoD to reach its sustainment goals for digitizing inventory, sustainment, and inspection.
Approach:
Siemens Global Technology with Allem Business Ventures, in partnership with the US Navy, leveraged a combination of DoD approved hardware to develop significant advances in UAV inspection capabilities for large structure with any prior information or knowledge. This includes:
- Software to navigate multiple UAVs (drones) simultaneously
- Semi-autonomous drone flight control to survey infrastructure equipment using Lidar without requirement or prior 3D models
- Automatic 3D mesh model creation
- Autonomous swarm inspection and high-resolution image (data) acquisition
- Automatic high-resolution volumetric 3D model creation for inspection and image data analysis
- Mixed reality (MR) interface to operate and interact with swarm of drones.
Results & Conclusions
This pervasive technology developed through the partnership between the ARM Institute, Siemens Global Technology, and the US Navy, is making automated inspection capabilities of large assets a reality. Methods that have been chosen and demonstrated eliminate the resource intensive processes and facilitate autonomous inspection in complex & remote environments with accelerated high-quality inspection. The Siemens team successfully demonstrated the following multi-drone inspection technology capabilities:
- Reduce the time needed to inspect large assets and assets in complex environments with little infrastructure support (e.g., ships, wind-turbines, off-shore oil-rigs, etc.)
- Improve safety by eliminating manual processes in complex or difficult environments.
- Accelerate the collection of high-quality inspections of critical assets and infrastructure.
This project is based on top of some existing design of Siemens Energy Autonomous drone and Defense Advanced Research Projects Agency (DARPA) OFFensive Swarm-Enabled Tactics (OFFSET) program. The ARM Consortium Intellectual Property (CDIP) developed under this partnership include:
- UAV Navigation Stack: Software that navigates UAVs to obtain the desired viewpoint
- Swarm Autonomy: Software that decomposes high-level tasks into UAV command on swarms
- Augmented Reality / Mixed Reality (AR/VR) Module: Human Machine Interface (HMI) utilize both AR and traditional touch screen
- UAV Reference Design: Reference design including sensors, controllers, motors, embedded
Impact for the Department of Defense (DoD):
The advancements from this project resulted in reduced costs for sustainment operations, reduced time to execute inspection processes in complex environments with little infrastructure support and increased safety by eliminating manual processes in complex or difficult environments.
On September 22, 2021, the ARM Institute convened a closed project review at the Battleship USS New Jersey in Camden, NJ that included representatives from the Office of Deputy Assistant Secretary of Defense for Materiel Readiness, Office of Secretary of Defense (OSD) ManTech and DoD Manufacturing USA Institutes, US Army, US Navy, and Siemens Energy and Siemens Government Affairs.
Next Steps
The ARM Institute, in partnership with the US Navy is pursuing continued technology development efforts focused on expanding swarm autonomous inspection capabilities.
ABOUT THE ARM INSTITUTE
The Advanced Robotics for Manufacturing (ARM) Institute is a Manufacturing Innovation Institute (MII) funded by the Office of the Secretary of Defense under Agreement Number W911NF-17-3-0004, and is part of the Manufacturing USA® network. The ARM Institute leverages a unique, robust, and diverse ecosystem of 300+ consortium members and partners across industry, academia, and government to make robotics, autonomy, and artificial intelligence more accessible to U.S. manufacturers large and small, train and empower the manufacturing workforce, strengthen our economy and global competitiveness, and elevate national security and resilience. Based in Pittsburgh, PA since 2017, the ARM Institute is leading the way to a future where people & robots work together to respond to our nation’s greatest challenges and to produce the world’s most desired products. For more information, visit www.arminstitute.org and follow the ARM Institute on LinkedIn and Twitter