Mine-Countermeasures Autonomous Technologies Evaluation Study
Mine Countermeasures (MCM) Autonomous Technologies Evaluation Study is a research project sponsored by the Office of Naval Research and advised by the Naval Sea Systems Command at Panama City, Florida. The project goal is to assess the impacts of autonomous technologies on the Mine Countermeasures mission. This involves modeling and simulation of the unmanned vehicle search operations as well as the shipboard planning, deployment, and support activities to evaluate system performance against several metrics of interest, including mission time and human involvement. The unmanned vehicle search operations and shipboard operations data come from the integration of response surface equations developed in a two-pronged approach. The search operations data came from an agent-based simulation environment developed at MIT and Oxford, known as MOOS. The shipboard operations data came from a discrete event simulation developed in the Arena software. Once transformed into RSEs, these data are represented in a single tool, called IMAGE, developed in Microsoft Excel. This tool represents the final deliverable and provides a way for an MCM analyst to measure, in a quantitative fashion, the benefits of adding autonomous technologies to various aspects of typical MCM operations.
Dr. Santiago Balestrini-Robinson
Eric Van Gehuchten|
This project was motivated by a global proliferation of mines, which make a very cost effective area denial technology. Mines have accounted for more attacks on naval ships since World War II than any other form of attack. In addition to being effective and dangerous, they are also a very cost effective weapon. For example, the mine that damaged the USS S.B. Roberts attack in 1988 was valued at just $1,500. Mines are also easily deployable. A potential minelayer may need just a simple fishing vessel to deploy mines. The complexity of mine warfare and the inherent danger that goes along with it make it a very pertinent and interesting grand challenge problem to tackle.
First, a little background of the problem, naval mines can be used in a variety of depths. The detection of mines becomes increasingly more difficult as one moves from deep water towards the surf zone. This is largely due to effects on SONAR caused by reflections off the sea floor and ocean surface. Also, the clutter and non-mine mine-like bottom objects (NOMBOs), increases as one moves towards shore. So the number of target detection false positives and false negatives increases. All this is increasingly important as today’s non-traditional warfare is pushing naval missions into littoral waters more often.
Mines come with a variety of triggering methods: remote, contact, and influence. A contact mine requires physical contact with a vessel in order to detonate. An influence mine can be triggered from a distance via magnetic and acoustic signature for example. Mines also provide varying attack methods. Some mines are stationary; whereas others provide some sort of locomotion like a torpedo or rocket mine, where the mine can actually track down a target once activated. The method by which mines inflict damage also varies. Contact mines create concentrated structural damage and expel hot gases into a vessel. Non-contact mines do their damage through bubble jets slamming into a vessel and/or a shock wave that can violently shake a vessel.
The above descriptions of the proliferation of mines and key mine warfare variables makes it easy to see why the United States Navy has a desire to continually innovate its mine countermeasures or MCM. MCM operations include a passive and active thread of operations. Active MCM can be broken down into two separate modes: mine sweeping and minehunting. Mine sweeping is definitely the more overt version of the two. It is usually performed by dragging something that either displaces mines or detonates them safely. Minehunting can be overt or covert. It has been decomposed into 5 steps: a detection phase where targets are initially noted for later reinvestigation, a classification phase where targets are classified as either mine-like or nonmine-like, and a localization phase in which contact positions are refined as precisely as possible, then an identification phase in which targets are identified using SONAR or optic sensors, and neutralization is the final step, in which the mine is rendered inoperative or removed.
In studying these operations, the team found that there’s an obvious material risk involved with any vessel in a minefield. There’s also a human risk for both explosive ordinance disposal divers and personnel aboard a MCM vessel. The team also learned that current post mission analysis entails MCM operators watching playback of SONAR data. So for every hour of SONAR data created, an operator must sit for an hour and watch it. There also exists an identified personnel requirement problem. Personnel must currently be on hand to maintain and prepare MCM equipment, operate that equipment, and deploy and recover that equipment.
An early step in tackling any complex problem with multiple solutions is to identify the benefits each solution may bring forth. This was precisely the type of project this grand challenge turned out to be. To put it concisely, the team’s goal was to quantitatively assess the impacts of autonomous technologies on naval MCM operations. The final deliverable was determined to be a tool that achieves such a capability. This tool was determined to require MCM operations data that could be used to generate a surrogate model that could then be tweaked to show improvements over a baseline that any autonomous technologies may bring. This baseline was set as a fully manual MCM operation. The fully autonomous MCM incorporated the use of computer aided detection, classification, and identification. It also included automated storage and retrieval of the unmanned systems from their containers, to the water, and back. The team also incorporated automated pre-mission planning for the MCM operation as well.
The first part of the approach to this project involved defining the problem scope of the MCM Mission. Our research considered only underwater vehicle minehunting scenarios, though multiple other techniques are used from MCM, such as minesweeping, USV minehunting, and more direct methods such as EOD divers. The scenarios researched were assumed to be in shallow water, with “favorable” environmental conditions to allow the use of common sensor and vehicle systems. The scope of the project included both onboard and offboard operations, with storage, deployment, search, and recovery operations all addressed.
mission effectiveness metrics were identified based on literature review, and down selected upon to produce a list of metrics by which mission performance could be quantitatively graded. These metrics included:
∗Time on mission: Total time from planning to post-mission storage
∗Confidence: Assurance that an area has been searched accurately
∗Utilization of resources: Measure of efficient asset usage
∗Cost of automation: Operating and support costs related to autonomy
Target Recognition Effectiveness
∗Detection performance: Ability to detect undersea targets at range and depth
∗Classification performance: Ability of humans or autonomous systems to distinguish Mine-Like Objects
After identifying mission effectiveness metrics, a Design Reference Mission (DRM) was developed. The DRM encompassed all operations from the initial order to perform a minehunting operation to replacing the equipment used in the proper storage facilities at the end of the operation. Each step in the mission was broken down in detail, based on available literature, and resources and time necessary to complete each step were estimated. These estimates formed the baseline mission into which autonomous technologies were introduced and the effects measured with the MEMs. The mission steps included:
1. Evaluate Mission
2. Generate Mission Plan
3. Remove Vehicles from Storage
4. Deploy Vehicles
5. Transit to Location
6. Begin Sensor Operation
8. End Sensor Operation
9. Transit from Target Location
10. Recover Vehicles
11. Maintenance and Post-Mission Analysis
12. Store Vehicles
With the baseline mission detailed, two simulation environments were obtained and parametric simulations developed. These were the Arena software, which is a Discrete Event Simulation software used for the onboard operations, and an open-source autonomous vehicle simulation software called MOOS for the search operations. Each simulation suite was used to develop the MEM measurements, with effects from both simulations integrated in the parametric design tool, IMAGE. The onboard operations were sensitive to parallel operations and process times, which are readily dealt with using Discrete Event Simulation such as Arena. The offboard operations were sensitive to the quantity of vehicles used in the search operations, and this was readily simulated using MOOS. Finally, various trust levels were used as levels of autonomy of the vehicle systems. These trust levels were defined based on literature review, and governed the tradeoff of workload between human and autonomous system agents. These trust levels had enormous effects on the MEMs, showing that higher trust levels in the autonomous systems and dramatically beneficial results in the MEMs, particularly in cases with multiple vehicle operations.
Analysis and Results
The parametric analysis performed involved combining the results of the shipboard operations simulation with the offboard operations simulation.
This analysis was performed by investigating the system configuration design space using the design of experiments. These intelligently selected cases were simulated, and the results used to develop response surface equations. After developing the response surface equations, they were validated against the model and development iterated as necessary. Error predictions and statistical checks were performed to maintain the most robustness and accuracy of the response surface equations.
The results of the analysis, in RSE form, were integrated and visualized in the Integrated Minehunting Autonomy Gap-Analysis Environment (IMAGE). This tool allowed rapid implementation of global autonomy characteristics, and easy visualization of the process performance metric trades. Primary metrics of interest included Mission Time, Utilizations, and Confidence. Most of the settings of the simulation environments were made available to the user to customize, allowing freedom for realtime mission development and trade studies.
The results of the research culminated in a Notional Case Study of minehunting operations in the Strait of Hormuz. This case study began a Human Operation with naval officers overseeing operations, a single crew deploying the vehicles, and human sensor operators review data to classify and identify mines. The results of these settings were then compared to an Autonomous Operation, where a computer system automatically planned the mission, automated systems deployed vehicles in parallel, and artificial intelligence was used to review data to find mines. The case study showed that human involvement was dramatically reduced, while the mission cost was increased. These results are expected when replacing human operators with automated equipment.
The project culminated with providing the sponsoring agencies with the IMAGE tool for use in rapid tradeoff studies and assessment of autonomous minehunting capabilities.
The final tool derived from this project, although wide in scope, is only as good as the underlying assumptions and models used to generate the data. This is true for any type of parametric analysis, but especially so for projects such as this one, which are very complex and have many variables in play. More than the number of variables, however, is the vast number of alternatives that autonomous behaviors may produce. This fact is one aspect of the many limiting factors of the tool. The fidelity of the tool could be greatly extended by the addition of more complex, more flexible autonomous behaviors. On the flip side of this, it would be a tremendous undertaking to incorporate such behaviors. So, although the tool is in some ways notional, it is very representative of what can be expected from the introduction of autonomy.
Outside of the need for more complex behaviors, the tool could also see improvements in its sensor modeling, which leads directly to improvements in realistic detection and classification. The team was able to prove the validity of the methodology behind this tool and convinced the Navy to share a copy of its simulator Autonomous Littoral Warfare Systems Evaluator - Monte Carlo (ALWSE-MC). ALWSE-MC will provide similar scope in terms of the search phase operations, but with a fidelity that is known to the Navy. It will also provide more realistic variables and variable interactions, such as SONAR reverberation and its confounded variables like water depth, temperature gradients, and frequencies.