1 Research Motivation
Robotic Swarms (RSs) have been widely utilized for their capability to tackle complex and repetitive tasks, including package delivery [
1], surveillance [
2], etc. Recently, these systems have been utilized for more strategic and tactical purposes, e.g., the use of Unmanned Aerial/Ground Vehicles (UAVs/UGVs) to enhance situational awareness in emergencies and aid law enforcement agencies. However, most existing swarm deployments are homogeneous and rely on commands from human administrators or operation centers [
3]. This homogeneous and centralized nature constrains the size and capability of RSs as manual decision-making is slow and inefficient with limited computational resources.
These existing swarm architectures are also incapable of continuous evolution and adaptation to dynamic environments, which limits their usage to relatively confined spaces and controlled scenarios. To improve efficiency and reliability, researchers traditionally rely on non-learning methods, which are based on, among others, control, graph, and game theories [
4,
5]. These methods provide solid theoretical foundations and transparent interpretability. However, they usually require precise modeling of the system [
6] and tend to simplify the dynamics of complex systems [
7], which limits their adaptability to evolving environments. The computation burden of many non-learning algorithms explodes as the size of the problem increases, especially those regarding combinatorial optimization problems [
8]. Most importantly, these methods also do not provide any systematic evolution mechanism that can enable the swarm to learn from its experience. Machine learning-based methods, on the other hand, provide excellent scalability and adaptability while dealing with high-dimensional input data. They outperform non-learning methods in various applications, such as aircraft aerobatic control [
9]. However, they are notorious as black boxes and are less trusted for safety-critical missions. Moreover, imposing constraints (e.g., battery capacity, collision avoidance, etc.) or prior knowledge to pure Deep Learning (DL) models is challenging and introduce additional optimization bottlenecks [
10].
Thus, we propose a neural-symbolic resilient threat-response framework for large-scale hierarchical swarms, combining the best of machine learning and non- learning theories to comprehend patterns in high-dimensional data while imparting reasoning and interpretability.
Figure 3 presents an example neural-symbolic architecture for persistent coverage problem, where a multi-agent system is designated to monitor a region, keeping every grid frequently inspected. The perception neural network efficiently observes the region and generates waypoints. Given waypoints, the MPC module generates trajectories following various constraints (e.g., collision avoidance).
Figure 4 is a specific example of resilient persistent coverage. The region of interests is discretized and the shade of blue indicates the “awareness” of a grid. Red circles are obstacles, yellow arrows are agents, and green crosses denote MPC predictions. The MPC module of the circled agent is adjusting its trajectory to avoid the obstacle (
Fig. 1 and
Fig. 2).
The major features of the proposed framework include:
1. Hierarchical structure: Our proposed framework is well suited for hierarchical heterogeneous robotic swarms, which follow the existing cloud-fog- edge structure and consist of various types of robots (e.g., UAVs, UGVs, etc.) [
11]. Using our proposed framework, regional data can be aggregated in the fog layer, which relieves the computation and communication burden and enhances system resiliency by reducing the reliance on the cloud. The diversity of robot types and performance enriches the collected data, thus, accelerates the evolution of the swarm.
2. Neural-symbolic approaches: Our proposed framework employs neural-symbolic methods across most phases of threat response, from network configuration and task assignment to motion planning and control. Powered by the high-level DL model, our neural- symbolic methods possess superior scalability and adaptability. At the same time, with constraints imposed on low-level non-learning modules, the final outputs are trustworthy and explainable, making the overall system resilient to unknown threats.
3. Real-time threat detection and continuous learning: With both proactive and reactive strategies for threat mitigation, our framework greatly enhances the swarm’s resilience to unexpected threats and anomalies. Our continuous learning strategies enable collective and task-specific evolution of the entire swarm network, exploiting the hierarchical structure.
2 Proactive Strategies
In this section, we demonstrate the proactive aspect of our framework by leveraging AI-enabled network vulnerability analysis and resilient motion planning with a neural-symbolic scheme to enhance the resilience of the hierarchical swarm.
2.1 Analysis on Network Vulnerability Under Potential Threats
Evaluating network vulnerabilities among edge agents is crucial to enhancing the resilience of large-scale swarm operations. Given the collaborative behaviors between agents (e.g., autonomous vehicles and UAVs) for the missions, improving
network connectivity is a critical concern that must be addressed. For instance, with respect to surveillance missions, the authors in [
12] have indicated that enhanced network connectivity between agents is crucial for real-time coordination, data sharing, and dynamic task allocation, which enables more efficient operations. In other words, it can improve decision-making and situational awareness by allowing the agents to collaborate, share sensor data, and adapt to changing mission conditions. However, in adversarial environments, a swarm’s network connectivity may fall below a desirable level due to malicious impacts (e.g., cyberattacks and unexpected threats). This can lead to poor mission performance and numerous safety violations, such as crashes and collisions, between agents and/or obstacles.
Additionally, as the number of agents increases, understanding the dynamics of the network through the non-learning model-based approaches (e.g., using Laplacian matrix and geometric properties) may encounter technical challenges due to the curse of dimensionality; and unknown dynamics and effects [
13]. Therefore, it is crucial to develop a resilient strategy that allows the swarm to maintain robust network connectivity despite unintentional malicious threats, including Denial-of-Service (DoS) cyberattacks.
Scenario 1: AI-Driven Resilient Coordination of Swarms: In the real world, attackers (those who launch cyberattacks) have limited resources and/or budgets to launch their attack strategies/policies. They will likely target the most critical network links or agents that play a significant role in maintaining important network features, such as network connectivity. With advancements in computational technologies, AI-driven frameworks with neural-symbolic schemes (e.g., pLogicNet and Express GNN) [
14] could be promising solutions to address this problem. Considering the coordination and control policies of agents, these AI- driven techniques enable us to predict/estimate which agents (even a group of agents) are more vulnerable to cyberattacks (e.g., DoS attacks). Additionally, these methods can classify the level of attack vulnerabilities for each agent, enabling ground stations—entities that manage swarm operations—to create resilient coordination/motion strategies to counter potential (unforeseen) cyber threats. Therefore, our proposed method ensures the resiliency of the hierarchical swarm system.
2.2 Neural-Symbolic Resilient Motion Planning For Swarms
Motion planning is a fundamental operation for almost every advanced mission for robotic swarms. Existing swarm systems employ control-theoretic approaches, such as distributed Model Predictive Control (MPC) and consensus formation control. These approaches have shown great success in generating efficient motions while maintaining state and input constraints (e.g., collision avoidance and battery constraint). However, these methods heavily rely on an exact model of robots in the swarm and environment, which is challenging to achieve in real-world scenarios [
15]. Deep learning approaches, on the other hand, are good at environmental perception and dealing with the imperfect model. But they demand a huge amount of expert supervision and hard to interpret.
To this end, we introduce Imperative Learning (IL) [
16], where deep learning models are employed to perceive the complex environment for task allocation, and control-theoretic methods are used to handle motion planning.
Scenario 2: Persistent Coverage with Continuous Connectivity: A fundamental problem in the task of RSs, including surveillance and sensor deployment, can be modeled as a persistent coverage problem, where a group of robots should monitor certain areas to maintain a desired coverage level. Traditionally, this problem has been formulated as a constrained optimization problem when a group of robots tries to optimize a density function representing the quality of coverage under fuel constraints. These challenges are exacerbated when nefarious actors, e.g., hackers, manipulate the behavior of some of the robots by spoof- ing sensors or hacking communication channels. Thus, it becomes essential to determine resilient motion planning for each robot in an RS to persistently cover a given environment under collision and connectivity constraints. Connectivity constraints ensure that the RS effectively utilizes locally detected information as an additional redundancy to mitigate the effects of faults and/or cyberattacks. To address this, we propose a neural-symbolic method including three major components: 1) a distributed DNN that perceives its local environment and allocates target waypoints to robots, 2) an MPC-based safe motion planning for each robot given the desired waypoints while satisfying obstacle and connectivity constraints, and 3) a rigid body theory-based anomaly detector, running under the assumption of connectivity.
3 Reactive Strategies
This section focuses on reactive strategies which enable an RS to evolve and adapt continuously from collective experiences. Using an federated learning framework, we demonstrate how to simultaneously achieve global evolution and regional specification. We further describe a reallocation framework that allows a swift reorganization of the swarm’s hierarchy once an anomaly or fault is detected.
3.1 Continuous Evolution and Adaptation
For all creatures, it is vital to constantly learn from experience and adapt to environment. This is the same for an RS to stay resilient and efficient in complex environments. RSs also have a huge advantage over natural creatures in that individuals in the swarm can rapidly exchange their experiences and evolve. However, such swift evolution cannot be implemented on RSs with a centralized swarm structure. This is because the centralized structure leads to a dilemma between data processing ability and the manufacturing cost of terminal agents. Moreover, agents can never perform optimally if they are all identical and do not adapt to specific situations. A promising solution to this dilemma is to combine federated learning with a hierarchical swarm structure. With the fog-level agents performing local pre-processing, the cloud server only needs to deal with highly abstract features, which relieves computation and communication workload and ensures security and privacy. This cloud-fog-edge structure is decentralized due to lower-level computations being done at the fog- and edge-level, and does not require constant communication with the central cloud [
11]. Encrypted parameters of a global model are downloaded by the agents from a cloud server, and model training is done locally so that sensitive information stays in the local system. Significant research has been done on encryption techniques for federated learning that could be used to provide sufficient privacy without impacting performance, such as homomorphic encryption and secret sharing technology. The encrypted gradient information from the agents is transmitted back to the cloud server for aggregation and update to the global model [
17].
Scenario 3: Federated Learning for Hierarchical Swarm The graph structure of the swarm may change frequently due to either mission requirements or unexpected anomalies. Incorporating federated learning with a hierarchical swarm structure, we can leverage the intermediate fog layer between the cloud and edge layers to tailor the global model for each fog. To do so, the fog agent can add a bias to the global model, specifying the task and environment of the local system. When an updated model is sent from the edge agents to the cloud for aggregation, the fog agent can also be used to remove the fog-specific bias.
This method allows edge agents to be quickly reassigned between fogs with negligible performance degradation due to the fog-level agent handling adaptation of the global model to the local swarm’s goal and environment.
Anomalies can also be invaluable learning opportunities for the swarm, making the swarm resilient to similar threats in the future. Directly transmitting raw data history is inefficient and insecure, not to mention the damaged agent could shut down very soon after the anomaly happens. Inspired by the idea of one- shot federated learning [
18], we propose a method where each edge agent (or its fog instead) periodically distills and stores an
encrypted gradient message. With local distillation in advance, the size of transmitted data is compressed significantly, yet containing most of the important information. Also, this communication form protects the privacy of raw data since it submits encrypted gradient information to the swarm.
3.2 Reallocation of Swarm Hierarchy
Another critical challenge in reactive threat response is reassigning the swarm hierarchy once agents are identified as faulty or attacked. It leads to a very complex combinatorial optimization problem to select proper agents to form a “special force” and hand over the unfinished work, considering the energy constraints, interventions to current plans, and the heterogeneity of the swarm. Fortunately, active research [
19] is going on to obtain near-optimal solutions for large-scale combinatorial optimization problems. Combining the idea of imperative learning, we can simultaneously achieve fast computing, scalability, and interpretability.
Scenario 4: Reorganizing Local Delivery Network: The delivery problem can be formulated as a Multiple Traveling Salesman Problem (MTSP), which is well- known for its NP-hardness. State-of-the-art non-learning approaches can only handle about MTSP with about 150 cities. Based on a novel imperative learning approach [
20], we can first decompose the MTSP into a multiple single-agent Traveling Salesman Problem (TSP). Then, we employ a non-learning solver to efficiently solve these sub-problems. This innovative framework can solve the MTSP with dozens of agents and thousands of cities within a few seconds, which can satisfy the demands of any current RS.
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (
http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third-party material in this chapter are included in the chapter’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
PDF
(1492KB)
