Hem

AI for Safety-Critical Systems

Future autonomous systems (drone swarms, mobile robot fleets, autonomous vehicles) must operate safely among humans without relying on a central controller. This research programme develops methods for safe, trustworthy, and adaptive multi-agent AI: systems where agents monitor themselves and each other, detect anomalies, and adapt their collective behaviour to maintain safety and mission effectiveness.

A distinctive feature of this programme is the DSS-AS duality: the same monitoring and logging infrastructure that enables real-time runtime assurance for Autonomous Systems (AS) also produces structured audit trails reusable for human Decision Support Systems (DSS), including forensic analysis, investigative triage, and regulatory compliance. Methods developed for one mode directly serve the other.

Research Focus

The Zenith-funded project centres on developing distributed monitoring methods that integrate formal specifications (temporal logics) with robust learned models of normal behaviour, enabling anomaly detection across heterogeneous robot platforms in dynamic, human-populated environments. The project combines three interconnected threads:

1. Distributed Monitoring with Learned and Formal Models

How can runtime monitoring be extended from single agents to cooperative teams? This thread develops methods that combine temporal logic specifications with probabilistic normality models learned from operational data. When Agent A observes Agent B behaving unexpectedly, the system must determine whether the anomaly is genuine (sensor failure, actuator degradation) or an artefact of observation error, model mismatch, or communication delay. The focus is on practical, scalable monitoring architectures for heterogeneous platforms with varying communication patterns.

2. Auditable Decision Traces and DSS Integration

Safety-critical systems must produce structured event traces and causal-temporal explanations that faithfully capture the detect-attribute-replan-execute loop. This thread develops logging and explanation methods that serve both real-time operators and post-hoc forensic analysis, bridging the gap between autonomous runtime assurance and human decision support. This is where the DSS-AS duality is realised in practice: the same data structures and anomaly correlation methods support both autonomous operation and investigative workflows.

3. Applied Validation with Industrial and Societal Partners

Methods are validated progressively through simulation, experiments at the LiU UAV Laboratory, and pilot studies with partners. This applied validation thread ensures that research outcomes translate to operational deployment and address EU AI Act obligations for high-risk systems.

Vision and Impact

The long-term aim is to establish a research group at the intersection of formal methods, machine learning, and multi-robot systems, focused on making autonomous teams safe, transparent, and accountable by design. The closed-loop methodology (anomaly detection, fault attribution, coordinated replanning, and auditable logging) is engineered to support EU AI Act obligations for high-risk systems, including continuous risk management, traceability, transparency, and post-market monitoring.

The breadth of partner support, from justice and law enforcement to industry and national AI strategy, confirms that the research addresses central national priorities. Results will be validated progressively through simulation, experiments at the LiU UAV Laboratory, and pilot studies with partners.

About the Project