Privacy-First AI: How Computer Vision and Federated Learning are Transforming Crisis-Resilient Digital Twins

In a recent Gemini Call presentation, Charalampos Karyotis from Interactive Coventry demonstrated how advanced AI and computer vision technologies can create resilient, privacy-respecting digital twins for urban environments—systems designed to operate precisely when cities need them most.

The Perfect Storm: When Cities Need Intelligence Most

Cities worldwide face an escalating crisis of extreme weather events. As Charalampos noted from his decade of research experience, what was once confined to specific regions has become a global phenomenon. “Ten years ago, I was trying to put together research proposals on detecting floods… I was finding very severe cases but in very specific regions. Nowadays, I found these phenomena happening more and more often in areas where it wasn’t expected—in Turkey, in Italy, even in the Middle East.”

This expanding threat landscape exposes a critical paradox: existing CCTV systems are static, creating blind spots across cities, yet expanding surveillance raises serious privacy and GDPR concerns. Meanwhile, when disasters strike—precisely when cities need real-time intelligence most—traditional networks often fail. Interactive Coventry’s solution tackles all three challenges simultaneously through two innovative projects: CODEMIA and IMMERSE.

CODEMIA: Building the Foundation for Privacy-Preserving Urban Intelligence

Developed in partnership with Singapore’s City Matrix, CODEMIA represents a fundamental rethink of urban monitoring systems. The project’s main pilot deployment in Singapore (March 2024 to February 2025) demonstrated how cities can achieve comprehensive monitoring whilst respecting privacy.

The Technical Architecture

CODEMIA’s innovation lies in its layered AI approach that processes multiple data streams:

• CCTV and mobile camera feeds
• Weather application data
• Open city data (bus stop occupancy, traffic patterns)
• Custom IoT sensors

Rather than transmitting raw video feeds—a privacy nightmare—the system processes data at the edge, extracting only numerical insights: three people here, five vehicles there, road surface wet. These anonymised metrics are then transmitted to partners’ VR environments, creating a real-time digital twin without compromising individual privacy.

Real-Time Anonymisation: The Privacy Breakthrough

When questioned about the trade-off between anonymisation and temporal consistency, Charalampos revealed their uncompromising approach: “We set our priorities… we couldn’t get around the anonymisation aspect.” The system processes video streams in near real-time, anonymising faces and number plates before any analysis occurs. While this introduces minimal delay, Charalampos argues the trade-off is worthwhile: “We felt that the delay… really didn’t matter for us if the same thing was detected just a few seconds ago and was replicated some moments later.”

This privacy-first approach extends beyond anonymisation. By transmitting only processed results rather than raw streams, the system dramatically reduces both bandwidth requirements and privacy risks—crucial advantages during crisis situations when networks are stressed.

IMMERSE: Engineering Resilience for Crisis Response

Building on CODEMIA’s foundation, IMMERSE addresses feedback from real-world deployments, particularly the need for systems that continue operating when disasters strike. The project introduces several critical innovations:

Federated Transfer Learning: Intelligence at the Edge

IMMERSE employs federated transfer learning, allowing AI models to learn and adapt locally without centralising sensitive data. As Charalampos explained: “An edge node… can update the model weights based on incoming data and then transmit only the model weights to a central node.”

This approach offers multiple benefits:

• Models become more intelligent without transmitting raw data
• Privacy regulations are respected by design
• Systems can quickly adapt to new cities through transfer learning
• Local processing continues even when central networks fail

From VR to AR: The Human Factor

Interestingly, IMMERSE shifted from VR to augmented reality based on operator feedback. “When a disaster is happening and operators need to react quickly, it’s good to have connection with the actual environment,” Charalampos explained. This design decision reflects a mature understanding that technology must serve human needs, not vice versa.

Satellite Failover: Ensuring Continuity

Perhaps most critically, IMMERSE includes automatic failover to satellite networks when terrestrial systems fail. This seemingly simple addition ensures the system operates precisely when traditional infrastructure collapses—during floods, fires, and other disasters.

Drone Integration: Extending the Digital Twin’s Reach

IMMERSE’s drone capabilities address a specific challenge identified in Malaysian flood responses: inaccessible areas where traditional monitoring fails. The system provides two crucial functions:

• Safe landing zone identification for supply drops
• Collision avoidance with other drones in crowded airspace

Computer vision modules detect hazards including fires, floods, and accidents, whilst scene understanding algorithms identify safe operational zones. This creates a dynamic, three-dimensional understanding of crisis situations that static sensors alone cannot provide.

The Human-in-the-Loop Imperative

When asked about liability for AI failures, Charalampos acknowledged the reality: “No AI is perfect. We will maybe miss some things.” Interactive Coventry’s solution embraces human oversight as a feature, not a bug. Operators can provide direct feedback when AI makes errors, creating a reinforcement learning loop that continuously improves system accuracy.

This human-centred approach extends to interface design. Charalampos envisions future systems where generative AI creates natural language interfaces: “Imagine you pass this information to generative AI… someone may be able to interact in natural language, ask what is happening there. The generative AI could reply that I have seen this many people and this type of event, and based on your policies I suggest you respond like this.”

Scalability Through Modularity

Both systems employ modular architectures that allow cities to select capabilities based on specific needs. “If someone is interested in traffic, they can keep the traffic modules. If someone is interested in monitoring pedestrian movements, they can focus on that,” Karyotis explained. This flexibility reduces costs whilst ensuring systems remain relevant to local requirements.

The commercial equipment approach further democratises access. Rather than requiring specialised hardware, the systems work with off-the-shelf cameras, drones, and even citizen-provided devices. This philosophy extends to deployment: transfer learning allows pre-trained models to adapt quickly to new cities with minimal local training data.

Future Horizons: Quantum Computing and Digital Twin Evolution

Looking ahead, Karyotis identified several emerging opportunities:

Citizen Participatory Sensing

Every device becomes a potential data source, enabling crowd-sourced crisis response

Explainable AI Integration

Understanding not just what AI detects, but why it makes specific decisions

Quantum Machine Learning

Potentially revolutionary improvements in processing speed for complex simulations

Industrial Applications

Extending urban techniques to critical infrastructure and manufacturing

Key Lessons for Digital Twin Practitioners

Interactive Coventry’s work offers crucial insights for organisations developing crisis-resilient digital twins:

1. Privacy by Design: Anonymisation and edge processing aren’t afterthoughts—they’re foundational
2. Embrace Failure Modes: Systems must operate when networks fail, not just when everything works perfectly
3. Human-Centred Design: Technology choices (like AR versus VR) should prioritise operator effectiveness
4. Modular Architecture: One-size-fits-all solutions fail; cities need customisable capabilities
5. Federated Intelligence: Keep data local, share only insights and model improvements

The Path to Resilient Cities

As extreme weather events become increasingly common and unpredictable, cities need digital twins that operate reliably during crises, respect citizen privacy, and adapt quickly to changing conditions. Interactive Coventry’s CODEMIA and IMMERSE projects demonstrate that these goals aren’t mutually exclusive—they’re mutually reinforcing.

By processing data at the edge, maintaining privacy by design, and building in redundancy from the ground up, these systems create urban intelligence that’s both powerful and trustworthy. As cities worldwide grapple with climate change, urbanisation, and the need for sustainable growth, such approaches offer a blueprint for digital twins that serve citizens when they need them most.

The Digital Twin Hub’s weekly Gemini Calls continue every Tuesday at 10:30am BST, showcasing innovations shaping the future of digital twin technology. All sessions are recorded and available to community members.

Ready to build privacy-preserving, crisis-resilient digital twins for your city? Join the Digital Twin Hub community to connect with pioneers developing these next-generation urban intelligence systems.