2026 Theses Doctoral

Scalable Edge-Cloud Architectures for Real-Time Video Analytics Applications: Design and Deployment

Ghasemi Dehkordi, Mahshid

Modern urban areas are increasingly equipped with large-scale camera networks comprising tens of thousands of connected cameras. Transforming these raw video feeds into actionable intelligence for applications such as public safety, pedestrian awareness, and cloud-connected vehicles requires automated, real-time video analytics systems that can operate under stringent latency, bandwidth, and computational constraints. Realizing this vision demands significant efforts that span efficient system architectures, adaptive processing pipelines, and seamless integration across the edge-cloud computing hierarchy. In this thesis, we adopt an end-to-end systems approach and address several critical challenges in real-time video analytics, including accurate spatial sensing from diverse camera perspectives, adaptive streaming under dynamic network and scene conditions, hierarchical distributed processing across edge, cloud, and end-user devices, and a modular analytics backbone for diverse human-centric applications. Our systems are designed, implemented, and evaluated on the NSF COSMOS testbed (a city-scale programmable platform deployed in West Harlem, New York City) using its geo-distributed camera network and edge-cloud computing infrastructure.

In Part I, we focus on video analytics for urban spatial sensing and social distancing monitoring. We design and deploy two complementary pipelines: (i) Auto-SDA, a high-accuracy system for street-level cameras that introduces a high-precision multi-area camera calibration module, an ID correction module for robust pedestrian tracking, and a group detection module with over 85\% accuracy. (ii) B-SDA, a privacy-preserving system leveraging bird's-eye-view cameras with specialized pre-processing techniques and a customized YOLOv4 architecture optimized for small pedestrian detection from elevated viewpoints. Using extensive real-world datasets spanning pre-pandemic through post-vaccine periods, we present one of the first automated, video-based assessments of social distancing compliance during the COVID-19 pandemic, revealing that non-compliance increased from less than 55\% post-lockdown to 65\% post-vaccine while pedestrian density decreased by approximately 50\% compared to pre-pandemic levels. Our findings are consistent with contemporaneous national surveys.

In Part II, we design adaptive and distributed video analytics architectures. We present ASTRA, a low-overhead architecture for executing iterative adaptation algorithms in live video analytics pipelines that dynamically optimizes configuration parameters such as resolution and frame rate. ASTRA reduces switching latency by approximately 20×and per-iteration execution overhead by an order of magnitude. It achieves performance within 10% of the offline optimal with system reliability exceeding 90% and adaptation overhead averaging around only 2% GPU utilization per camera. We further present two systems for hierarchical video analytics across the compute continuum. PAVE distributes computation across edge servers and end-user mobile devices for privacy-preserving pedestrian safety. Our live field tests at an intersection in New York City show that PAVE can warn pedestrians at risk approximately 0.9 s before a vehicle reaches identified danger zones. PAVE preserves users’ privacy through a one-way communication design where pedestrian data never leaves personal devices. EdgeCloudAI uses lightweight edge pre-processing and Convolutional Neural Network (CNN)-based detection to trigger selective cloud-hosted Vision Language Model queries for detecting user-defined anomalies, including flooding, obstacles, and traffic incidents. EdgeCloudAI consumes only 57% of edge GPU capacity per camera and reduces cloud query costs by around 100×.

In Part III, we present U-VA Platform, a unified, modular video analytics backbone built on GStreamer C library, comprising three distinct planes: a Control Plane for offline and online parameter configuration, a Data Processing Plane for real-time analytics execution, and a Data Export Plane that exposes intermediate metadata at each pipeline stage to external systems. We demonstrate the U-VA Platform’s versatility through three human-centric applications: (i) StreetNav, a camera-based navigation system for blind and low-vision pedestrians that reduces path deviation by 53% compared to GPS-based alternatives and achieves 0.41 m localization error. (ii) An Urban Sensing framework that fuses camera-based pedestrian tracking with mobile Inertial Measurement Unit (IMU) sensor data for pedestrian-device matching. It achieves 75% online matching accuracy with 39 ms end-to-end latency. (iii) A Digital Twin system that maintains a synchronized digital replica of a physical intersection with 98.7% uptime over four weeks of continuous operation, enabling adaptive traffic signal control that reduces average pedestrian wait time from ∼42 s to ∼28 s.

Throughout this thesis, we make several system-level and algorithmic contributions to real-time video analytics, spanning from sensor-level processing to edge-cloud distributed architectures and end-user applications. All systems are validated through real-world deployments on the NSF COSMOS testbed, demonstrating their effectiveness under realistic urban conditions. Collectively, these contributions advance the state of the art in urban video analytics and provide a practical foundation for safer, more responsive, and privacy-aware smart city environments.