TL;DR

NYU researchers created SeeUnsafe, an AI system that automatically analyses traffic video to identify collisions and near-misses using multimodal language-vision models. The system achieved 76.71% classification accuracy on test data and won New York City’s Vision Zero Research Award. Transportation agencies can now leverage existing camera infrastructure for proactive safety interventions without manual video review.

Unlocking Existing Infrastructure Value

New York City operates thousands of traffic cameras capturing continuous footage, but analysing this volume manually exceeds most transportation agencies’ resources. NYU Tandon School of Engineering researchers developed an artificial intelligence system combining language reasoning and visual intelligence to automatically identify safety incidents in existing video feeds.

The SeeUnsafe system represents one of the first applications of multimodal large language models to long-form traffic video analysis. Professor Kaan Ozbay notes that with thousands of cameras running continuously, manual examination is untenable—SeeUnsafe enables cities to extract full value from their existing camera investments without requiring computer vision expertise or custom training data.

Proactive Risk Identification

Tested on the Toyota Woven Traffic Safety dataset, SeeUnsafe correctly classified videos as collisions, near-misses, or normal traffic 76.71% of the time, with road user identification success rates reaching 87.5%. The system generates natural language “road safety reports” explaining its decisions, describing factors like weather conditions, traffic volume, and specific movements leading to incidents.

Traditional safety interventions occur only after accidents. By analysing near-miss patterns—vehicles passing too close to pedestrians or risky intersection manoeuvres—agencies can proactively identify danger zones and implement preventive measures like improved signage, optimised signal timing, and redesigned road layouts before serious accidents occur.

Looking Forward

Whilst the system faces limitations including sensitivity to object tracking accuracy and low-light challenges, it establishes a foundation for AI-powered road safety analysis from vast video archives. Researchers suggest extending the approach to in-vehicle dash cameras for real-time risk assessment from drivers’ perspectives, potentially transforming how cities and vehicles collaborate to prevent accidents before they happen.

Source: TechXplore