Astri Camera Pipeline

About Astri

Astri Group is a leading retail developer and operator in Estonia, managing six major shopping centers across the country, including Lõunakeskus, Pärnu Keskus, Astri Keskus, Fama Keskus, Balti Jaama Turg, and Keskturg. The company has been in the business for over 30 years, focusing on creating vibrant commercial environments that blend retail, entertainment, and community spaces. Astri Group is committed to sustainability, offering modernized and eco-friendly shopping experiences that cater to both local and international brands.

The Challenge

Astri's extensive network of malls is equipped with CCTV cameras that track visitor counts. However, the cameras are notoriously unpredictable, with issues ranging from poor APIs to power outages and network disruptions, making data extraction difficult.

The previous system struggled with these challenges, often leading to downtime and data synchronization problems. Moreover, the inability to isolate issues from one camera or mall without affecting the entire system further complicated the situation, necessitating a more robust and scalable solution.

The Result

The newly developed system for Astri is a robust, scalable, and efficient solution designed to overcome the complexities of managing data from unpredictable CCTV cameras across multiple malls.

Central to this system is the use of an AWS SQS-based message queuing architecture that allows multiple consumers to process messages independently. This design ensures that if a message fails to be resolved, it can be retried by any available consumer after some time, significantly enhancing the system's scalability and fault tolerance.

The use of a Turborepo Monorepo structure, combined with Docker, further optimizes development by enabling the rapid, independent deployment of various Node.js processes, effectively isolating tasks in a microservices-like manner. This setup is particularly beneficial for light data analytics, ensuring that these processes remain isolated and efficient.

Moreover, the integration of a Next.js dashboard provides a streamlined interface for stakeholders to monitor system performance, access basic analytics, and download reports, making the system not only powerful but also user-friendly.

Enhanced monitoring tools, such as Axiom for logging and Slack-based alerts, ensure real-time awareness of system status, allowing for prompt resolution of any issues. This comprehensive solution not only addresses Astri’s previous challenges but also sets a new standard for reliability and scalability in managing retail data analytics.