This article is 1st in the series of creating advanced video analytics solutions at scale
Next BIG in Artificial Intelligence and Computer Vision is the vision of a SAFE AND SECURE SMART CITY
For Data Scientists there are a lot of challenges with Video Analytics — having to maintain costly hardware , highly efficient and accurate models , inference with speed and deployment at scale
Without meeting the above challenges an organisation cannot meet business needs such as
- Security Alarm System — Imagine a space where every camera is converted into an IOT sensor and can easily pickup Threat behaviour / Weapon detection and can send an immidiate alarm to the central security and automatically start tracking the person responsible
- Access Control — Access Controls via Facial Recognition are becoming more popular POST Covid , Creating an Access Control system with scale still remains a challenge for most organisations
- Retail Analytics — Retail stores are constaly looking for advanced intelligent insights to predict consumer behaviour and find innovative ways to attract more customers , Post Covid era has accelerated the adoption of online shopping and thus physical stores / malls are feeling the punch. With Intelligent Video Analytics smart solutions can be created to attract more customers and provide them the same thrill and personalised customer experience requires in this era.
Today , millions of cameras are deployed throughtout the states in order to achieve better traffic management , hospital management , security , construction management , logistics management.
All these camera generate PETA Bytes of Data which need to be streamlined with Artificial Intelligence in order to produce useful video analytics for supporting the ecosystem.
INTRODUCTING NVIDIA DEEPSTREAM
Nvidia supports end to end video analytics via the Deepstream module. With deepstream Data Scientists can utilize the Nvidia Transfer Learning Toolkit to train a custom Image Classification / Object Detection Model with their own use case to create a highly accurate / fast model optimised to run on Nvidia Edge Devices
NVIDIA TRANSFER LEARNING TOOLKIT
Transfer Learning Toolkit provides pre-built models for specific use cases in Smart City applications / other than that Data scientists can easily start with a TLT Model to optimise their own use case.
Custom models can be created by the workflow pipeline as mentioned below
First step is Data Preparation (TLT expects the data in a certain format , for more information please visit the official documentation , it provides excellent resources to get started with Nvidia Transfer Learning Toolkit (https://docs.nvidia.com/metropolis/TLT/tlt-getting-started-guide/index.html)
TLT requires a configuration file which can be used to specity certain instructions while the training takes place , for example no of GPUs to utilise , hyperparameters , Ouput model etc
After training pruning is an efficient way to decrease the overall size of the model to be run efficiently on an NVIDIA Edge device such as Jetson Nano. Pruning reduces the size of the model considerably while maintaining the overall accuracy of the model.
After pruning the model below is an estimate of the accuracy achieved with the pre-built models provided with Transfer Learning Toolkit
Once the model is created we can start integrating the model with Deepstream SDK
With Deepstream 5.0 NVIDIA has release PYTHON BINDINGS which means exactly as you understood , with deepstream 5 all the python community can start exploring the magic of highly optimised intelligent video applications
The Entire Deepstream stack is supported on NVIDIA Computing platform which supports NVIDIA Edge devices such as Jetson and Servers such as TESLA
With the latest release of Deepstream now the SDK supports Bi-Directional IOT messaging which means the communication can be made from the edge devices to the cloud as well as from the cloud to the edge devices
For Example the edge devices such as Jetson Nano can be used to inference on the edge while sending the inference data to an Azure IOT Hub.
Azure IOT Hub can also send an instruction back to individual edge devices to start recotding the video stream if a threat is detected.
Deepstream SDK also comes packed up with Triton Inference Server which allows Data Scientists to deploy model to deepstream directly from their favourite platforms such as Tensorflow / Pytorch / MXNet etc withouth having to convert the modes into a TensorRT engine — We will discuss in detail about Triton Inference Server later in a later series
Above image shows an end to end pipeline on how the data is collected from the camera as RTSP streams and analytics are shown on the cloud
In the next series we will discuss more in depth about deepstream architecture and how deepstream can be used more efficientlly to build a safe and efficient ecosystem utilising the existing inftastructure.