USENIX NSDI |
With the widespread deployment of video analytics applications, edge compute servers are preferred for the analytics of the videos (for reasons of bandwidth and privacy). DNNs are deployed on the edge servers for inference, but when out in the field, they suffer from data drift where the live video data is significantly different than the training data. Continuous learning techniques are designed to handle data drift by periodically retraining the models on new data. Our work addresses the challenge of jointly supporting inference and retraining tasks on edge servers. The fundamental tradeoff that we address in our work is the tradeoff between the retrained model’s accuracy and the inference accuracy, because during retraining, resources are diverted away from inference tasks thereby lowering their accuracy. We design a scheduling solution Ekya that balances this tradeoff, across multiple video streams simultaneously, and optimizes for the inference accuracy over the retraining window’s duration.
Ekya is a system which enables continuous learning on resource constrained devices. Given a set of video streams and pre-trained models, Ekya can continuously fine-tune the models to maximize accuracy by intelligently allocating resources between live inference and retraining in the background.
