Towards Extracting Highlights From Recorded Live Videos: An Implicit Crowdsourcing Approach

Live streaming platforms need to store a lot of recorded live videos
on a daily basis. An important problem is how to automatically
extract highlights (i.e., attractive short video clips) from these massive, long recorded live videos. One approach is to directly apply
a highlight detection algorithm to video content. While various
algorithms have been proposed, it is still hard to generalize them
well to different types of videos without large training data or high
computing resources. In this paper, we propose to tackle this problem with a novel implicit crowdsourcing approach, called Lightor.
The key insight is to collect users’ natural interactions with a live
streaming platform, and then leverage them to detect highlights.
Lightor consists of two major components. Highlight Initializer
collects time-stamped chat messages from a live video and then
uses them to predict approximate highlight positions. Highlight
Extractor keeps track of how users interact with these approximate
highlight positions and then refines these positions iteratively. We
find that the collected user chat and interaction data are very noisy,
and propose effective techniques to deal with noise. Lightor can
be easily deployed into existing live streaming platforms, or be
implemented as a web browser extension. We recruit hundreds of
users from Amazon Mechanical Turk, and evaluate the performance
of Lightor on real live video data. The results show that Lightor
can achieve high extraction precision with a small set of training
data and low computing resources.