We investigate the problem of efficient panoramic video construction based on time-synchronized input video streams. No additional constraints are imposed regarding the motion of the capturing video cameras. The presented work is, to the best of our knowledge, the first attempt to construct in real-time a panoramic video stream from input video streams captured by freely moving cameras. The main contribution is in proposing an efficient panoramic video construction algorithm that exploits temporal information to avoid solving the stitching problem fully on a frame by frame basis. We provide detailed experimental evaluation of different methodologies that employ previous frames stitching results such as tracking interest points using optical flow and using areas of overlap to limit the search space for interest points. Our results clearly indicate that making use of temporal information in video stitching can achieve a significant reduction in execution time while providing a comparable effectiveness.