The simplest algorithm that comes to mind for generating a thumbnail for a video is to randomly pick a frame in the video (even simpler could be to just pick the first frame of the video). While the algorithm is very intuitive and looks good at first glance, it has one problem – it can pick a monotonous frame, like an all black frame, which obviously is not a good thumbnail because it doesn’t convey any information regarding the video.
A good heuristic algorithm (referred from this stackoverflow post) to tackle this problem is to select a few random frames (let say, 5) in the video and pick the frame that has largest file size as video thumbnail – the idea being that the JPEG of a monotonous frame will compress into much smaller file than that of an interesting frame (one with lots of objects and colors).
The approach mentioned in the algorithm looks neat and hence I thought of evaluating it on few videos and see for myself if it really solves the problem of picking monotonous frames as video thumbnails.
I evaluated it on 10 videos – 2 random videos each (720p at 24 FPS) from Friend’s (American sitcom) season 3 to 7. I divided the video frames into 3 categories – bad, interesting, and transition. Monotonous frames are bad frames. Interesting frames are those that contain lots of objects and colors. The frames which lead to transition from bad to interesting frames, or vice versa, are transition frames. For each video, I picked 5 frames randomly out of the first 120 frames (initial 5s of the video) which consists of 2 bad frames, 1 transition frame, and 2 interesting frames.
Here are the results:
[Test Video 1] Friends S3 E3
The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –
[Test Video 2] Friends S3 E16
The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –
[Test Video 3] Friends S4 E5
The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –
[Test Video 4] Friends S4 E13
The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –
[Test Video 5] Friends S5 E7
The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –
[Test Video 6] Friends S5 E22
The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –
[Test Video 7] Friends S6 E10
The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –
[Test Video 8] Friends S6 E24
The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –
[Test Video 9] Friends S7 E1
The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –
[Test Video 10] Friends S7 E19
The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –
Out of the 10 thumbnails selected by the algorithm, 9 are interesting frames and 1 is transition frame. The algorithm did a very good job of not picking monotonous frames as video thumbnails.
P.S. This algorithm is not a perfect solution but it certainly does a very good job of generating a good thumbnail for a video.
My Journey in Open Source World 
Probably not a good idea to link to illegal media downloads, Harish.
You can also see some of the places where the approach doesn’t get you a good result, so I’d love to see some more analysis and options if possible.
LikeLike