Blog

My journey with GNOME (Pitivi) so far…

In my last post, I talked about how an email on GNOME’s newcomers email list helped me to make up my mind for participating in GSoC 2018. In this blog post, I will talk about my journey of contributing to Pitivi so far. So, LETS GET STARTED…

When I first went to the IRC channel of Pitivi, my first message on the channel was “Hi, I would like to contribute to Pitivi and take up a project for GSoC 2018. I have good exposure to Python. I also have experience working with large code bases. How can I start contributing to Pitivi. Thanks!”. I didn’t get a reply at that moment but I received an email next day from Alexandru Băluț (one of the current maintainers of Pitivi and primarily known with his IRC nick “aleb”). He actually searched my profile on GitHub, took my email address from there, and sent me an email with all the relevant information and links. The effort aleb put in just to answer the question I asked on the IRC channel was IMPRESSIVE.

After setting up my dev environment, I dived a little bit into Pitivi’s code structure. After this, I went to Pitivi’s bug tracker. They already had some bugs (or issues) tagged for newcomers. After going through those issues, I assigned the issue #1861 to myself and started working on it. I closed this issue within 10 days (link to commit). After this I took the issue #1399. This issue was more of a feature request – the main idea was to implement shift click functionality in Pitivi to allow for selection of multiple clips in the project timeline. While working on this issue, I realized that there is a bug in Pitivi that needs to be fixed first before implementing this feature request. I reported the bug to aleb and he filed it in (#2155). In the meantime I worked on the issue #1951. This issue required diving into GES code and finding out why the logic of pasting clips was failing. GES is the back-end of Pitivi written in C programming language. After going through GES code, I figured out the issue with logic of pasting clips and mentioned it in the commit message (link to commit). While working on this issue aleb and thiblahute (Thibault Saunier – current maintainer of Pitivi and GES) realized that it would be good API-wise to implement a new API call for pasting clips onto the timeline. So I filed a bug #793820 in Bugzilla and closed it after implementing the new API call (link to commit). After this, I got back to issue #2155. While working on this issue, I realized that the bug was in GES and not in Pitivi. The bug was caused due to improper refcount handling in ges_layer_get_clips_in_interval(start, end) function in GES. I filed a bug #793874 in Bugzilla and closed it after resolving the bug (link to commit). This bug was blocking issue #1399 and once it got resolved, I went back to this issue. In the meantime, I worked on issues #2161 and #2171 and resolved them (link to commit closing issue #2161) (link to commit closing issue #2171). After working on the issue #1399 for about a month, I finally got the feature for selecting multiple clips in a project’s timeline using shift + click pushed into Pitivi’s master (link to commit).

My journey of contributing to Pitivi so far has been WONDERFUL. It has been an amazing learning experience for me so far. Contributing to such a large code base has also improved my software development skills – I would like to thank aleb specially for  this because he keeps telling me how to improve my code through code reviews.

I hope that this journey continues in the time to come 🙂

Advertisements

How I got drawn into GSoC 2018?

Participating in GSoC was nowhere in my mind at the start of this year, but an email changed a lot of things and pulled me into GSoC 2018. Lets flashback a little…

I thought of participating in GSoC 2016. GNOME was my organization of choice at that time also primarily because of my love for the cool and useful desktop applications they make for Linux and also because most of their applications are developed using C/C++ which was my favorite language at that time (I was yet not awestruck by Python and Java). Around Feb 2016, I joined GNOME newcomers email list, hanged around a bit, setup my gnome developer environment and looked at few bugs but I never got around to actually solving them primarily because I lacked determination and motivation that is required to contribute to open source for GSoC.

I joined B.Tech. program of IIIT-Delhi in monsoon of 2014. This winter 2018 semester would have been my last semester but because I took a semester leave in 2017 for my 6 month internship at Amazon, India, I have one more semester to do, i.e. I will graduate in Dec 2018. I got a job offer at the end of my internship to join Amazon in Dec 2018 after I complete my degree. In a nutshell all it means is that I didn’t had anything planned for the upcoming summers when I started this semester (I was still going through my options).

One thing I failed to mention is that I never unsubscribed from GNOME newcomers email list. It all started in Jan 2018 from a query someone asked on this list “How can I get into GSoC GNOME project?”. Surprisingly until then the thought of doing GSoC in summers didn’t cross my mind. The moment I read the mail, I got what I was looking for – something cool and fun to do in summers with pretty decent stipend (and obviously something to brag in my CV and social media platforms).  The next thing I did was to look for a project under GNOME to work on, but this time I was more interested in Python or Java related projects. I looked at a few of them and finally decided to give a shot to Pitivi – a non-linear video editing application for Linux platform.

I think this is it for this blog post. I will share my journey of contributing to GNOME so far and how I got selected in GSoC 2018 in another blog post.

STAY TUNED 🙂

Simple and robust algorithm for video thumbnail generation

The simplest algorithm that comes to mind for generating a thumbnail for a video is to randomly pick a frame in the video (even simpler could be to just pick the first frame of the video). While the algorithm is very intuitive and looks good at first glance, it has one problem – it can pick a monotonous frame, like an all black frame, which obviously is not a good thumbnail because it doesn’t convey any information regarding the video.

A good heuristic algorithm (referred from this stackoverflow post) to tackle this problem is to select a few random frames (let say, 5) in the video and pick the frame that has largest file size as video thumbnail – the idea being that the JPEG of a monotonous frame will compress into much smaller file than that of an interesting frame (one with lots of objects and colors).

The approach mentioned in the algorithm looks neat and hence I thought of evaluating it on few videos and see for myself if it really solves the problem of picking monotonous frames as video thumbnails.

I evaluated it on 10 videos – 2 random videos each (720p at 24 FPS) from Friend’s (American sitcom) season 3 to 7. I divided the video frames into 3 categories – bad, interesting, and transition. Monotonous frames are bad frames. Interesting frames are those that contain lots of objects and colors. The frames which lead to transition from bad to interesting frames, or vice versa, are transition frames. For each video, I picked 5 frames randomly out of the first 120 frames (initial 5s of the video) which consists of 2 bad frames, 1 transition frame, and 2 interesting frames.

Here are the results:

[Test Video 1] Friends S3 E3

This slideshow requires JavaScript.

The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –

114_interesting
Interesting Frame 1 – 164 KB File Size

[Test Video 2] Friends S3 E16

This slideshow requires JavaScript.

The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –

54_transition
Transition Frame – 170 KB File Size

[Test Video 3] Friends S4 E5

This slideshow requires JavaScript.

The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –

104_interesting
Interesting Frame 1 – 250 KB File Size

[Test Video 4] Friends S4 E13

This slideshow requires JavaScript.

The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –

99_interesting
Interesting Frame 2 – 150 KB File Size

[Test Video 5] Friends S5 E7

This slideshow requires JavaScript.

The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –

93_interesting
Interesting Frame 2 – 234 KB File Size

[Test Video 6] Friends S5 E22

This slideshow requires JavaScript.

The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –

116_interesting
Interesting Frame 2 – 150 KB File Size

[Test Video 7] Friends S6 E10

This slideshow requires JavaScript.

The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –

98_interesting
Interesting Frame 1 – 238 KB File Size

[Test Video 8] Friends S6 E24

This slideshow requires JavaScript.

The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –

80_interesting
Interesting Frame 1 – 228 KB File Size

[Test Video 9] Friends S7 E1

This slideshow requires JavaScript.

The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –

89_interesting
Interesting Frame 1 – 223 KB File Size

[Test Video 10] Friends S7 E19

This slideshow requires JavaScript.

The above slideshow is displaying the 5 randomly picked frames along with their respective types and JPEG file sizes. The thumbnail for this video is –

101_interesting
Interesting Frame 2 – 246 KB File Size

Out of the 10 thumbnails selected by the algorithm, 9 are interesting frames and 1 is transition frame. The algorithm did a very good job of not picking monotonous frames as video thumbnails.

P.S. This algorithm is not a perfect solution but it certainly does a very good job of generating a good thumbnail for a video.