Hacking FFmpeg With Python – Part One

Hack FFmpeg With Python, Part Two

FFmpeg is a great multimedia framework that helps you deal with your audio and video files. What do I mean by this? Well, you can easily convert from one format to another, extract audio from a video, compress a video and even extract pictures from a video. There are many cool things you can do with this tool. It is written mostly in C programming language and the best part is that FFmpeg is open source.

Through this tutorial you are going to learn how to use the python programming language for interacting with this open source tool in order to automate some simple tasks.

Setting up python and ffmpeg on your linux box

The python version being used through this article is Python 2.7.x so make sure to install it on your own linux machine before going any further as a different version may give you all kind of errors when running the code in the python interactive shell.

Compiling code from source is a little bit tricky, but since it is needed to cover this installation for almost any kind of linux distribution going through all the package managers is not possible.

Open the terminal and download the right python package using the wget utility, as shown here:

wget --no-check-certificate https://www.python.org/ftp/python/2.7.11/
Python-2.7.11.tgz

Note: wget is a command line utility which can be used to download stuff over the http protocol.

Once the python package 2.7.x has finished downloading the next command line tool which is needed is tar:

tar -xzf Python-2.7.11.tgz

Change the working directory by using the following command:

cd Python-2.7.11

To compile and install python on your machine run the following commands, one by one. This is the last step in ensuring that you have the right version of python installed. Without it, you won’t be able to take advantage of this FFmpeg hacking series:

./configure  
make  
sudo make install

Before compiling and installing FFmpeg it’s always a good idea to test the new version of python you just installed.

The following command can help to run python2.7.x on your linux machine to check it:

python2.7

If you get the python shell showing up it means that this part of the setup is done correctly.

The same thing should be done for having FFmpeg utility on your nix box. The first step is to download the source code from the official site by using the command shown below:

wget https://github.com/FFmpeg/FFmpeg/archive/master.zip

As you can see from the above command the wget tool is used to download the source code, which we need to compile.

The command for unzipping the .zip archive is as follows:

unzip master.zip

Once the directory is ready, run the following command to make sure the configuration is done right:

./configure

If the above command is completed without any errors it means you can easily compile and install FFmpeg on your machine.

Use the following commands:

make
su -c 'make install'

Once the installation of FFmpeg is finished we highly recommend you do the same test as we did with the python shell to make sure the tool works correctly.

The following command can be used to run FFmpeg from the command line:

ffmpeg

A few practical FFMpeg examples

Before going straight into hacking FFmpeg by using a programming language such as Python, it is recommended that you warm up with some of the features this leading multimedia open source framework has to offer.

Included in the installation of the FFmpeg box is the ffprobe utility. It can be used to get information about an audio or video file such as the streams, format or both using the following code:

ffprobe -show_format -show_streams test.mp4

The above command probes the ‘test.mp4’ video for format tags and streams. The output displayed on console by this command is very lengthy, as shown below.

[STREAM]
index=1
codec_name=aac
codec_long_name=AAC (Advanced Audio Coding)
profile=LC
codec_type=audio
codec_time_base=1/44100
codec_tag_string=mp4a
codec_tag=0x6134706d
sample_fmt=fltp
sample_rate=44100
channels=2
channel_layout=stereo
bits_per_sample=0
id=N/A
r_frame_rate=0/0
avg_frame_rate=0/0
time_base=1/44100
start_pts=0
start_time=0.000000
duration_ts=9262080
duration=210.024490
bit_rate=95999
max_bit_rate=103664
bits_per_raw_sample=N/A
nb_frames=9045
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=1
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
DISPOSITION:timed_thumbnails=0
TAG:creation_time=2015-04-23T08:06:29.000000Z
TAG:language=eng
TAG:handler_name=IsoMedia File Produced by Google, 5-11-2011
[/STREAM]
[FORMAT]
filename=test.mp4
nb_streams=2
nb_programs=0
format_name=mov,mp4,m4a,3gp,3g2,mj2
format_long_name=QuickTime / MOV
start_time=0.000000
duration=210.023333
size=8691199
bit_rate=331056
probe_score=100
TAG:major_brand=mp42
TAG:minor_version=0
TAG:compatible_brands=isommp42
TAG:creation_time=2015-04-23T08:06:28.000000Z
[/FORMAT]

The output produced by this command gives us a lot of information on the streams of the file. Each stream has its own index. The first stream has an index of 0, but for the purpose of this article not all the streams probed by ffprobe are displayed here.

Suppose one wants to extract just the audio stream from a music video. The perfect command from my experience is the one which copies the audio stream without re-encoding it:

ffmpeg -i video.mp4 -vn -acodec copy audio.aac

The above command is really useful when it comes to extracting the audio stream. However, there is one big problem with it. Copying the audio stream needs the right extension which in the above case is .acc. This is where ffprobe becomes really useful!

Finding the correct extension for saving the audio file which is being copied is as easy as taking a look at the codec_name value inside the audio stream information that comes from probing the file with the following ffprobe command line utility:

[STREAM]
index=1
codec_name=aac

........
........
.......
[/STREAM]

For more information on how to use the ffmpeg command line utility read our article on practical commands that can be used in real situations.

Start using python to interact with ffmpeg

Python is a high level computer programming language perfect for scripting stuff one likes to automate. One does not need to be a python expert to follow this tutorial or utilize other Python codes. Even a beginner in coding can follow, understand and run the codes being shared through this article.

Fire up a python shell using the following command on your terminal:

python2.7

There are many modules available in the standard library python offers. The one which truly fits our need is the subprocess library.

To make use of a python module, first you must import it using the following command:

import subprocess

But what is the purpose of the module we just imported? According to the official python documentation the subprocess module allows you to spawn a process. In others words you can run a program via python. Not only this, but you can also connect to its input/output pipes and even obtain return codes.

The following example is really a good one to give you a better idea of how the subprocess module can be used to spawn a process.

p = subprocess.Popen('ls', stderr=subprocess.PIPE, stdin=subprocess.PIPE, 
stdout=subprocess.PIPE)

The above python code runs the ls command which lists files inside a directory. Try it yourself!

Then, get the output of the above opened process by communicating with it as shown below.

output, _ = p.communicate()

To print the output, this python print statement can be used:

print(_)

The above statement is going to display the output returned by the communication with the process. If everything worked correctly you should get a list of the files and subdirectories inside the current working directory.

Python can also help to spawn a process for the ffmpeg utility as shown in the following code:

cmds = ['/usr/local/bin/ffmpeg']
ffmpeg_p = subprocess.Popen(cmds, stdin=subprocess.PIPE,
                            stdout=subprocess.PIPE, 
                            stderr=subprocess.PIPE)

 Communicate with the process to get the output:

output, _ = ffmpeg_p.communicate()

Print the output returned by ffmpeg on the console by using this python print statement:

print(_)

The above print statement will print the following:

ffmpeg version 3.2 Copyright (c) 2000-2016 the FFmpeg developers
  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.2 --enable-shared 
--enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables 
--enable-avresample --cc=clang --host-cflags= --host-ldflags= 
--enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl 
--disable-lzma --enable-vda
  libavutil      55. 34.100 / 55. 34.100
  libavcodec     57. 64.100 / 57. 64.100
  libavformat    57. 56.100 / 57. 56.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options]
 outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

It is the same thing as running the ffmpeg command on the console, but this time we make use of python to do the job for us.

Getting the streams and format of the video file with python is as easy as defining the commands needed for spawning the process with the subprocess module.

In our case the commands are as follows:

cmds = ['/usr/local/bin/ffprobe', '-show_format', '-show_streams', 
'test.mp4']

As you can see we store the commands in a python object which is know as a list. The syntax for defining a list is the following:

[]

 Then open a new process and pass the cmds as an argument, as shown below:

ffprobe_p = subprocess.Popen(cmds, stdin=subprocess.PIPE, 
stdout=subprocess.PIPE, stderr=subprocess.PIPE)

To get the streams and the format of the video file you need to make use of the communicate method like we did in the previous examples. Use this code:

output, _ = ffprobe_p.communicate()

Then use the print statement to print the streams and the format information returned by the probing operation.

Conclusion

FFmpeg is not just a geeky tool used by geeks like us! The multimedia industry makes use of it too. There is a lot of buzz around Youtube using it for encoding its videos. Memorizing the entire list of commands that this tool has to offer is not worth it, but this is a perfect case for when a scripting language such as Python becomes really useful.

 The fun is just getting started, let’s enjoy it!