Compiling FFmpeg for TDM - Revision history

Stgatilov: 2.08: conan is used to build it now

2019-04-11T03:35:49Z

2.08: conan is used to build it now

← Older revision		Revision as of 03:35, 11 April 2019
Line 1:		Line 1:
	This is a guide for building FFmpeg library manually.		This is a guide for building FFmpeg library manually.
	It describes exactly how FFmpeg was built for TDM 2.07.		It describes exactly how FFmpeg was built for TDM 2.07.

			Starting from TDM 2.08, conan package manager is used to build all dependencies, including FFmpeg.
			Thus, this article should '''no''' longer be used as a guide to action.
			It is still valuable for in-depth explanation of how and why is FFmpeg configured.
			See more info in [[Libraries and Dependencies]].

	'''Note:''' If you decide to change FFmpeg version, then you have to rebuild it for all the four official platforms: Windows/Linux 32/64-bit.		'''Note:''' If you decide to change FFmpeg version, then you have to rebuild it for all the four official platforms: Windows/Linux 32/64-bit.

Petike: adding category tag

2018-11-22T04:37:19Z

adding category tag

← Older revision		Revision as of 04:37, 22 November 2018
Line 306:		Line 306:

	[[Category:Coding]]		[[Category:Coding]]
			[[Category:Video and cutscenes]]

Stgatilov: Added a note about multithreading

2018-07-30T16:24:56Z

Added a note about multithreading

← Older revision		Revision as of 16:24, 30 July 2018
Line 196:		Line 196:
	If some excessive options are present, disable them directly by adding appropriate flag in <tt>tdm_configure.sh</tt>.		If some excessive options are present, disable them directly by adding appropriate flag in <tt>tdm_configure.sh</tt>.
	Then rerun this script in MSYS console until you get desired result.		Then rerun this script in MSYS console until you get desired result.

			Make sure that '''multithreading is enabled''': w32threads on Windows and pthreads on Linux.


	===== Build FFmpeg =====		===== Build FFmpeg =====

Stgatilov: Wrote how to build 32-bit Linux binaries

2018-06-24T09:32:32Z

Wrote how to build 32-bit Linux binaries

← Older revision		Revision as of 09:32, 24 June 2018
Line 4:		Line 4:
	'''Note:''' If you decide to change FFmpeg version, then you have to rebuild it for all the four official platforms: Windows/Linux 32/64-bit.		'''Note:''' If you decide to change FFmpeg version, then you have to rebuild it for all the four official platforms: Windows/Linux 32/64-bit.
	This is because all platforms use same headers, so if you rebuild it only on some platforms, then the other platforms will have headers/libs conflict.		This is because all platforms use same headers, so if you rebuild it only on some platforms, then the other platforms will have headers/libs conflict.



Line 52:		Line 53:

	This guide mostly follows [https://trac.ffmpeg.org/wiki/CompilationGuide/MSVC the official compilation guide on FFmpeg wiki].		This guide mostly follows [https://trac.ffmpeg.org/wiki/CompilationGuide/MSVC the official compilation guide on FFmpeg wiki].


	===== Install MSYS =====		===== Install MSYS =====
Line 59:		Line 59:

	In my case its root directory is <tt>C:\msys64</tt>.		In my case its root directory is <tt>C:\msys64</tt>.


	===== Install yasm =====		===== Install yasm =====
Line 68:		Line 67:

	Rename/copy the downloaded executable as <tt>yasm.exe</tt>. For instance, I have it as <tt>C:\yasm\yasm.exe</tt>.		Rename/copy the downloaded executable as <tt>yasm.exe</tt>. For instance, I have it as <tt>C:\yasm\yasm.exe</tt>.


	===== Prepare MSYS =====		===== Prepare MSYS =====
Line 87:		Line 85:

	Note: It seems that MSYS uses [https://wiki.archlinux.org/index.php/pacman pacman] manager for packages, which is also used on ArchLinux.		Note: It seems that MSYS uses [https://wiki.archlinux.org/index.php/pacman pacman] manager for packages, which is also used on ArchLinux.


	===== Open MSYS console properly =====		===== Open MSYS console properly =====
Line 103:		Line 100:
	The parameter <tt>-use-full-path</tt> is important here: without it environment variables will be dropped.		The parameter <tt>-use-full-path</tt> is important here: without it environment variables will be dropped.
	You can check that everything is still OK by running <tt>cl</tt> and <tt>yasm</tt> commands in the MSYS console.		You can check that everything is still OK by running <tt>cl</tt> and <tt>yasm</tt> commands in the MSYS console.


	===== Configure FFmpeg =====		===== Configure FFmpeg =====
Line 200:		Line 196:
	If some excessive options are present, disable them directly by adding appropriate flag in <tt>tdm_configure.sh</tt>.		If some excessive options are present, disable them directly by adding appropriate flag in <tt>tdm_configure.sh</tt>.
	Then rerun this script in MSYS console until you get desired result.		Then rerun this script in MSYS console until you get desired result.


	===== Build FFmpeg =====		===== Build FFmpeg =====
Line 275:		Line 270:

	Finally, [http://wiki.thedarkmod.com/index.php?title=The_Dark_Mod_-_Compilation_Guide#Linux rebuild x64 version of TDM as usual] and proceed to '''Testing''' section.		Finally, [http://wiki.thedarkmod.com/index.php?title=The_Dark_Mod_-_Compilation_Guide#Linux rebuild x64 version of TDM as usual] and proceed to '''Testing''' section.


	===== 32-bit version =====		===== 32-bit version =====

	~~TODO~~		To build 32-bit Linux version, you have to follow the same general instructions (starting from section '''Build on Linux'''), with the following differences:
			# Set <tt>--arch=x86_32</tt> instead of <tt>--arch=x86_64</tt> in configure script.
			# Copy resulting binaries to <tt>C:\TheDarkMod\darkmod_src\linux\ffmpeg\</tt>.
			However, the is one important caveat: the <tt>--arch</tt> options is '''not''' enough to change bitness of your build !

			You have two options to workaround the problem:
			# Use 32-bit Linux OS --- yeah, the second one. It is rather easy to do if you use VM.
			# Add an additional option <tt>--cc="gcc -m32"</tt> into the configure script. Then you can hopefully build 32-bit FFmpeg on 64-bit Linux.

Stgatilov: /* Build FFmpeg */

2018-06-24T06:02:57Z

Build FFmpeg

← Older revision		Revision as of 06:02, 24 June 2018
Line 272:		Line 272:

	Now search for all the <tt>.a</tt> files in the source code directory --- these are the built libs.		Now search for all the <tt>.a</tt> files in the source code directory --- these are the built libs.
	Copy them to <tt>C:\TheDarkMod\darkmod_src\linux\ffmpeg\</tt>.		Copy them to <tt>C:\TheDarkMod\darkmod_src\linux\ffmpeg\lib64\</tt>.

	Finally, [http://wiki.thedarkmod.com/index.php?title=The_Dark_Mod_-_Compilation_Guide#Linux rebuild x64 version of TDM as usual] and proceed to '''Testing''' section.		Finally, [http://wiki.thedarkmod.com/index.php?title=The_Dark_Mod_-_Compilation_Guide#Linux rebuild x64 version of TDM as usual] and proceed to '''Testing''' section.

Stgatilov: Added Linux 64-bit build instructions

2018-06-24T05:18:59Z

Added Linux 64-bit build instructions

← Older revision		Revision as of 05:18, 24 June 2018
Line 237:		Line 237:

	= Build on Linux =		= Build on Linux =

			The steps on Linux are generally similar to the ones on Windows, but greatly simplified.

			You need a 64-bit Linux (preferably Ubuntu) with GCC already installed.
			A virtual machine is perfectly OK, just share FFmpeg source code directory with it.
			All the steps are completed in the Linux terminal.

			===== Install yasm =====

			sudo apt-get install yasm

			===== Configure FFmpeg =====

			Go to source code root directory, which in my case is <tt>/media/sf_TheDarkMod/ffmpeg-3.3.7</tt>.

			Edit <tt>tdm_configure.sh</tt> script you have from Windows build:
			# Remove <tt>--toolchain=msvc</tt> option from it.
			# Set <tt>--arch=x86_64</tt> for initial 64-bit build.

			Now run the script:
			./tdm_configure.sh
			If it complains with something like "--disable-debug: command not found", then you need to convert EOLs:
			sudo apt-get install dos2unix
			dos2unix tdm_configure.sh
			After that run <tt>./tdm_configure.sh</tt> again.

			Check the output of configure script, make sure it is the same as on Windows.
			If you see something excessive, add a flag to disable it in <tt>tdm_configure.sh</tt> and rerun the script.

			===== Build FFmpeg =====

			Run the make command:
			make -j6

			Now search for all the <tt>.a</tt> files in the source code directory --- these are the built libs.
			Copy them to <tt>C:\TheDarkMod\darkmod_src\linux\ffmpeg\</tt>.

			Finally, [http://wiki.thedarkmod.com/index.php?title=The_Dark_Mod_-_Compilation_Guide#Linux rebuild x64 version of TDM as usual] and proceed to '''Testing''' section.

			===== 32-bit version =====

	TODO		TODO

Stgatilov: Added section about testing FFmpeg build

2018-06-23T18:50:10Z

Added section about testing FFmpeg build

← Older revision		Revision as of 18:50, 23 June 2018
Line 244:		Line 244:
	= Testing =		= Testing =

	~~TODO: prepare a set of test video files, explain how to easily run them with TDM~~.		This testing routine covers all the enabled features. It should be performed for each supported platform.

			Run TheDarkMod game and start any mission. "Closemouthed Shadows" FM suits well because it is fast to load.

			When in game, open game console and execute the following commands '''one by one''':

			testVideo video/ffmpeg_test/sykes_roq_roq.roq
			testVideo video/ffmpeg_test/looped_m4v_h264.m4v
			testVideo video/ffmpeg_test/tennis_mp4_h264_aac.mp4 withAudio
			testVideo video/ffmpeg_test/tearsofsteel_avi_mpeg4_mp3.avi withAudio

			After each command, a video must be shown on the screen. View it up to the end.

			Ensure that video plays properly, that sound is played when withAudio is specified, that no serious warnings appear in console.

	[[Category:Coding]]		[[Category:Coding]]

Stgatilov: First version: includes Windows-side build instructions.

2018-06-23T09:31:44Z

First version: includes Windows-side build instructions.

New page

This is a guide for building FFmpeg library manually.
It describes exactly how FFmpeg was built for TDM 2.07.

'''Note:''' If you decide to change FFmpeg version, then you have to rebuild it for all the four official platforms: Windows/Linux 32/64-bit.
This is because all platforms use same headers, so if you rebuild it only on some platforms, then the other platforms will have headers/libs conflict.

= Build: common steps =

As a first step on any platform, you have to download [https://www.ffmpeg.org/download.html#releases FFmpeg source code].
It is recommended to choose the major/minor version which was ''cut from master'' at least a year ago, and take the latest build of it.

Unpack the source code into some directory. In my case it is at <tt>C:\TheDarkMod\ffmpeg-3.3.7\</tt>.

In the FFmpeg root directory, create a text file named <tt>tdm_configure.sh</tt> with the following contents:
./configure \
--toolchain=msvc \
--disable-debug \
--arch=x86_64 \
--enable-static --disable-shared \
--enable-gpl \
--disable-all \
--disable-autodetect \
--enable-avformat --enable-avcodec --enable-swresample --enable-swscale \
--enable-demuxer=roq,mov,m4v,avi \
--enable-parser=h264,mpeg4video,aac,mpegaudio \
--enable-decoder=roq,h264,mpeg4,aac,mp3 \

This is the command for configuring FFmpeg. The parameters are most important here.

The basic parameters include:
* <tt>--toolchain=msvc</tt>: Use Visual C++ compiler in build.
* <tt>--disable-debug</tt>: Strip debug information, otherwise the resulting libs will be huge.
* <tt>--arch=x86_64</tt>: Build 64-bit version.
* <tt>--enable-static --disable-shared</tt>: Make sure to get static libraries.
* <tt>--enable-gpl</tt>: Make sure resulting binaries are GPL-licensed (just as TDM is).

The next set of parameters is necessary for disabling feature which we don't explicitly allow:
* <tt>--disable-all</tt>: Disable all the features of FFmpeg.
* <tt>--disable-autodetect</tt>: Disable all the external libraries.
Note: If your FFmpeg package does not support <tt>--disable-autodetect</tt> yet, then you have to manually disable every external library which FFmpeg tries to use in your case.

The remaining parameters enable back precisely the features which are necessary for TDM:
* <tt>--enable-avformat --enable-avcodec --enable-swresample --enable-swscale</tt>: Build these four libraries.
* <tt>--enable-demuxer=roq,mov,m4v,avi</tt>: Support this set of containers.
* <tt>--enable-decoder=roq,h264,mpeg4,aac,mp3</tt>: Include this set of codecs (for decoding only).
* <tt>--enable-parser=h264,mpeg4video,aac,mpegaudio</tt>: Include this set of parsers. Note that they cover all the decoders included.

= Build on Windows =

This guide mostly follows [https://trac.ffmpeg.org/wiki/CompilationGuide/MSVC the official compilation guide on FFmpeg wiki].

===== Install MSYS =====

Download and install [https://www.msys2.org/ MSYS2 for windows]. Better choose 64-bit version, e.g. [http://repo.msys2.org/distrib/x86_64/msys2-x86_64-20180531.exe msys2-x86_64-20180531.exe].

In my case its root directory is <tt>C:\msys64</tt>.

===== Install yasm =====

Download [https://yasm.tortall.net/Download.html yasm] as a single exe file. Better choose 64-bit version, e.g. [http://www.tortall.net/projects/yasm/releases/yasm-1.3.0-win64.exe yasm-1.3.0-win64.exe].

No installation is required. I put the executable into <tt>C:\yasm\</tt>.

Rename/copy the downloaded executable as <tt>yasm.exe</tt>. For instance, I have it as <tt>C:\yasm\yasm.exe</tt>.

===== Prepare MSYS =====

Run MSYS console with <tt>msys2_shell.cmd</tt> located at MSYS root directory. The steps described here must be done in this console.

Sync package database and update all packages with command:
pacman -Syu
Agree to whatever it wants from you.

When it stops installing, close the console.
Then start the fresh console and run "<tt>pacman -Syu</tt>" command once again.

Now you need to install packages for <tt>make</tt> and <tt>cmp</tt> commands. So run the following:
pacman -S make
pacman -S diffutils
After that, close the MSYS console.

Note: It seems that MSYS uses [https://wiki.archlinux.org/index.php/pacman pacman] manager for packages, which is also used on ArchLinux.

===== Open MSYS console properly =====

Open Visual C++ command prompt with environment set for 64-bit MSVC compilation.
In my case it is called "x64 Native Tools Command Prompt for VS 2017", located in <tt>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars64.bat</tt>.
To check that you started proper console, run <tt>cl</tt> in it: it will print compiler version and bitness.

Add the directory with <tt>yasm.exe</tt> to PATH:
set PATH=%PATH%;C:\yasm
Check that it works by running <tt>yasm</tt> command: it must print something like "yasm: No input files specified".

Now open MSYS console '''from within the MSVC command prompt''':
C:\msys64\msys2_shell.cmd -use-full-path
The parameter <tt>-use-full-path</tt> is important here: without it environment variables will be dropped.
You can check that everything is still OK by running <tt>cl</tt> and <tt>yasm</tt> commands in the MSYS console.

===== Configure FFmpeg =====

Go to FFmpeg root directory within the MSYS console (note forward slashes):
cd C:/TheDarkMod/ffmpeg-3.3.7

Run the configuration script <tt>tdm_configure.sh</tt> you created at the very beginning:
./tdm_configure.sh

Review the output with full attention. In my case I get the following output (expand the collapsed section):
<div class="toccolours mw-collapsible mw-collapsed" style="width:1000px; overflow:auto;>
$ ./tdm_configure.sh
install prefix /usr/local
source path .
C compiler cl
C library msvcrt
ARCH x86 (generic)
big-endian no
runtime cpu detection yes
yasm yes
MMX enabled yes
MMXEXT enabled yes
3DNow! enabled yes
3DNow! extended enabled yes
SSE enabled yes
SSSE3 enabled yes
AESNI enabled yes
AVX enabled yes
XOP enabled yes
FMA3 enabled yes
FMA4 enabled yes
i686 features enabled yes
CMOV is fast yes
EBX available no
EBP available no
debug symbols no
strip symbols no
optimize for size no
optimizations yes
static yes
shared no
postprocessing support no
network support no
threading support w32threads
safe bitstream reader yes
texi2html enabled no
perl enabled no
pod2man enabled no
makeinfo enabled no
makeinfo supports HTML no

External libraries:

External libraries providing hardware acceleration:

Libraries:
avcodec avformat avutil swresample swscale

Programs:

Enabled decoders:
aac h263 h264 mp3 mpeg4 roq

Enabled encoders:

Enabled hwaccels:

Enabled parsers:
aac h263 h264 mpeg4video mpegaudio

Enabled demuxers:
avi m4v mov roq

Enabled muxers:

Enabled protocols:

Enabled filters:

Enabled bsfs:

Enabled indevs:

Enabled outdevs:

License: GPL version 2 or later
Creating configuration files ...
libavutil/avconfig.h is unchanged
libavcodec/bsf_list.c is unchanged
libavformat/protocol_list.c is unchanged

WARNING: pkg-config not found, library detection may fail.
</div>
You should have the same features listed in your output.
If some excessive options are present, disable them directly by adding appropriate flag in <tt>tdm_configure.sh</tt>.
Then rerun this script in MSYS console until you get desired result.

===== Build FFmpeg =====

After FFmpeg has been configured properly, run build in the MSYS console:
make install -j6
The parameter <tt>-j6</tt> sets the number of build threads, set it to the number of CPU cores you have.
When build finishes successfully, close the MSYS console.

Now it remains only to copy all the files to proper place for TDM.
In my case, build results are located in <tt>C:\msys64\usr\local</tt> directory.
Suppose that svn root of TDM source code is located at <tt>C:\TheDarkMod\darkmod_src</tt>.
Do the following:

# Delete directory <tt>C:\TheDarkMod\darkmod_src\include\ffmpeg</tt> with old include files.
# Copy <tt>C:\msys64\usr\local\include</tt> directory to <tt>C:\TheDarkMod\darkmod_src\include\ffmpeg</tt>.
# Delete all the libraries which look like parts of FFmpeg from <tt>C:\TheDarkMod\darkmod_src\win64\lib</tt>.
# Copy all the five produced .a files from <tt>C:\msys64\usr\local\lib</tt> to <tt>C:\TheDarkMod\darkmod_src\win64\lib</tt>.

After you have copied everything, clean all the trash produced by FFmpeg build:
make distclean

Then rebuild TDM 64-bit in Visual Studio and proceed to the '''Testing''' section to make sure everything is built properly.

===== 32-bit version =====

It is also necessary to build 32-bit version.
To achieve it, do all the same steps starting from '''Open MSYS console properly''' section, with the following changes:

# In <tt>tdm_configure.sh</tt>, change <tt>--arch=x86_64</tt> to <tt>--arch=x86_32</tt>.
# When opening MSVC command prompt, open the 32-bit version.
# When copying build results, not that 32-bit libraries are located in <tt>C:\TheDarkMod\darkmod_src\win32\lib</tt> instead of <tt>C:\TheDarkMod\darkmod_src\win64\lib</tt>.

= Build on Linux =

TODO

= Testing =

TODO: prepare a set of test video files, explain how to easily run them with TDM.

[[Category:Coding]]