The Dark Mod - Compilation Guide: Difference between revisions

From The DarkMod Wiki
Jump to navigationJump to search
No edit summary
 
(73 intermediate revisions by 12 users not shown)
Line 1: Line 1:
This guide should provide you with enough information to compile The Dark Mod's game code from source.
'''Intended article audience:''' engine coders


The sources are available through "snapshots", i.e. whenever the Dark Mod team is releasing a new update (e.g. TDM 1.01) the corresponding sources are released as well.
This guide should provide you with enough information to compile The Dark Mod's game code from source.
It applies to version 2.09 or later version of TDM. If you want to compile older version, see the history of this wiki page.
Brief compilation instructions are also included in <tt>COMPILING.txt</tt> file in the source code package.


== Download the sources ==
 
You can download the latest source package from our website: http://www.thedarkmod.com/download_source.php
== Get the sources ==
 
The sources are available through "snapshots", i.e. whenever the Dark Mod team is releasing a new update (e.g. TDM 2.09) the corresponding sources are released as well.
* You can download a source code archive from the Downloads page: http://www.thedarkmod.com/downloads/
The executables built from these sources are compatible with the latest official release of TDM.
 
 
Also you can get the very latest source code directly from SVN:
* Public read-only access: https://svn.thedarkmod.com/publicsvn/darkmod_src/trunk/
* Team members only: https://svn.thedarkmod.com/svn/darkmod_src/trunk 
 
Be warned though that these sources '''might be incompatible with your local darkmod installation'''.
To minimize the problem, developer builds of the game are available in tdm_installer.
Each developer build is named like <tt>devXXXXX-YYYY</tt>, where YYYY is the SVN revision number which this version was built from.
If you checkout this exact revision from the source code SVN and build executable from it, it will surely be compatible with the corresponding dev. build of TDM.
 
== Directory structure ==
 
Put the source code directory next to your game directory, such that the directory structure looks like this:
 
C:\Games\darkmod      <-- your darkmod installation
C:\Games\darkmod_src  <-- your source folder (containing the solution)
 
This directory structure is recommended for both Windows and Linux.
 
For windows users (this may also apply to linux users) - Install tortoise SVN and then right click on the \darkmod_src folder and select checkout. Then in the url field put 'https://svn.thedarkmod.com/publicsvn/darkmod_src/trunk/' then click ok (see screenshot below)
 
[[File:Darkmod src.jpg|thumb]]
{{clear}}


== Windows ==
== Windows ==
For Windows you'll need Visual Studio to compile the project, you can get the free Express editions from the Microsoft website. It should work with every version starting from VC++ 2005 upwards, although I'd recommend the VC++ 2008 one. If you go for the older VC++ 2005, you'll need the Platform SDK as well, which is another large download. You don't need a Platform SDK for VC++ 2008.


Download the sources and unpack them somewhere. You'll find a <tt>d3game.sln</tt> solution file (which is in VC++ 2005 format), which you can double-click to open in Visual Studio. The newer VC++ 2008 will present you an upgrade wizard to convert the solution to 2008.
Since TDM 2.11 you'll need Visual Studio 2022 to compile the project (VS2017 was used before that).
The free Community Edition works fine, it can be downloaded [https://visualstudio.microsoft.com/vs/ approximately here]. Additionally, you have to:
* Make sure "Visual C++ MFC for x86 and x64" is being installed by MSVC installer, along with "Desktop development with C++" workflow (see screenshot below).


Once opened, select the Configuration Type in the topmost toolbar (either "release" or "debug", depending on what you want to do) and hit "Build Solution" ({{Ctrl}}-{{Shift}}-{{key|B}}). The compilation usually takes a few minutes, watch the output window at the bottom of Visual Studio. Once completed, you'll find the compiled <tt>gamex86.dll</tt> in a subfolder named "releasedll" (for release builds" or "debugdll" (for debug builds).
[[File:Unknown.png|thumb]]{{clear}}


In order to "install" the gamex86.dll, you need to copy it somewhere where Doom3.exe can find it. There are two ways to do that: either you place the dll right next to your DOOM3.exe which will override any other mechanism (the game will always load the DLL right next to DOOM3.exe ''first''), or you put them in a PK4 (along with a binary.conf file) and place it in your mod folder. There is some info on www.iddevnet.com/doom3 about how to do that.
You'll find a <tt>TheDarkMod.sln</tt> solution file in the source code folder, which you can double-click to open in Visual Studio. The solution is designed to put the compiled binaries into the ../darkmod folder nearby, that's why we recommend using the directory layout shown above. (You can change the output paths in the property sheets, in case you know how to do that).


=== Debugging the Game ===
Once the solution is opened, select the Configuration in the topmost toolbar (either "release" or "debug", depending on what you want to do) and Platform (either Win32 or x64). Then hit "Build Solution" ({{Ctrl}}-{{Shift}}-{{key|B}} or {{F7}}). The compilation usually takes a minute or two, watch the output window at the bottom of Visual Studio. After completion you'll find the compiled binary in your darkmod/ folder. It would be either <tt>TheDarkMod.exe</tt> or <tt>TheDarkModx64.exe</tt>, depending on the configuration used.
To debug your custom built game code, you need to attach Visual Studio's debugger to the Doom3.exe process. There are two ways to accomplish that:
 
=== Debugging the Engine/Game ===
To debug your custom built code, you need to attach Visual Studio's debugger to the TheDarkMod.exe process. There are two ways to accomplish that:


The quick one:
The quick one:
# Go to Visual Studio and open the d3game solution
# Go to Visual Studio and open the TheDarkMod solution
# Start the game (either through tdmlauncher.exe or through DOOM3.exe plus command lines)
# Make sure the "DarkModTools" project is marked bold (as "Startup project")
# Compile and hit run (F5), Studio will start your TheDarkMod.exe and attach automatically
 
The manual way:
# Start your custom TheDarkMod.exe through Windows Explorer or shortcuts
# Once the game is up and running, Alt-Tab back to Visual Studio
# Once the game is up and running, Alt-Tab back to Visual Studio
# Go to menu "Debug" > "Attach to Process..." and select the DOOM3.exe process from the list in that dialog popping up.
# Go to menu "Debug" > "Attach to Process..." and select the TheDarkMod.exe process from the list in that dialog popping up.
# The debugger will now attach to Doom3 (along with all loaded DLLs) and you can now place breakpoints or intercept game crashes.
# The debugger will now attach to TDM and you can now place breakpoints or intercept game crashes.
 
The longer one (if you debug your game more than once or twice):
# Go to Visual Studio and open the d3game solution
# Right-click the Solution in the "Solution Explorer" and select "Add" > "Existing Project..."
# Browse to DOOM3.exe and select that, you'll see the DOOM3.exe project added to your solution explorer view.
# Right-Click the DOOM3.exe and choose "Select as startup project".
# Now you can start the debugger just by hitting F5 and Visual Studio will automatically attach to the Doom3 process.
# [Optional]: You can specify command line parameters to Doom3.exe by opening the Doom3.exe project properties and adding the desired parameters (e.g. com_allowConsole or fs_game_base/fs_game) to the "Command Arguments" item. This is necessary if you want to debug a specific TDM mission (although in this case, employing the "quick" method above is easier).
 
Note: you can still attach your debugger to any running DOOM3.exe process with the second method.


=== Troubleshooting ===
=== Troubleshooting ===
;My breakpoints don't work (they are hollow circles instead of full ones)
;My breakpoints don't work (they are hollow circles instead of full ones)
:Check if you copied the correct gamex86.dll to the folder where DOOM3.exe is located. The code you're viewing and the DLL must match exactly, otherwise Visual Studio won't load your symbols. Make sure that the configuration type (release or debug build) is matching as well. Also make sure that the code is properly compiled, e.g. hit "Build Solution" again - the output window should tell you that everything is up to date.
:Make sure you're attached to the correct TheDarkMod.exe binary. If you're attaching to an older version (e.g. from an outdated compilation process) or one you haven't built in Studio yourself, VC++ won't be able to load the symbols from the .pdb files. Make sure that the configuration type (release or debug build) is matching as well.


;I cannot inspect all the variables / The instruction pointer is skipping code
;I cannot inspect all the variables / The instruction pointer is skipping code
:You are probably running a release build, which comes with some optimisations. When debugging a release build, you'll notice that your instruction pointer (the yellow arrow) is sometimes skipping statements. These have most likely been optimised out of the binary during compilation/linking. You'll also have troubles when trying to inspect temporary variables or inlined functions. Use a debug build if this prevents you from figuring out things during debugging.
:You are probably running a release build, which comes with some optimisations. When debugging a release build, you'll notice that your instruction pointer (the yellow arrow) is sometimes skipping statements, which have most likely been optimised out of the binary during compilation/linking. You'll also have troubles when trying to inspect temporary variables or inlined functions. Use a debug build if this prevents you from figuring out things during debugging.
 
;Debugging works well, but the game plays too slowly
:You are running debug build, which is much slower than release. There are several ways to make life easier. First, you can get to the place where problem happens in release build and save your game there, then run debug build and load the game to do debugging. Second, you can try the "Debug with inlines" configuration, which is faster than full debug build, but is still very convenient to debug. If this is not fast enough for you, you can also debug the release build directly, but be aware that breakpoints and watches do not always work due to optimizations. Note that even release build is slower with debugger than without it because of debug heap. You can disable it by setting environment variable [https://ofekshilon.com/2014/09/20/accelerating-debug-runs-part-1-_no_debug_heap-2/ _NO_DEBUG_HEAP=1]. If you do it properly, then your release build should run with full speed.


== Linux ==
== Linux ==


For Linux you'll need gcc and scons, plus a few packages depending on your distribution. There is a README.linux file contained in the source package you downloaded, check it out for some details.
You need GCC 5 or newer to build TDM on Linux.
Anything older than GCC 4.7 surely won't work.
CMake >= 3.14 is required starting from TDM 2.08.
 
You first need to create a build directory. Easiest option is to create a subdirectory in the source code
root directory ("darkmod_src"):
  mkdir build && cd build
From your build directory, call CMake:
  cmake -DCMAKE_BUILD_TYPE="Release" ..
The '..' indicate the path to the source directory. If you chose a different location for your build directory
instead of the suggested subdirectory, you have to adjust this parameter accordingly.
 
If you need to force a specific GCC version because your Linux distro does not support GCC 5 but you've installed a newer GCC you may add flags to the cmake invoke:
  cmake -DCMAKE_BUILD_TYPE="Release" .. -DGAME_DIR=/home/user/darkmod -DCMAKE_C_COMPILER=gcc-11 -DCMAKE_CXX_COMPILER=g++-11
( Replace gcc-11 g++-11 with whichever GCC version you intend to use. )
<br><br>Finally, build TDM by calling
  make -j
where the '-j' parameter instructs make to build the project in parallel for faster compile times.
 
For newer Ubuntu and Linux Mint versions, it may be better to use:
  make -j$(nproc)
to ensure that the number of processes is limited to available cores.
 
 
If you wish to compile a debug executable, pass '-DCMAKE_BUILD_TYPE="Debug"' to the CMake call.
 
=== Ubuntu 16.04: native ===
 
The simplest approach is to do a native build, i.e. produce 64-bit binaries on 64-bit OS, or 32-bit binaries on 32-bit OS.
Starting from a clean Ubuntu installation, here is the list of packages you need to install:
 
  sudo apt-get install subversion              //svnversion: not found
  sudo apt-get install mesa-common-dev          //no such file: "Gl/gl.h", <=2.07
  sudo apt-get install libxxf86vm-dev          //no such file: "X11/extensions/xf86vmode.h"
  sudo apt-get install libopenal-dev            //no such file: "AL/al.h"
  sudo apt-get install libxext-dev              //no such file: "X11/extensions/Xext.h"
  sudo snap install cmake                      //we need a recent version of CMake, so install from snap instead of apt
 
Now run CMake as instructed above.
 
=== Ubuntu 16.04: 32-bit version on 64-bit OS ===
 
If you have 64-bit Linux, you can also build and run 32-bit TDM.
Note that this approach is slightly more complicated.
 
You have to install the following packages in addition to the ones listed above:
 
  sudo apt-get install g++-multilib            //no such file: 'sys/cdefs.h'
  sudo apt-get install libx11-dev:i386          //cannot find "-lX11"
  sudo apt-get install libxxf86vm-dev:i386      //cannot find "-lXxf86vm"
  sudo apt-get install libopenal-dev:i386      //cannot find "-lopenal"
  sudo apt-get install libxext-dev:i386        //cannot find "-lXext"
 
Then you need to call CMake with a toolchain file to target 32-bit:
 
  cmake -DCMAKE_BUILD_TYPE="Release" -DCMAKE_TOOLCHAIN_FILE="../sys/cmake/gcc_32bit.cmake" ..
 
The 64-bit and 32-bit versions can be built independently on a single 64-bit Linux, but you need to use different
build directories for each version.
 
=== Other distros ===
 
Even if you have a Linux distro different from Ubuntu, the instructions above will most likely help you.
If you still have problems with your build, please report to the forums.
 
E.g. for Kali Linux
 
  sudo apt-get update
  sudo apt-get install build-essential manpages-dev
  sudo apt-get install cmake
  sudo apt install libx11-dev     
  sudo apt-get install libxxf86vm-dev 
  sudo apt-get install libxext-dev


== I have a bugfix for the team ==
== I have a bugfix for the team ==
In case you figured out a problem in the TDM game code and you maybe even have a fix available, please drop by at our forums to tell the coding staff. Your fix might be incorporated in the main development branch.
In case you figured out a problem in the TDM game code and you maybe even have a fix available, please drop by at our forums to tell the coding staff. Your fix might be incorporated in the main development branch.


== Additional Info about the TDM Code ==
See also [[Reporting Problem]] article.
The Dark Mod codebase is quite large in the meantime and we have reorganised and refactored/replaced a great number of things. There are a few things which haven't changed too much (like the animation code), but most gameplay code has been altered to fit TDM's needs.
 
Additionally, the TDM code relies on a few static libraries which are not required in the vanilla D3 code, see below.
 
=== Boost Libraries ===
TDM's code is using [http://www.boost.org boost] libraries. Most boost libraries we're using are header-only (i.e. they work without special linking), with the exception of boost::filesystem. The latter requires a static library to be built for your OS and linked into the game code. For Windows the source package you downloaded already contains the correct binaries for VC++ 2005 and VC++ 2008 in both release and debug flavours. They can be found in the win32\lib folder.


=== ZipLoader ===
== Working in SVN ==
For the FM installation code, we made use of the minizip/zlib sources. For Windows and Linux the package already comes with the correct static .lib and .a files, so this should work out of the box. In case you need to compile these libraries from scratch (e.g. for a newer compiler or a different OS) there is a .vcproj file available in win32\ziploader (Windows) or a sconscript in <tt>sys/scons/Sconscript.ziploader</tt>. Use these to compile a new static library and make sure the library is referenced by the d3game project.


=== DevIL ===
If you work directly with [https://svn.thedarkmod.com/svn/darkmod_src/trunk SVN], make sure that [http://svnbook.red-bean.com/en/1.7/svn.ref.svnversion.re.html svnversion] command works properly in OS console:
Our lightgem code makes use of some image processing routines, which are contained in the open source DevIL package. The correct static library is already contained in the source package you downloaded, hence it should work right out of the box. In case you need to compile that library from scratch (applies to Windows only, Linux users will just need to download the correct package from their repositories), there are the DevIL sources in win32/devil, including a vcproj file, which you can use as basis.
* ''Windows'': install TortoiseSVN '''with "command line client tools" included'''.
* ''Linux'': install "subversion" package.
To check that it works, build TDM yourself, run it, and then open TDM console. You should see the correct SVN revision number in the lower-right corner.


== TDMLauncher ==
The Dark Mod comes with a custom launching app named tdmlauncher which makes mission installation/uninstallation easier for players. The source code for the tdmlauncher is contained in the source package linked above, head to the folder dmlauncher/ where you'll find a Visual Studio solution file as well as a sconscript.tdmlauncher. TDMLauncher is making use of boost::filesystem, that's why there is a separate lib/ folder in there as well (for Linux only). See above for info about the boost libraries.


[[Category:Tutorial]]
[[Category:Tutorial]]
[[Category:Coding]]
[[Category:Coding]]

Latest revision as of 01:53, 11 June 2024

Intended article audience: engine coders

This guide should provide you with enough information to compile The Dark Mod's game code from source. It applies to version 2.09 or later version of TDM. If you want to compile older version, see the history of this wiki page. Brief compilation instructions are also included in COMPILING.txt file in the source code package.


Get the sources

The sources are available through "snapshots", i.e. whenever the Dark Mod team is releasing a new update (e.g. TDM 2.09) the corresponding sources are released as well.

The executables built from these sources are compatible with the latest official release of TDM.


Also you can get the very latest source code directly from SVN:

Be warned though that these sources might be incompatible with your local darkmod installation. To minimize the problem, developer builds of the game are available in tdm_installer. Each developer build is named like devXXXXX-YYYY, where YYYY is the SVN revision number which this version was built from. If you checkout this exact revision from the source code SVN and build executable from it, it will surely be compatible with the corresponding dev. build of TDM.

Directory structure

Put the source code directory next to your game directory, such that the directory structure looks like this:

C:\Games\darkmod       <-- your darkmod installation
C:\Games\darkmod_src   <-- your source folder (containing the solution)

This directory structure is recommended for both Windows and Linux.

For windows users (this may also apply to linux users) - Install tortoise SVN and then right click on the \darkmod_src folder and select checkout. Then in the url field put 'https://svn.thedarkmod.com/publicsvn/darkmod_src/trunk/' then click ok (see screenshot below)

Windows

Since TDM 2.11 you'll need Visual Studio 2022 to compile the project (VS2017 was used before that). The free Community Edition works fine, it can be downloaded approximately here. Additionally, you have to:

  • Make sure "Visual C++ MFC for x86 and x64" is being installed by MSVC installer, along with "Desktop development with C++" workflow (see screenshot below).

You'll find a TheDarkMod.sln solution file in the source code folder, which you can double-click to open in Visual Studio. The solution is designed to put the compiled binaries into the ../darkmod folder nearby, that's why we recommend using the directory layout shown above. (You can change the output paths in the property sheets, in case you know how to do that).

Once the solution is opened, select the Configuration in the topmost toolbar (either "release" or "debug", depending on what you want to do) and Platform (either Win32 or x64). Then hit "Build Solution" (Ctrl-Shift-B or F7). The compilation usually takes a minute or two, watch the output window at the bottom of Visual Studio. After completion you'll find the compiled binary in your darkmod/ folder. It would be either TheDarkMod.exe or TheDarkModx64.exe, depending on the configuration used.

Debugging the Engine/Game

To debug your custom built code, you need to attach Visual Studio's debugger to the TheDarkMod.exe process. There are two ways to accomplish that:

The quick one:

  1. Go to Visual Studio and open the TheDarkMod solution
  2. Make sure the "DarkModTools" project is marked bold (as "Startup project")
  3. Compile and hit run (F5), Studio will start your TheDarkMod.exe and attach automatically

The manual way:

  1. Start your custom TheDarkMod.exe through Windows Explorer or shortcuts
  2. Once the game is up and running, Alt-Tab back to Visual Studio
  3. Go to menu "Debug" > "Attach to Process..." and select the TheDarkMod.exe process from the list in that dialog popping up.
  4. The debugger will now attach to TDM and you can now place breakpoints or intercept game crashes.

Troubleshooting

My breakpoints don't work (they are hollow circles instead of full ones)
Make sure you're attached to the correct TheDarkMod.exe binary. If you're attaching to an older version (e.g. from an outdated compilation process) or one you haven't built in Studio yourself, VC++ won't be able to load the symbols from the .pdb files. Make sure that the configuration type (release or debug build) is matching as well.
I cannot inspect all the variables / The instruction pointer is skipping code
You are probably running a release build, which comes with some optimisations. When debugging a release build, you'll notice that your instruction pointer (the yellow arrow) is sometimes skipping statements, which have most likely been optimised out of the binary during compilation/linking. You'll also have troubles when trying to inspect temporary variables or inlined functions. Use a debug build if this prevents you from figuring out things during debugging.
Debugging works well, but the game plays too slowly
You are running debug build, which is much slower than release. There are several ways to make life easier. First, you can get to the place where problem happens in release build and save your game there, then run debug build and load the game to do debugging. Second, you can try the "Debug with inlines" configuration, which is faster than full debug build, but is still very convenient to debug. If this is not fast enough for you, you can also debug the release build directly, but be aware that breakpoints and watches do not always work due to optimizations. Note that even release build is slower with debugger than without it because of debug heap. You can disable it by setting environment variable _NO_DEBUG_HEAP=1. If you do it properly, then your release build should run with full speed.

Linux

You need GCC 5 or newer to build TDM on Linux. Anything older than GCC 4.7 surely won't work. CMake >= 3.14 is required starting from TDM 2.08.

You first need to create a build directory. Easiest option is to create a subdirectory in the source code root directory ("darkmod_src"):

 mkdir build && cd build

From your build directory, call CMake:

 cmake -DCMAKE_BUILD_TYPE="Release" ..

The '..' indicate the path to the source directory. If you chose a different location for your build directory instead of the suggested subdirectory, you have to adjust this parameter accordingly.

If you need to force a specific GCC version because your Linux distro does not support GCC 5 but you've installed a newer GCC you may add flags to the cmake invoke:

 cmake -DCMAKE_BUILD_TYPE="Release" .. -DGAME_DIR=/home/user/darkmod -DCMAKE_C_COMPILER=gcc-11 -DCMAKE_CXX_COMPILER=g++-11

( Replace gcc-11 g++-11 with whichever GCC version you intend to use. )

Finally, build TDM by calling

 make -j

where the '-j' parameter instructs make to build the project in parallel for faster compile times.

For newer Ubuntu and Linux Mint versions, it may be better to use:

 make -j$(nproc)

to ensure that the number of processes is limited to available cores.


If you wish to compile a debug executable, pass '-DCMAKE_BUILD_TYPE="Debug"' to the CMake call.

Ubuntu 16.04: native

The simplest approach is to do a native build, i.e. produce 64-bit binaries on 64-bit OS, or 32-bit binaries on 32-bit OS. Starting from a clean Ubuntu installation, here is the list of packages you need to install:

 sudo apt-get install subversion               //svnversion: not found
 sudo apt-get install mesa-common-dev          //no such file: "Gl/gl.h", <=2.07
 sudo apt-get install libxxf86vm-dev           //no such file: "X11/extensions/xf86vmode.h"
 sudo apt-get install libopenal-dev            //no such file: "AL/al.h"
 sudo apt-get install libxext-dev              //no such file: "X11/extensions/Xext.h"
 sudo snap install cmake                       //we need a recent version of CMake, so install from snap instead of apt

Now run CMake as instructed above.

Ubuntu 16.04: 32-bit version on 64-bit OS

If you have 64-bit Linux, you can also build and run 32-bit TDM. Note that this approach is slightly more complicated.

You have to install the following packages in addition to the ones listed above:

 sudo apt-get install g++-multilib             //no such file: 'sys/cdefs.h'
 sudo apt-get install libx11-dev:i386          //cannot find "-lX11"
 sudo apt-get install libxxf86vm-dev:i386      //cannot find "-lXxf86vm"
 sudo apt-get install libopenal-dev:i386       //cannot find "-lopenal"
 sudo apt-get install libxext-dev:i386         //cannot find "-lXext"

Then you need to call CMake with a toolchain file to target 32-bit:

 cmake -DCMAKE_BUILD_TYPE="Release" -DCMAKE_TOOLCHAIN_FILE="../sys/cmake/gcc_32bit.cmake" ..

The 64-bit and 32-bit versions can be built independently on a single 64-bit Linux, but you need to use different build directories for each version.

Other distros

Even if you have a Linux distro different from Ubuntu, the instructions above will most likely help you. If you still have problems with your build, please report to the forums.

E.g. for Kali Linux

 sudo apt-get update
 sudo apt-get install build-essential manpages-dev 
 sudo apt-get install cmake
 sudo apt install libx11-dev      
 sudo apt-get install libxxf86vm-dev  
 sudo apt-get install libxext-dev

I have a bugfix for the team

In case you figured out a problem in the TDM game code and you maybe even have a fix available, please drop by at our forums to tell the coding staff. Your fix might be incorporated in the main development branch.

See also Reporting Problem article.

Working in SVN

If you work directly with SVN, make sure that svnversion command works properly in OS console:

  • Windows: install TortoiseSVN with "command line client tools" included.
  • Linux: install "subversion" package.

To check that it works, build TDM yourself, run it, and then open TDM console. You should see the correct SVN revision number in the lower-right corner.