TDM Release Mechanics: Difference between revisions

From The DarkMod Wiki
Jump to navigationJump to search
Building the Manifest: do manifests manually!
Updated packager build instructions (#5076 is done).
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This article describes the process of creating a TDM point release. It will be fleshed out over the next few days/weeks.
This article describes the process of creating TDM releases.


The team is using a version control system (SVN) for development, and the objective is to create a set of PK4 file from the repository which can be uploaded to the TDM mirrors. Let's go through this step by step:


= Development =  
= Preliminaries =
Just for the sake of completeness, this step includes the regular development between two TDM releases. This day-to-day development is always happening on the trunk, so all commits are ending up there. Stuff gets broken, stuff gets fixed, stuff gets evaluated, some might be ripped out again.


= Libraries and License =
Before even starting to build a package, make sure you have all the following tools ready.


A few weeks before release, it is a good idea to revise the set of third-party libraries used.
Starting from TDM 2.08, it should be easy to check it thanks to the new conventions on third-party stuff (see also [[Libraries and Dependencies]]).


It is necessary to update the <tt>LICENSE.txt</tt> file in SVN repositories if something is not up-to-date.
=== Packager ===
Note that you should commit the same updated file into '''both''' source code and assets repos.
 
The tdm_package tool inspects your working copy of assets SVN, and generates PK4 archives from it.
 
Prebuilt Windows executable can be downloaded from TDM server: [http://update.thedarkmod.com/zipsync/tdm_package.exe link]
 
 
The source code is located in source code SVN in <tt>tdm_package</tt> subdirectory.
It is built using CMake both on Windows and Linux, without any special details.
 
For instance:
* Run cmake-gui.
* Point "Where is the source code" to <tt>tdm_package</tt> directory.
* Point "Where to build the binaries" to <tt>tdm_package/output</tt> directory.
* Click "Configure" button, select the compiler & platform you need.
* Click "Finish".
* Click "Generate", and you should get build files in <tt>output</tt> subdirectory.
* Build:
** On Windows: click "Open Project", select "RelWithDebInfo" configuration and use "Build -> Build Solution" option in menu.
** On Linux: you should probably run "make" in the <tt>output</tt> subdirectory.
 
In the end, find packager executable <tt>tdm_package.???</tt> in the <tt>output</tt> subdirectory.
 
 
If the packager tool is modified, update the prebuilt executable on TDM server.
Contact admin if you don't have SSH access for it.
 
=== Zipsync ===
 
Zipsync command-line tool is used to integrate a freshly generated set of pk4 archives into the database of releases.
More details provided in article [[Tdm_installer_and_zipsync#Command-line_tool]].
 
Prebuilt Windows executable can be downloaded here: [http://update.thedarkmod.com/zipsync/zipsync.exe link]
 
 
The source code is located in source code SVN in <tt>tdm_installer/zipsync</tt> subdirectory.
Unlike all the other TDM projects, this one does not use artefacts from <tt>ThirdParty</tt> directory.
Hence, you need to install and use [https://conan.io/ conan] directly if you want to build it.
 
Build instructions:
* Get packages: <tt>conan install . -if build -s build_type=RelWithDebInfo --build=outdated</tt>
* Build: <tt>conan build . -bf build</tt>
The binaries should be ready in <tt>build/RelWithDebInfo</tt>.
In case of MSVC, the CMake-generated solution will be somewhere nearby.
 
 
If the zipsync tool is modified, update the prebuilt executable on TDM server.
Contact admin if you don't have SSH access for it.
 
 
=== Installer ===
 
Tdm_installer is GUI application, used to install/update the game on player's machine.
More details provided in article [[Tdm installer and zipsync#tdm_installer]].
 
Prebuilt executables can be obtained from Downloads section on the website.
 
 
The source code is located in source code SVN in <tt>tdm_installer</tt> subdirectory.
It can be built with CMake in pretty standard way.
For example:
* Create <tt>build</tt> directory, switch to it.
* Run <tt>cmake ..</tt> to generate project or makefile.
* Windows: Open generated solution, build RelWithDebInfo '''Win32''' configuration.
* Linux: Run <tt>make</tt> to finish build.
 
You only need to build 32-bit executable on Windows: it will work fine on 64-bit OS too.
For Linux, build both 64-bit and 32-bit executables separately.
 
In order to build 32-bit executable on 64-bit Linux, run cmake with additional parameters:
* <tt>cmake .. -DCMAKE_CXX_FLAGS=-m32 -DCMAKE_C_FLAGS=-m32</tt>
 
 
If tdm_installer is modified, update all prebuilt executables on TDM server.
 
For the self-update to work, each executable must be packaged into a special zip archive:
* <tt>zipsync hashzip tdm_installer.exe</tt>
* <tt>zipsync hashzip tdm_installer.linux64</tt>
* <tt>zipsync hashzip tdm_installer.linux32</tt>
 
The resulting zip archives should be uploaded onto the TDM server (contact admin for access).
Note that tdm_installer automatically updates itself to the executable on the server: if you put a bad build, make sure to fix it quickly!
 


= Branching off =
= Development Phase =
At some point the team decides that it might be time for a new release - a so-called branch date (some sort of semi-freeze) is suggested and agreed upon. To this date, team members are supposed to wrap up their pending changes and commit them to the trunk. About two to three weeks are long enough of a time span for people to get everything into SVN.


Once the deadline arrives, the '''release branch''' is created: [[Image:Create release branch.png|right|350px]]
Development phase takes the majority of time. During this time, day-to-day development happens on SVN trunk, so all commits end up there. Stuff gets broken, stuff gets fixed, stuff gets evaluated, some might be ripped out again. See also [[SVN]] article.
* Update your darkmod working copy.
* Update the darkmod_src repository too.
* Right-click the darkmod folder to create the branch
** Select SVN > Branch/Tag...
** A new dialog appears asking for a URL. Enter a URL like this: [https://server/svn/darkmod/branches/release1.xx https://<darkmod_server_here>/svn/darkmod/branches/release1.xx] where xx is replaced by whatever version the next release will be.
** You'll probably want to switch to the new branch once it is created, so make sure the check button at the bottom of the dialog is active.
** Hit OK to let the SVN server create the branch (the server will create a copy of the trunk, placing it at the URL you specified. It might be notable that the files are not physically copied, the branch requires a marginal amount of additional server space).
* The same should happen with the darkmod_src repository, the code needs to be frozen as well. Perform the same steps as above, using the URL scheme [https://server/svn/darkmod_src/branches/release1.xx https://<darkmod_server_here>/svn/darkmod_src/branches/release1.xx]


At this point there are two new branches, one for the darkmod repo, one for darkmod_src. Consider them a snapshot of the trunk at the point in time you created the branch.


'''Important:''' Commits to the trunk won't affect this newly created branch. The team members not involved in the release can just continue working as usual. Also note that if you as branch creator had the check button ("switch working copy") activated, your working copy has been ''switched'' to the release branch. ''Your commits will go to the branch'', keep that in mind. To learn how to ''merge'' stuff between the trunk and the release branch, see below.
=== Dev Builds ===


= Compile Game Binaries =
New development build/release is created roughly every 2-4 weeks (might depend on the amount of changes) from trunk. Even though these builds are unstable and untested, and may include features will be heavily changed of even removed by the time of the next major release, there are many reasons to provide them:
Before the PK4 files can be assembled into a release package, the code from darkmod_src needs to be re-compiled for the Windows and Linux platforms. This is to make sure that the release is built from the most recent (and hopefully most stable) source. If you're not a coder, you can either ask your fellow coders to switch to the branch and compile .exe and .dll files for you, or you can bite the bullet and follow the [[TDM Compilation Guide]] to compile them yourself. As Windows user, I recommend setting up an Ubuntu 16.04 64-Bit instance in VirtualBox. The downside of using virtualisation is that compilation takes 30-40 Minutes depending on your CPU power, but you at least don't need a dual-boot setup, let alone a second PC.  
* Beta-testers are able to bisect over development history to find when an issue started happening.
* External programmers can get assets compatible with recent source code trunk.
* Curious players can test new features and provide early feedback.


Make sure you compile a '''release build''' (debug builds are terribly slow), I also recommend re-compiling everything from scratch everytime you build a binary:
The rest of this chapter describes how to create such a dev build.
* Windows/VC++: Menu ''Build'' > ''Rebuild Solution''
Note: mostly the same steps are done when building beta releases and major releases.
* Linux: <tt>scons -c && scons BUILD="release" TARGET_ARCH="x86"</tt> (or <tt>"x64"</tt>)
Note that you have to compile both 32-bit and 64-bit build for every platform (Windows and Linux).


Once compiled, move freshly created thedarkmod.x86 and thedarkmod.x64 files from darkmod_src (where they are created by scons) to darkmod directory.
Then commit the following files ''to the branch'' (all of them must be treated as "modified" by SVN client):
* ExtLibs.dll
* ExtLibsx64.dll
* thedarkmod.x86
* thedarkmod.x64
* TheDarkMod.exe
* TheDarkModx64.exe


= Compile tdm_update =
=== Compile Game Binaries ===
tdm_update must be built separately for both Windows and Linux.


Make sure you compile a '''release build''', and recompile everything from scratch:
The assets SVN repo contains TDM executable files, which are copied into PK4 archives by the packager.
* Windows/VC++: Menu ''Build'' > ''Rebuild tdm_update_mfc''
They are rarely up-to-date, so the very first step is to build new executables from the source code and commit them to the assets repo.
* Linux: descend into the tdm_update directory and then do <tt>scons -c && scons BUILD="release"</tt>


Once compiled, copy the results into the darkmod folder:
Ensure you have clean and most recent revision of darkmod_src repo.
The detailed build instructions are provided in [[TDM Compilation Guide]], and the process of updating executables is explained in [[SVN#Executables Update]].
Just a few remainders:
* Better explicitly delete all executables before building. For major releases, better delete <tt>build</tt> directory and build everything from scratch.
* Always build Release configuration.
* Build binaries for all officially supported platforms: Linux and Windows. Deprecated 32-bit executables are '''not''' built, except for the major releases and the last beta.
* Don't forget to save debug symbols (.pdb and .debug files) somewhere when you commit executables, so that you don't accidentally overwrite them later!
* Note that shaders (<tt>glprogs</tt> directory) are also copied to assets repo when you build TDM, so don't forget to commit them too.


* Windows/VC++: bin/GuiUpdater/tdm_update.exe -> darkmod/tdm_update.exe
The easiest way to build executables for Windows and Linux at once is to use Windows machine with Linux virtual machine. Detailed setup instructions for virtual machine are provided in [[VirtualBox: Virtual machine with Linux for TDM|VirtualBox]] and [[VMWare: Virtual machine with Linux|VMWare]] articles.
* Linux: tdm_update.linux -> darkmod/tdm_update.linux


Commit these ''to the branch''.
Better choose fairly popular and '''old''' version of Linux, for instance a five-year-old Ubuntu LTS (still using 16.04 in 2021).
Due to how GLIBC symbols versioning works, sometimes TDM executable built on a new Linux won't start on an old one.


The debug symbols take quite a lot of space, but are indispensible for [[Analyze a Memory Dump|analyzing crashes]] on players' machines.
So release manager must keep them safe on a local disk, and probably share them with other programmers on some cloud storage.
For major releases, Windows debug symbols are even committed to the assets repo (see below).


'''TODO for 2.07''': now we also have 64-bit Linux version of tdm_update (built if you add <tt>TARGET_ARCH="x64"</tt> at the end of scons command line).
It should be renamed to something like tdm_update.linux64 and also committed to the branch.


= The Manifest =
=== Release Manifest ===
The "manifest" is a huge text files listing all the files that are nominated to go into the release. Not all files in SVN are supposed to be released (like test files, test maps, other broken or unfinished stuff), so we need a white list. The manifest text file is located in the SVN repository: '''devel/manifests/darkmod.txt'''


== Building the Manifest ==
The "manifest" is a huge list of all the files that are nominated to go into the release package. Not all files in SVN are supposed to be released (like test files, test maps, other broken or unfinished stuff), so a white list is needed. Aside from that, some rules are needed to decide how files are divided among pk4 archives.


Note: The TDM server can now build manifest and package archives automatically.
Here are the files related to manifest:
However, it is strongly recommended to do this step manually at least for the first beta package of every release. During this:
* <tt>devel/manifests/base.txt</tt>: an explicit list of files which are packaged regardless of any other conditions.
# Check which files were added and removed, by looking at the changes in manifest after update.
* <tt>devel/manifests/darkmod_maps.txt</tt>: sequence of include/exclude commands with regexes.
# Probably include or exclude some of the new files. You can have to do some sort of SVN diff against previous release.
* <tt>devel/manifests/darkmod.txt</tt>: full manifest file generated from the previous two files using tdm_package.
# Fix pk4 mappings if '''tdm_package''' gives you any warnings when it starts creating full package.
* <tt>devel/manifests/darkmod_pk4s.txt</tt>: set of regex-powered rules describing which files go into which pk4 archive.


==== Build Manifest ====
The manifest file <tt>devel/manifests/darkmod.txt</tt> is generated by the tdm_package application.
I run it with the following command:
* <tt>tdm_package --create-manifest --darkmoddir=G:\TheDarkMod\darkmod  >update_manifest.log</tt>
Here darkmoddir argument points to your working copy of assets SVN (it should be clean and up-to-date).


The manifest file is generated by the tdm_package application. (The tdm_package sources are in the darkmod_src repository, in a subfolder called tdm_update. There is a VC++ 2013 solution and a sconscript for Linux and Mac builds available. You don't need to compile it if you're in Windows, there is a pre-compiled version in the darkmod repo as well, in the devel/packager folder.)
After the packager finishes, review the packager log file and the changes in the file <tt>devel/manifests/darkmod.txt</tt>.
If everything looks good, commit the updated manifest to the assets repo.
If not, then adjust some files (e.g. the maps file) and try again. Commit your adjustments to SVN of course.


After the generation step the file '''devel/manifests/darkmod.txt''' will have been updated. You might want to review the modifications the script made to that file and commit it to the release branch if you're satisfied.
==== Include/Exclude From Manifest ====
The main place to specify what should be included into release is <tt>darkmod_maps.txt</tt> file.
As a rare exception, some files are specified in <tt>base.txt</tt>. But it is very small, and should probably be merged into the maps file in the future.
Here is how maps file works.


== What is included in the Manifest ==
In the first step, all files in specific folders are included:
The package application is using the file <tt>devel/manifests/darkmod_maps.txt</tt> to decide whether to include or exclude stuff. In a first internal step the script will include all files in specific folders:
  # Include all these files (but without parsing them like maps), each
  # Include all these files (but without parsing them like maps), each
  # statement will include files (from SVN) in that folder:
  # statement will include files (from SVN) in that folder:
Line 92: Line 164:
  INCLUDE video/
  INCLUDE video/
  INCLUDE xdata/
  INCLUDE xdata/
As visible in the above example, the sharp character # is used to denote comments. Put that at the beginning of a line to disable the statement.
As seen in the above example, the sharp character # is used to denote comments. Put that at the beginning of a line to disable the statement.


As next step the algorithm will exclude certain files matching the regular expressions in EXCLUDE statements like this:
As the next step, the algorithm will exclude certain files matching the regular expressions in EXCLUDE statements like this:
  EXCLUDE dds/darkmod/test
  EXCLUDE dds/darkmod/test
In 95% of the cases it's enough to just specify the path of the files you want to exclude (use forward slashes), but you can do more fancy stuff like this:
In 95% of the cases it's enough to just specify the path of the files you want to exclude (use forward slashes), but you can do more fancy stuff like this:
Line 103: Line 175:
In summary, all files that are INCLUDE'd as denoted above and afterwards manage to get through the hundreds of lines of EXCLUDE filters will end up in the manifest file.
In summary, all files that are INCLUDE'd as denoted above and afterwards manage to get through the hundreds of lines of EXCLUDE filters will end up in the manifest file.


= Distribute files into PK4s =
==== Distribute files into PK4s ====
Each of the roughly 15000 files in the manifest needs to be sorted into the correct PK4. This is done by defining a rules in the file <tt>devel/manifests/darkmod_pk4s.txt</tt>. Each of the lines there defines a PK4 and which files go into it:
Each of the thousands of files in the manifest needs to be sorted into the correct PK4.
This is done by defining a rules in the file <tt>devel/manifests/darkmod_pk4s.txt</tt>. Each of the lines there defines a PK4 and which files go into it:
  # Miscellaneous stuff (GL Progs, Script, Language Files, Rope Arrow)
  # Miscellaneous stuff (GL Progs, Script, Language Files, Rope Arrow)
  tdm_base01.pk4: ^glprogs, ^script, ^strings, sound\.wav, _emptyname\.wav, ^models/md5/environments, ^dds/models/md5/environments, ...
  tdm_base01.pk4: ^glprogs, ^script, ^strings, sound\.wav, _emptyname\.wav, ^models/md5/environments, ^dds/models/md5/environments, ...
Line 111: Line 184:
Leftmost is the PK4 filename, followed by a colon character. To the right of the colon a list comma-separated patterns is defined, whereas each pattern is interpreted as regular expression. As with the <tt>darkmod_maps.txt</tt> you don't need to know very much about regular expressions to define those rules, it's usually enough to write the folder names and use wildcards like <tt>.*.md5anim</tt>
Leftmost is the PK4 filename, followed by a colon character. To the right of the colon a list comma-separated patterns is defined, whereas each pattern is interpreted as regular expression. As with the <tt>darkmod_maps.txt</tt> you don't need to know very much about regular expressions to define those rules, it's usually enough to write the folder names and use wildcards like <tt>.*.md5anim</tt>


= Build the Package =
Note that a file will not get into the release package if no rule covers it in the darkmod_pk4s.txt, even if it is present in the full manifest.
The package is built right on the TDM server - there is a full darkmod repository checked out on the server's filesystem, and it will be processed through some scripts that are listening to your commands.
The packager prints a warning in such case when pk4 files are created.
 
 
=== Create Package ===
 
After the manifest is ready, we can finally use packager to pack working copy of assets SVN into a set of pk4 files.
Make sure your working copy is clean beforehand: unversioned files will be ignored, but local modifications will get into the package!
* <tt>tdm_package --create-package --darkmoddir=G:/TheDarkMod/darkmod --outputdir=G:/TheDarkMod/tmp_package    >create_package.log</tt>
Here darkmoddir specifies path to SVN working copy, and outputdir specifies path to an empty directory where pk4 files will be generated.
 
Be sure to look through the log file!
If some file is not covered by pk4 distribution rules, you will see the corresponding message there.
 
 
=== Add Release ===
 
All versions of TDM are stored in one zipsync database, which looks like a big tree of differential packages.
Read [[Tdm_installer_and_zipsync#Version_conventions]] carefully and decide 1) how to name the new version, and 2) which previously released version should serve as a parent for the new version.
 
First of all, you need the parent version somewhere on local disk (in clean state: without any additional files).
If you don't have it, you can download it via tdm_installer:
# Create new directory. You can optionally copy some tdm_XXX.pk4 files to it to reduce download time.
# Copy tdm_installer to the directory, start it.
# On the first page, check '''"Bitwise exact zips"''' advanced setting! Also check "Get custom version".
# On the second page, choose the parent version.
# Optional: After installation is complete, you can delete <tt>.zipsync</tt> directory and all files with extensions other than pk4 and zip.
'''Note:''' It is much easier to save all versions you create on the local disk, so that you can use them as parent versions later.
 
Rename the directories with new version and with parent version: the directory name must match the name of the version.
Let's say parent version is <tt>dev01234-8765</tt>, and new version is <tt>dev12345-9876</tt>.


Once triggered, the process goes roughly like this, I'll comment on these steps below:
Run analysis on the new version (remove <tt>-j0</tt> unless data is stored on SSD):
* <tt>zipsync analyze -r G:\tdmrel\dev12345-9876 -cn -j0 *.pk4 *.zip</tt>
Then create a differential package:
* <tt>zipsync diff -r G:\tdmrel\dev12345-9876 -s G:\tdmrel\dev01234-8765\manifest.iniz -o G:\tdmrel\dev12345-9876_from_dev01234-8765</tt>
Detailed explanation of how to use zipsync command line tool is provided in [[Tdm installer and zipsync]].


* The working copy is reverted, such that it is clean.
Now you need to connect to TDM server via SSH.
* The working copy is switched to the release branch of the version you're about to release (it must exist at this point).
For simplicity, I recommend storing exact copy of the whole zipsync database locally.
* The working copy is implicitly updated by the switch operation.
In order to push your modifications, you can log to TDM server using WinSCP and use Commands:Synchronize with the properly set Direction/Target.
* The manifest is generated.
* The full set of PK4 files is generated (it takes a while, the server CPU is not very powerful).
* The differential package is generated.
* The files are copied to a public folder reachable through HTTP such that it can be tested by the team. The team will test it by storing a manipulated tdm_mirrors.txt file next to their tdm_update application.


== Prepare the packaging process ==
Look at the directory structure in the database and put the new differential package appropriately, e.g. into <tt>dev/210/dev12345-9876_from_dev01234-8765</tt>.
A few steps need to be done before kicking off the process on the server.
Then open <tt>tdm_installer.ini</tt> in text editor, find appropriate place for the new version, and add it there, e.g.:
[Version dev12345-9876]
manifestUrl=${MIRROR}/dev/210/dev12345-9876_from_dev01234-8765/manifest.iniz
depends=dev01234-8765
folder=dev/2.10
When you are sure that you edited it properly, synchronize your changes to the server.
Immediately after that try to switch some of your TDM installations to the new version using tdm_installer (with "Bitwise exact zips" flag).
If something does not work for you, fix the problem as soon as possible.


* The release branch must be created, with a specific naming convention: https://darkmod_server_here/svn/darkmod/branches/releaseY.XX where Y is the major version and XX is the two-digit minor version, e.g. "release1.08".
=== Changelog and Announcement ===
* The inclusion statements in the <tt>darkmod_maps.txt</tt> file need to be up to date, make sure everything is committed to the branch.
* The <tt>tdm_version.txt</tt> file should be containing the correct version number, it's located in <tt>devel/release/tdm_version.txt</tt>. Commit that change to the release branch, needless to say.


== Kicking off the packaging process ==
The current tradition for changelog is:
There is a special admin page on the server which can be used to kick off the process. The URL has been [http://forums.thedarkmod.com/topic/13179-packaging-and-testing-107/ posted on the forums before], greebo or grayman know the URL. It's password protected, so you'll be asked to enter credentials. Once there, you can enter the two version numbers, the one you're building, and the previous one (the latest released one). E.g.: 1.08 and 1.07. It's sufficient to enter the currently released one, e.g. 1.07, into the second field and the website will automatically fill in the next version number in the first field - you can override it if really necessary.
* Changelog for all the dev builds after the last major release is in the second post of [https://forums.thedarkmod.com/index.php?/topic/20824-public-access-to-development-versions/ this forum thread].
* When beta phase is started, a separate thread is created for it, and changelog of all dev builds is moved to there.
* Changelog for beta releases is added to the beta testing thread. One post should contain changelog for all dev builds and beta releases in chronological order.
* In the message with changelog, post link to the changelog for previous versions (making a linked list of forum posts).


Once you start the process you'll be presented a text window with some progress information, and you can watch the messages emitted by the svn client and the tdm_package application running on the server. It takes a while, about half an hour, until the package is built.
Use SVN log over both repos to write changelog.
Try to write in language which users can understand, whenever possible.
Post as much links as you can: links to bugtracker as especially welcome, or links to feature discussion thread.
Some very minor things can be omitted in changelog, but don't miss anything which can break something or cause any type of negative consequences.
Most likely you will often read this changelog yourself =)


= Prepare the internal package test =
When changelog is ready, post short message that new version is available in the appropriate forum thread.
The team needs to have access to the package as created above for internal testing, so tell them to download a specially prepared tdm_mirrors.txt. You can find one attached to the first post here: http://forums.thedarkmod.com/topic/13179-packaging-and-testing-107/


The linked post above already contains a few instructions you can copy from. Basically, instruct the team to use the <tt>--keep-mirrors</tt> switch when starting the tdm_update application, and to copy the tdm_mirrors.txt file into their test folder before running it. If the <tt>--keep-mirrors</tt> option is omitted the updater will connect to the regular TDM update mirrors, which is not the point.


= The Test & Bugfix cycle =
=== Dev Build: Recap ===
Once the testers post their bug reports, the dev team needs to get moving to fix them. It's recommended that all fixes will be committed to the trunk (both for darkmod and darkmod_src repos), and the release manager should merge the changes into the release branch.


== Merge Fixes into the Release Branch ==
Here are all steps for creating a new dev build:
Let's assume there is a fix that got committed to darkmod SVN in order to fix an issue discovered in the test package. Proceed like this to merge it to the release branch.
# [[#Compile Game Binaries]]
* Make sure your local working copy is switched to the release branch.  
# [[#Build Manifest]]
* Now right-click the darkmod folder and choose SVN > Merge
# [[#Create Package]]
* Select ''Merge a range of revisions''
# [[#Add Release]] to zipsync database.
* As ''URL to merge from'' use the address of the trunk: [https://server/svn/darkmod/trunk https://<darkmod_server>/svn/darkmod/trunk]
# Write [[#Changelog and Announcement]].
* In the field revision range to merge you need to specify which exact change you intend to merge. It's easier if you click the "Show Log" button right next to the entry field to view the SVN Log of the trunk. You can either select a single commit, or you can select a range of commits. Perhaps double-check the changes of that checkin.
 
 
 
= Beta Phase =
 
At some moment the development phase is over, and beta phase starts.
The main goal of beta phase is to create a new major release, which will be used by most players for the next year or so.
Of course, it is preceded by lengthy public beta testing.
 
 
=== Agree on Schedule ===
 
The beta phase starts with branching off the release, and typically lasts for 1-2 months after that.
There is usually a rough plan about when to hold beta phase and when to release, but as time comes close, it's better to discuss it with team members again.
Team members are supposed to wrap up their pending changes and commit them to the trunk before beta phase starts.
About two weeks are long enough of a time span for people to get everything into SVN.
 
 
=== Libraries and License ===
 
A few weeks before branching, it is a good idea to revise the set of third-party libraries used.
Starting from TDM 2.08, it should be easy to check it thanks to the new conventions on third-party stuff (see also [[Libraries and Dependencies]]).
Some minor code pieces get embedded directly into the source, try to recall all of them.
 
It is necessary to update the <tt>LICENSE.txt</tt> file in SVN repositories if something is not up-to-date.
Note that you should commit the same updated file into '''both''' source code and assets repos.
 
 
=== Update Script Reference ===
 
Another thing which needs update prior to branching is the script reference.
It can be regenerated using the <tt>tdm_gen_script_event_doc</tt> command in game console:
* <tt>tdm_gen_script_event_doc tdm_events.script d3script</tt>
* <tt>tdm_gen_script_event_doc forwiki.mw mediawiki</tt>
The files will be created in the FM directory.
Copy mediawiki output to the wiki article [[TDM Script Reference]].
Copy script output to <tt>script/tdm_events.script</tt> file in the assets repo and commit.
 
 
=== Branching off ===
 
Once the deadline arrives, create the '''release branch''' in both repos: [[Image:Create release branch.png|right|350px]]
* Update your working copy of assets repo (darkmod).
* Update working copy of source code repo (darkmod_src) too.
* Right-click the darkmod_src folder to create the branch
** Select SVN > Branch/Tag...
** A new dialog appears asking for a URL. Enter a URL like this: [https://server/svn/darkmod_src/branches/releaseX.YY https://<darkmod_server_here>/svn/darkmod_src/branches/releaseX.YY] where X.YY is replaced by whatever version the next release will be.
** You'll probably want to switch to the new branch once it is created, so make sure the check button at the bottom of the dialog is active.
** Hit OK to let the SVN server create the branch.
* Do the same for darkmod repository, using the URL scheme [https://server/svn/darkmod/branches/releaseX.YY https://<darkmod_server_here>/svn/darkmod/branches/releaseX.YY]
 
At this point there is new "release" branch in each of the two repos.
The release branch and trunk live independently from each other: each working copy is attached to one of these branches, and commits done to one branch do not change the state of the other.
All the beta releases and the final major release will be packaged from the release branch, thus it is important to keep it stable and only commit there the changes which are necessary for the release.
Trunk will continue to evolve after the beta phase ends, so it is in principle possible to commit there some work which is not intended for release (although not desirable).
 
The release manager has to establish the workflow for developers, in order to avoid confusion with changes and branches.
The typical workflow is: everyone commits his changes to trunk and notifies release manager that changes are intended for release, release manager merges the necessary changes to release branch when needed.
If developer knows well how to deal with branches and commits merging in SVN, he can also merge his commits to release branch himself.
 
While trunk is technically independent from release branch, it is desirable to freeze or at least cool down all development not intended for the upcoming release.
The reason for that is that during beta phase it is necessary to merge commits between branches a lot.
If you merge 100% of commits from trunk to release, then beta phase can last forever or some major bug can slip into release at the last moment.
On the other hand, the more unmerged commits you have, the more likely merge conflicts become.
Refactoring, files renaming, and massive changes over the codebase are strongly discouraged.
 
 
=== Publish Beta Release ===
 
Beta release is created and published the same way as dev build.
The main difference is that dev builds are created on SVN trunk, and beta releases are created on the release branch.
So make sure your SVN working copy is switched to release branch both in the source code repo and in the assets repo.
Another difference between dev builds and beta releases is [[Tdm_installer_and_zipsync#Version_conventions|naming TDM versions in zipsync database]].
 
Once again, the steps are:
# [[#Compile Game Binaries]]
# [[#Build Manifest]]
# [[#Create Package]]
# [[#Add Release]] to zipsync database.
# Write [[#Changelog and Announcement]].
 
The announcement differs greatly when beta phase starts.
Usually release manager creates a dedicated forum thread in public subforum, like e.g. [https://forums.thedarkmod.com/index.php?/topic/20691-beta-testing-209/ Beta Testing 2.09].
This thread should contain changelog of beta releases. Changelog of dev builds should be moved here too.
I usually reserve three forum post immediately: the first one on how to install and what is beta about, the second one giving high-level overview of changes and new cvars/commands, and the third one with detailed changelog.
 
 
=== Testing Cycle ===
 
After beta release is published, testers update to it and post their bug reports.
Developers investigate them and fix whatever they can.
In typical workflow, the fixes are committed to the trunk (both for darkmod and darkmod_src repos), and the release manager merges them into the release branch.
 
One point to keep in mind is that SVN does not have real branches: a branch is merely a copy of the whole directory.
As the result, SVN does not have proper branch merging either: it simply takes the selected commits as patches and reapplies them onto the working copy (more like cherry-picking).
This is not a big problem in our context, since usually you don't want to merge everything: only a nontrivial subset of commits from trunk will be merged into the release branch.
In order to avoid confusion, keep an eye on svn:mergeinfo property: it should list the commits which were merged, so that you don't merge them again.
 
Here is how I merge commits from trunk to release branch using TortoiseSVN:
* Make sure your working copy is switched to the release branch and is clean from local modifications.
* Now right-click the SVN root directory and choose SVN > Merge.
* Select ''Merge a range of revisions''.
* As ''URL to merge from'' use the address of the trunk.
* In the field ''Revision range to merge'', click the ''Show Log'' button right next to the entry field. You will see log of commits in trunk, with the already merged commits greyed out. You can select one commit, or hold Ctrl to select an arbitrary subset of commits. You can press Ctrl+C when you finish selection, then paste the list of commits to a text file: later you can post it to developers so that they understand what was merged.
* Hit ''Next'', then ''Merge''.
* Hit ''Next'', then ''Merge''.
* Important: the changes will be incorporated into your ''working copy''. If you want these changes to actually be stored in the release branch, you need to perform the ''commit''.
* Now that the changes are incorporated into your working copy, test and review them briefly and commit to release branch.
 
After some number of fixes has been committed and merged to release branch, repeat again from section [[#Publish Beta Release]].
Publish new beta release, wait for feedback, fix, merge fixes to release branch, publish new release, etc.
 
= Major Release =
 
At some moment beta phase ends, and the next major release is created from the latest state of the release branch.


Special note for code changes: once these are committed to darkmod_src trunk, you need to merge them just like described above. Then you're going to recompile the game binaries (I recommend from scratch) and commit the tdm_game01.pk4 / tdm_game02.pk4 files to the darkmod release branch too.


== Update the Test Package ==
=== Finalization ===
Re-package the TDM package, as described above. Just make sure everything is committed to the release branch and kick off another build process.


== Test again ==
The common approach is to turn the latest beta release (called "release candidate") into the major release without even rebuilding/repackaging it.
Inform your testers about the update and which things got changed since the last test. Instruct them to update their test installation (don't forget to mention --keep-mirrors) and enter the next round of testing.
In such case you can take the full package of release candidate and proceed directly to [[#Add Release]] section: determine how major release is named (e.g. <tt>release210</tt>), its parent version (it is always the previous major release), create differential package, upload it to server and update tdm_installer.ini.
When editing tdm_installer.ini, move the <tt>default=1</tt> line from the previous major release to the new one.


= Release Time: Upload & Cleanup =
The next thing is saving Windows debug symbols. You must have saved them when you built executables which are in the final package. If you did not save them, then you cannot simply publish the existing package, and you have to start over from section [[#Compile Game Binaries]].
Once the day has come to release the next version, connect to the TDM server using a secure shell (SSH) client, like [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html PuTTY].
Compress debug symbols (TheDarkModx64.pdb and thedarkmod.x64.debug) with 7z, put them into devel/release/debugging/X.YY in assets repo. Commit them to both to trunk and to release branch (commit + merge).


Follow these steps, there are some things that need to be done afterwards, they are necessary for the ''next'' release to function properly:
Exact state of SVN repos should be saved for every major release.
So create a tag <tt>tags/X.YY</tt> in both SVN repos from the state of the release branch that you used to create release from.


* Rebuild binaries from scratch, place in darkmod folder, and commit the darkmod folder.
While 32-bit executables are deprecated, we still provide them as a separate download for the latest major version.
* 7-zip the Windows files: TheDarkMod.pdb, TheDarkModx64.pdb, ExtLibs.pdb, ExtLibsx64.pdb, and place them into the trunk darkmod/devel/release/debugging/2.0x folder. Add that folder to SVN and commit it.
Thus, you should take 32-bit executables <tt>TheDarkMod.exe</tt> and <tt>thedarkmod.x86</tt> and pack them into <tt>tdm_exe32.zip</tt>.
* Update the darkmod 2.0x branch tree and merge the just-placed debugging files to that branch. Commit the 2.0x branch.
If you did not build 32-bit executables previously, build them exactly from the same source code which was used to build 64-bit executables.
* Create the final package remotely as described above (through the website).
Put them at <tt>_aux/XYY/tdm_exe32.zip</tt> in the zipsync database (where X.YY is the TDM version).
* The above step will generate a new crc_info.txt and a tdm_version_info.txt file on the server.
* The packaging script will take care of checking these two files into SVN for "Release" packages.
* Check in the newly generated manifest, it should still be in the folder where the packaging script left it. Note: The packaging script will check this file in automatically for "Release" packages.
* Create a tag for the darkmod and darkmod_src repositories (/tags/2.0x) - copy it from the latest revision in the corresponding release branch.
** The packaging script will perform this task for "Release" packages.
* Copy package from /usr/local/www/update.thedarkmod.com/ to /usr/local/tdm_releases/2.0x/ for archiving. The next package process will need it there.
** The packaging script will perform this task for "Release" packages.
* Copy /usr/local/tdm_releases/1.05/index.html to /usr/local/tdm_releases/2.0x
** The packaging script will perform this task for "Release" packages.
* Move all differential packages back to /usr/local/www/update.thedarkmod.com/
** The packaging script will perform this task for "Release" packages.
* Copy the one differential update package updating to the new 2.0x version back to /usr/local/tdm_releases/2.0x. In the following line, (y = x-1).
** The packaging script will perform this task for "Release" packages.
* At this point, the /usr/local/tdm_releases/2.0x folder contains a valid TDM 2.0x release, the one that will be pushed to all the mirrors. Examine it for double safety.
* Examine the upload_to_all_mirrors.sh shell script, to see if it fits your needs.
* Start uploading to all mirrors.
** <tt>./upload_to_all_mirrors.sh 2.0x</tt>
** Update the rsyncd config on update.thedarkmod.com to point to the new release folder (taaaki will need to do this).
* Update tdm_mirrors.txt and tdm_version.xml on thedarkmod.com/update: ask taaaki to do that. The tdm_mirrors.txt file is probably fine, but the tdm_version.xml needs to be updated.


== Update the Wiki ==
=== Update Website ===
 
Be sure to update the following pages on the website to reflect the latest release version:
* [http://www.thedarkmod.com/downloads/ http://www.thedarkmod.com/downloads/]
* [http://www.thedarkmod.com/download-sourcecode/ http://www.thedarkmod.com/download-sourcecode/]
 
To generate the latest source code package for the pages above:
* Export the code from the release tag. E.g. svn export https://svn.thedarkmod.com/svn/darkmod_src/tags/x.yy
* Create a 7zip archive with the following naming convention: thedarkmod.x.yy.src.7z
** 7z a -r ../thedarkmod.x.yy.src.7z ./*
* Upload the source code archive to http://www.thedarkmod.com/sources (ask admin for assistance with this or use the ftp upload user)
 
Also update the link to 32-bit executables.
It should point to <tt>https://update.thedarkmod.com/zipsync/_aux/XYY/tdm_exe32.zip</tt> --- that's where it was put inside the zipsync database.
 
=== Update Wiki ===


The wiki contains one page for each release. You can find them all in the [[:Category:What's New]] category.  
The wiki contains one page for each release. You can find them all in the [[:Category:What's New]] category.  
Line 204: Line 421:
Also insert a link to the current roadmap on the bugtracker, you can find out the version_id by looking at [http://bugs.thedarkmod.com/roadmap_page.php the roadmap page].
Also insert a link to the current roadmap on the bugtracker, you can find out the version_id by looking at [http://bugs.thedarkmod.com/roadmap_page.php the roadmap page].


== Update the Website ==


Be sure to update the following pages on the website to reflect the latest release version:
=== Forum Announcement ===
* [http://www.thedarkmod.com/downloads/ http://www.thedarkmod.com/downloads/]
 
* [http://www.thedarkmod.com/download-the-mod/ http://www.thedarkmod.com/download-the-mod/]
Post thread about new version on the [https://forums.thedarkmod.com/index.php?/forum/10-news-amp-announcements/ News & Announcements] subforum.
* [http://www.thedarkmod.com/download-sourcecode/ http://www.thedarkmod.com/download-sourcecode/]
 
 
=== Back to Development Phase ===
 
After the dust of release settles, it's time to start the next release cycle and return to development phase.
* Make sure all changes from release branch are also present in trunk, since you will unlikely return to the release branch in future.
* Switch working copies of both repos back to trunk.
* Open <tt>framework/Licensee.h</tt> in the source code repo, and update <tt>TDM_VERSION_MAJOR</tt>, <tt>TDM_VERSION_MINOR</tt>, <tt>ENGINE_VERSION</tt> for the next major release. Commit the change.
* Go to [https://bugs.thedarkmod.com/manage_proj_page.php Manage section of bugtracker], choose The Dark Mod project. Click "Edit" for the just released version, set it to "Released" state. Add new version if not yet present.
 
 
=== Hotfix Release ===
 
Ideally, players use the major release for about a year, until the next major release comes out.
But in the real world, we often discover some issues which cannot/shouldn't be delayed until the next major release.
In such case, it is possible to create a special release, which is historically called "hotfix" release.
It is something like a "patch release" from semantic versioning: a new version which is almost the same as the major release, but with a few small fixes added.
 
The hotfix release is created from the previous release branch.
For instance, if 2.09 is the last published major release and 2.10 is currently in development, then hotfix release 2.09a is created from release2.09 branch (in both SVN repos).
Review Changelog or SVN commits since the last major release, and choose the changes which should be added in the hotfix. Discuss them with team members.
Proper beta phase lasts months, and you don't want to waste much time on hotfix release, so be very careful and critical about the changes you include:
* Included commits should be done some time ago, so that at least developers have already tested them.
* Included commits must not break savegame format, i.e. they must not change any calls to <tt>idSaveGame</tt>/<tt>idRestoreGame</tt> objects.
* Included commits should be small and have little risk of breaking something.
* Do not include commits which provide a new feature: new features are for new major release.
Usually, commits are included in the following cases:
* Fixing game crashes, race conditions, data corruption. Especially the ones which players bump into.
* Fixing wrong usage of OpenGL API. Or adding workarounds for bad OpenGL drivers. Same for OpenAL and other machine-dependent stuff.
* Fixing interaction with outside projects (FM database, DarkRadiant, tdm_installer, etc.) in case it has been broken.
 
Note that unlike typical convention, we guarantee full savegame compatibility between the hotfix release and its predecessor.
The game writes SVN revision into every savegame and checks that it is the same when it loads game, complaining if it does not match.
To silence this warning, go to the method <tt>RevisionTracker::GetSavegameRevision</tt> and make it return the hardcoded number: the revision reported by the original major release.
Commit this change to branch (but '''not''' to trunk): now all executables created from this branch will use the same revision number when writing and reading savegames.
Don't forget to test that savegames created in original release load properly in hotfix release and vice versa.
 
When everyone agrees on the set of included commits, merge them all to the release branch.


To generate the latest source code package for the page above:
Now do the typical steps for creating a new release:
* Check out the code from the release tag. E.g. svn checkout https://svn.thedarkmod.com/svn/darkmod_src/tags/x.yy
# [[#Compile Game Binaries]]
* Remove (or exclude) the .svn from the working copy folder
# [[#Build Manifest]]
* Create a 7zip archive with the following naming convention: thedarkmod.x.yy.src.7z
# [[#Create Package]]
** 7z a -r ../thedarkmod.x.yy.src.7z ./*
# [[#Add Release]] to zipsync database.
* Upload the source code archive to http://www.thedarkmod.com/sources (ask taaaki for assistance with this or use the ftp upload user)
# Write [[#Changelog and Announcement]].
Use release branch. Name the added version appropriately in zipsync database (e.g. <tt>release209a</tt>).
Write announcement on public forums with changelog describing the merged commits, and instructions how to get hotfix (like [https://forums.thedarkmod.com/index.php?/topic/20930-beta-testing-209a-aka-hotfix/ this one]).


== Hotfixes ==
After a very brief public beta-testing, move the <tt>default=1</tt> line from the original release to the hotfix release in <tt>tdm_installer.ini</tt>.
In the event of game-breaking bugs making it into the release, it is possible to rebuild the release with the same version number. This should only be done when absolutely necessary. The process is essentially as follows:
Now everyone who runs tdm_installer without explicitly specifying version will get the hotfix release.
* Merge the bug fixes into the current release branch (include the recompiled binaries).
* Re-run the "Testing" package build and test the updated release from http://update.thedarkmod.com/testing/
* If all is in order, proceed to generate the "Release" package as above and deploy the updated release to the mirrors.
** Note: it may be necessary to tweak the create_package.sh file to exclude the creation of the release tag - be sure to revert back to the original after the hotfix process has been completed (TODO: an additional option should be added to the release management screen to handle with automatically).
** Manually create a new release tag for darkmod/ and darkmod_src/ (E.g. darkmod/tags/2.05hotfix and darkmod_src/tags/2.05hotfix repectively).
** Update the source code package and re-upload to the website if necessary.


= Glossary =
Now do the ordinary finalization steps:
* Release Fileset: the PK4 files forming the actual release.  
# Publish 32-bit executables.
* SVN: an acronym for Subversion, a version control system
# Commit 7zipped Windows PDB files to <tt>devel/release/debugging</tt>, replacing the files from original release.
* Commit: The process of a team member uploading files to the SVN repository - a commit always creates a new SVN revision.
# Create SVN tags in both repos.
* Mirrors: The servers where the PK4 files are stored for download by the updater.
# Update website: replace link to 32-bit executables and source code archive.
* updater: The TDM Updater executable, used for downloading and updating TDM.
# Write announcements, so that people know they should update.
* binary: Source code is compiled into so-called binaries. This can be an executable (.exe) file or a module (.dll/.so).
* darkmod_src: The code repository holding all the TDM source files, including the tdm_update application.
* branch: A tree in the SVN repository. Several branches can co-exist side-by-side without interfering.
* trunk: The main branch where all the ongoing development is happening.


{{todo}}
Of course, you can provide more than one hotfix release for one major release: 2.09a, 2.09b, 2.09c, etc.
Also note that the original release is not lost: players can even install it if they want.

Revision as of 13:59, 16 December 2023

This article describes the process of creating TDM releases.


Preliminaries

Before even starting to build a package, make sure you have all the following tools ready.


Packager

The tdm_package tool inspects your working copy of assets SVN, and generates PK4 archives from it.

Prebuilt Windows executable can be downloaded from TDM server: link


The source code is located in source code SVN in tdm_package subdirectory. It is built using CMake both on Windows and Linux, without any special details.

For instance:

  • Run cmake-gui.
  • Point "Where is the source code" to tdm_package directory.
  • Point "Where to build the binaries" to tdm_package/output directory.
  • Click "Configure" button, select the compiler & platform you need.
  • Click "Finish".
  • Click "Generate", and you should get build files in output subdirectory.
  • Build:
    • On Windows: click "Open Project", select "RelWithDebInfo" configuration and use "Build -> Build Solution" option in menu.
    • On Linux: you should probably run "make" in the output subdirectory.

In the end, find packager executable tdm_package.??? in the output subdirectory.


If the packager tool is modified, update the prebuilt executable on TDM server. Contact admin if you don't have SSH access for it.

Zipsync

Zipsync command-line tool is used to integrate a freshly generated set of pk4 archives into the database of releases. More details provided in article Tdm_installer_and_zipsync#Command-line_tool.

Prebuilt Windows executable can be downloaded here: link


The source code is located in source code SVN in tdm_installer/zipsync subdirectory. Unlike all the other TDM projects, this one does not use artefacts from ThirdParty directory. Hence, you need to install and use conan directly if you want to build it.

Build instructions:

  • Get packages: conan install . -if build -s build_type=RelWithDebInfo --build=outdated
  • Build: conan build . -bf build

The binaries should be ready in build/RelWithDebInfo. In case of MSVC, the CMake-generated solution will be somewhere nearby.


If the zipsync tool is modified, update the prebuilt executable on TDM server. Contact admin if you don't have SSH access for it.


Installer

Tdm_installer is GUI application, used to install/update the game on player's machine. More details provided in article Tdm installer and zipsync#tdm_installer.

Prebuilt executables can be obtained from Downloads section on the website.


The source code is located in source code SVN in tdm_installer subdirectory. It can be built with CMake in pretty standard way. For example:

  • Create build directory, switch to it.
  • Run cmake .. to generate project or makefile.
  • Windows: Open generated solution, build RelWithDebInfo Win32 configuration.
  • Linux: Run make to finish build.

You only need to build 32-bit executable on Windows: it will work fine on 64-bit OS too. For Linux, build both 64-bit and 32-bit executables separately.

In order to build 32-bit executable on 64-bit Linux, run cmake with additional parameters:

  • cmake .. -DCMAKE_CXX_FLAGS=-m32 -DCMAKE_C_FLAGS=-m32


If tdm_installer is modified, update all prebuilt executables on TDM server.

For the self-update to work, each executable must be packaged into a special zip archive:

  • zipsync hashzip tdm_installer.exe
  • zipsync hashzip tdm_installer.linux64
  • zipsync hashzip tdm_installer.linux32

The resulting zip archives should be uploaded onto the TDM server (contact admin for access). Note that tdm_installer automatically updates itself to the executable on the server: if you put a bad build, make sure to fix it quickly!


Development Phase

Development phase takes the majority of time. During this time, day-to-day development happens on SVN trunk, so all commits end up there. Stuff gets broken, stuff gets fixed, stuff gets evaluated, some might be ripped out again. See also SVN article.


Dev Builds

New development build/release is created roughly every 2-4 weeks (might depend on the amount of changes) from trunk. Even though these builds are unstable and untested, and may include features will be heavily changed of even removed by the time of the next major release, there are many reasons to provide them:

  • Beta-testers are able to bisect over development history to find when an issue started happening.
  • External programmers can get assets compatible with recent source code trunk.
  • Curious players can test new features and provide early feedback.

The rest of this chapter describes how to create such a dev build. Note: mostly the same steps are done when building beta releases and major releases.


Compile Game Binaries

The assets SVN repo contains TDM executable files, which are copied into PK4 archives by the packager. They are rarely up-to-date, so the very first step is to build new executables from the source code and commit them to the assets repo.

Ensure you have clean and most recent revision of darkmod_src repo. The detailed build instructions are provided in TDM Compilation Guide, and the process of updating executables is explained in SVN#Executables Update. Just a few remainders:

  • Better explicitly delete all executables before building. For major releases, better delete build directory and build everything from scratch.
  • Always build Release configuration.
  • Build binaries for all officially supported platforms: Linux and Windows. Deprecated 32-bit executables are not built, except for the major releases and the last beta.
  • Don't forget to save debug symbols (.pdb and .debug files) somewhere when you commit executables, so that you don't accidentally overwrite them later!
  • Note that shaders (glprogs directory) are also copied to assets repo when you build TDM, so don't forget to commit them too.

The easiest way to build executables for Windows and Linux at once is to use Windows machine with Linux virtual machine. Detailed setup instructions for virtual machine are provided in VirtualBox and VMWare articles.

Better choose fairly popular and old version of Linux, for instance a five-year-old Ubuntu LTS (still using 16.04 in 2021). Due to how GLIBC symbols versioning works, sometimes TDM executable built on a new Linux won't start on an old one.

The debug symbols take quite a lot of space, but are indispensible for analyzing crashes on players' machines. So release manager must keep them safe on a local disk, and probably share them with other programmers on some cloud storage. For major releases, Windows debug symbols are even committed to the assets repo (see below).


Release Manifest

The "manifest" is a huge list of all the files that are nominated to go into the release package. Not all files in SVN are supposed to be released (like test files, test maps, other broken or unfinished stuff), so a white list is needed. Aside from that, some rules are needed to decide how files are divided among pk4 archives.

Here are the files related to manifest:

  • devel/manifests/base.txt: an explicit list of files which are packaged regardless of any other conditions.
  • devel/manifests/darkmod_maps.txt: sequence of include/exclude commands with regexes.
  • devel/manifests/darkmod.txt: full manifest file generated from the previous two files using tdm_package.
  • devel/manifests/darkmod_pk4s.txt: set of regex-powered rules describing which files go into which pk4 archive.

Build Manifest

The manifest file devel/manifests/darkmod.txt is generated by the tdm_package application. I run it with the following command:

  • tdm_package --create-manifest --darkmoddir=G:\TheDarkMod\darkmod >update_manifest.log

Here darkmoddir argument points to your working copy of assets SVN (it should be clean and up-to-date).

After the packager finishes, review the packager log file and the changes in the file devel/manifests/darkmod.txt. If everything looks good, commit the updated manifest to the assets repo. If not, then adjust some files (e.g. the maps file) and try again. Commit your adjustments to SVN of course.

Include/Exclude From Manifest

The main place to specify what should be included into release is darkmod_maps.txt file. As a rare exception, some files are specified in base.txt. But it is very small, and should probably be merged into the maps file in the future. Here is how maps file works.

In the first step, all files in specific folders are included:

# Include all these files (but without parsing them like maps), each
# statement will include files (from SVN) in that folder:
INCLUDE def/
INCLUDE dds/
...
INCLUDE video/
INCLUDE xdata/

As seen in the above example, the sharp character # is used to denote comments. Put that at the beginning of a line to disable the statement.

As the next step, the algorithm will exclude certain files matching the regular expressions in EXCLUDE statements like this:

EXCLUDE dds/darkmod/test

In 95% of the cases it's enough to just specify the path of the files you want to exclude (use forward slashes), but you can do more fancy stuff like this:

EXCLUDE models/md5/chars/undead/revenant/.*.md5anim
EXCLUDE ^(dds/)?models/md5/chars/undead/revenant

The first line will exclude all MD5ANIM files in the revenant folder. The second line will exclude all files in models/md5/chars/undead/revenant and the ones in dds/models/md5/chars/undead/revenant (the dds/ part is marked to be optional).

In summary, all files that are INCLUDE'd as denoted above and afterwards manage to get through the hundreds of lines of EXCLUDE filters will end up in the manifest file.

Distribute files into PK4s

Each of the thousands of files in the manifest needs to be sorted into the correct PK4. This is done by defining a rules in the file devel/manifests/darkmod_pk4s.txt. Each of the lines there defines a PK4 and which files go into it:

# Miscellaneous stuff (GL Progs, Script, Language Files, Rope Arrow)
tdm_base01.pk4: ^glprogs, ^script, ^strings, sound\.wav, _emptyname\.wav, ^models/md5/environments, ^dds/models/md5/environments, ...

Again, lines starting with the sharp # character denote comments.

Leftmost is the PK4 filename, followed by a colon character. To the right of the colon a list comma-separated patterns is defined, whereas each pattern is interpreted as regular expression. As with the darkmod_maps.txt you don't need to know very much about regular expressions to define those rules, it's usually enough to write the folder names and use wildcards like .*.md5anim

Note that a file will not get into the release package if no rule covers it in the darkmod_pk4s.txt, even if it is present in the full manifest. The packager prints a warning in such case when pk4 files are created.


Create Package

After the manifest is ready, we can finally use packager to pack working copy of assets SVN into a set of pk4 files. Make sure your working copy is clean beforehand: unversioned files will be ignored, but local modifications will get into the package!

  • tdm_package --create-package --darkmoddir=G:/TheDarkMod/darkmod --outputdir=G:/TheDarkMod/tmp_package >create_package.log

Here darkmoddir specifies path to SVN working copy, and outputdir specifies path to an empty directory where pk4 files will be generated.

Be sure to look through the log file! If some file is not covered by pk4 distribution rules, you will see the corresponding message there.


Add Release

All versions of TDM are stored in one zipsync database, which looks like a big tree of differential packages. Read Tdm_installer_and_zipsync#Version_conventions carefully and decide 1) how to name the new version, and 2) which previously released version should serve as a parent for the new version.

First of all, you need the parent version somewhere on local disk (in clean state: without any additional files). If you don't have it, you can download it via tdm_installer:

  1. Create new directory. You can optionally copy some tdm_XXX.pk4 files to it to reduce download time.
  2. Copy tdm_installer to the directory, start it.
  3. On the first page, check "Bitwise exact zips" advanced setting! Also check "Get custom version".
  4. On the second page, choose the parent version.
  5. Optional: After installation is complete, you can delete .zipsync directory and all files with extensions other than pk4 and zip.

Note: It is much easier to save all versions you create on the local disk, so that you can use them as parent versions later.

Rename the directories with new version and with parent version: the directory name must match the name of the version. Let's say parent version is dev01234-8765, and new version is dev12345-9876.

Run analysis on the new version (remove -j0 unless data is stored on SSD):

  • zipsync analyze -r G:\tdmrel\dev12345-9876 -cn -j0 *.pk4 *.zip

Then create a differential package:

  • zipsync diff -r G:\tdmrel\dev12345-9876 -s G:\tdmrel\dev01234-8765\manifest.iniz -o G:\tdmrel\dev12345-9876_from_dev01234-8765

Detailed explanation of how to use zipsync command line tool is provided in Tdm installer and zipsync.

Now you need to connect to TDM server via SSH. For simplicity, I recommend storing exact copy of the whole zipsync database locally. In order to push your modifications, you can log to TDM server using WinSCP and use Commands:Synchronize with the properly set Direction/Target.

Look at the directory structure in the database and put the new differential package appropriately, e.g. into dev/210/dev12345-9876_from_dev01234-8765. Then open tdm_installer.ini in text editor, find appropriate place for the new version, and add it there, e.g.:

[Version dev12345-9876]
manifestUrl=${MIRROR}/dev/210/dev12345-9876_from_dev01234-8765/manifest.iniz
depends=dev01234-8765
folder=dev/2.10

When you are sure that you edited it properly, synchronize your changes to the server. Immediately after that try to switch some of your TDM installations to the new version using tdm_installer (with "Bitwise exact zips" flag). If something does not work for you, fix the problem as soon as possible.

Changelog and Announcement

The current tradition for changelog is:

  • Changelog for all the dev builds after the last major release is in the second post of this forum thread.
  • When beta phase is started, a separate thread is created for it, and changelog of all dev builds is moved to there.
  • Changelog for beta releases is added to the beta testing thread. One post should contain changelog for all dev builds and beta releases in chronological order.
  • In the message with changelog, post link to the changelog for previous versions (making a linked list of forum posts).

Use SVN log over both repos to write changelog. Try to write in language which users can understand, whenever possible. Post as much links as you can: links to bugtracker as especially welcome, or links to feature discussion thread. Some very minor things can be omitted in changelog, but don't miss anything which can break something or cause any type of negative consequences. Most likely you will often read this changelog yourself =)

When changelog is ready, post short message that new version is available in the appropriate forum thread.


Dev Build: Recap

Here are all steps for creating a new dev build:

  1. #Compile Game Binaries
  2. #Build Manifest
  3. #Create Package
  4. #Add Release to zipsync database.
  5. Write #Changelog and Announcement.


Beta Phase

At some moment the development phase is over, and beta phase starts. The main goal of beta phase is to create a new major release, which will be used by most players for the next year or so. Of course, it is preceded by lengthy public beta testing.


Agree on Schedule

The beta phase starts with branching off the release, and typically lasts for 1-2 months after that. There is usually a rough plan about when to hold beta phase and when to release, but as time comes close, it's better to discuss it with team members again. Team members are supposed to wrap up their pending changes and commit them to the trunk before beta phase starts. About two weeks are long enough of a time span for people to get everything into SVN.


Libraries and License

A few weeks before branching, it is a good idea to revise the set of third-party libraries used. Starting from TDM 2.08, it should be easy to check it thanks to the new conventions on third-party stuff (see also Libraries and Dependencies). Some minor code pieces get embedded directly into the source, try to recall all of them.

It is necessary to update the LICENSE.txt file in SVN repositories if something is not up-to-date. Note that you should commit the same updated file into both source code and assets repos.


Update Script Reference

Another thing which needs update prior to branching is the script reference. It can be regenerated using the tdm_gen_script_event_doc command in game console:

  • tdm_gen_script_event_doc tdm_events.script d3script
  • tdm_gen_script_event_doc forwiki.mw mediawiki

The files will be created in the FM directory. Copy mediawiki output to the wiki article TDM Script Reference. Copy script output to script/tdm_events.script file in the assets repo and commit.


Branching off

Once the deadline arrives, create the release branch in both repos:

  • Update your working copy of assets repo (darkmod).
  • Update working copy of source code repo (darkmod_src) too.
  • Right-click the darkmod_src folder to create the branch
    • Select SVN > Branch/Tag...
    • A new dialog appears asking for a URL. Enter a URL like this: https://<darkmod_server_here>/svn/darkmod_src/branches/releaseX.YY where X.YY is replaced by whatever version the next release will be.
    • You'll probably want to switch to the new branch once it is created, so make sure the check button at the bottom of the dialog is active.
    • Hit OK to let the SVN server create the branch.
  • Do the same for darkmod repository, using the URL scheme https://<darkmod_server_here>/svn/darkmod/branches/releaseX.YY

At this point there is new "release" branch in each of the two repos. The release branch and trunk live independently from each other: each working copy is attached to one of these branches, and commits done to one branch do not change the state of the other. All the beta releases and the final major release will be packaged from the release branch, thus it is important to keep it stable and only commit there the changes which are necessary for the release. Trunk will continue to evolve after the beta phase ends, so it is in principle possible to commit there some work which is not intended for release (although not desirable).

The release manager has to establish the workflow for developers, in order to avoid confusion with changes and branches. The typical workflow is: everyone commits his changes to trunk and notifies release manager that changes are intended for release, release manager merges the necessary changes to release branch when needed. If developer knows well how to deal with branches and commits merging in SVN, he can also merge his commits to release branch himself.

While trunk is technically independent from release branch, it is desirable to freeze or at least cool down all development not intended for the upcoming release. The reason for that is that during beta phase it is necessary to merge commits between branches a lot. If you merge 100% of commits from trunk to release, then beta phase can last forever or some major bug can slip into release at the last moment. On the other hand, the more unmerged commits you have, the more likely merge conflicts become. Refactoring, files renaming, and massive changes over the codebase are strongly discouraged.


Publish Beta Release

Beta release is created and published the same way as dev build. The main difference is that dev builds are created on SVN trunk, and beta releases are created on the release branch. So make sure your SVN working copy is switched to release branch both in the source code repo and in the assets repo. Another difference between dev builds and beta releases is naming TDM versions in zipsync database.

Once again, the steps are:

  1. #Compile Game Binaries
  2. #Build Manifest
  3. #Create Package
  4. #Add Release to zipsync database.
  5. Write #Changelog and Announcement.

The announcement differs greatly when beta phase starts. Usually release manager creates a dedicated forum thread in public subforum, like e.g. Beta Testing 2.09. This thread should contain changelog of beta releases. Changelog of dev builds should be moved here too. I usually reserve three forum post immediately: the first one on how to install and what is beta about, the second one giving high-level overview of changes and new cvars/commands, and the third one with detailed changelog.


Testing Cycle

After beta release is published, testers update to it and post their bug reports. Developers investigate them and fix whatever they can. In typical workflow, the fixes are committed to the trunk (both for darkmod and darkmod_src repos), and the release manager merges them into the release branch.

One point to keep in mind is that SVN does not have real branches: a branch is merely a copy of the whole directory. As the result, SVN does not have proper branch merging either: it simply takes the selected commits as patches and reapplies them onto the working copy (more like cherry-picking). This is not a big problem in our context, since usually you don't want to merge everything: only a nontrivial subset of commits from trunk will be merged into the release branch. In order to avoid confusion, keep an eye on svn:mergeinfo property: it should list the commits which were merged, so that you don't merge them again.

Here is how I merge commits from trunk to release branch using TortoiseSVN:

  • Make sure your working copy is switched to the release branch and is clean from local modifications.
  • Now right-click the SVN root directory and choose SVN > Merge.
  • Select Merge a range of revisions.
  • As URL to merge from use the address of the trunk.
  • In the field Revision range to merge, click the Show Log button right next to the entry field. You will see log of commits in trunk, with the already merged commits greyed out. You can select one commit, or hold Ctrl to select an arbitrary subset of commits. You can press Ctrl+C when you finish selection, then paste the list of commits to a text file: later you can post it to developers so that they understand what was merged.
  • Hit Next, then Merge.
  • Now that the changes are incorporated into your working copy, test and review them briefly and commit to release branch.

After some number of fixes has been committed and merged to release branch, repeat again from section #Publish Beta Release. Publish new beta release, wait for feedback, fix, merge fixes to release branch, publish new release, etc.

Major Release

At some moment beta phase ends, and the next major release is created from the latest state of the release branch.


Finalization

The common approach is to turn the latest beta release (called "release candidate") into the major release without even rebuilding/repackaging it. In such case you can take the full package of release candidate and proceed directly to #Add Release section: determine how major release is named (e.g. release210), its parent version (it is always the previous major release), create differential package, upload it to server and update tdm_installer.ini. When editing tdm_installer.ini, move the default=1 line from the previous major release to the new one.

The next thing is saving Windows debug symbols. You must have saved them when you built executables which are in the final package. If you did not save them, then you cannot simply publish the existing package, and you have to start over from section #Compile Game Binaries. Compress debug symbols (TheDarkModx64.pdb and thedarkmod.x64.debug) with 7z, put them into devel/release/debugging/X.YY in assets repo. Commit them to both to trunk and to release branch (commit + merge).

Exact state of SVN repos should be saved for every major release. So create a tag tags/X.YY in both SVN repos from the state of the release branch that you used to create release from.

While 32-bit executables are deprecated, we still provide them as a separate download for the latest major version. Thus, you should take 32-bit executables TheDarkMod.exe and thedarkmod.x86 and pack them into tdm_exe32.zip. If you did not build 32-bit executables previously, build them exactly from the same source code which was used to build 64-bit executables. Put them at _aux/XYY/tdm_exe32.zip in the zipsync database (where X.YY is the TDM version).

Update Website

Be sure to update the following pages on the website to reflect the latest release version:

To generate the latest source code package for the pages above:

Also update the link to 32-bit executables. It should point to https://update.thedarkmod.com/zipsync/_aux/XYY/tdm_exe32.zip --- that's where it was put inside the zipsync database.

Update Wiki

The wiki contains one page for each release. You can find them all in the Category:What's New category.

First update the current release page:

  • Change the "roadmap" link on the current release to a "changelog" link by swapping "roadmap_page.php?version_id=XX" with "changelog_page.php?version=XX".
  • Add the {{released|2.01|2014-01-30}} Template on top of the current release page with the correct date.

Then add a new page for the next release. To do this, add a link to it in this template: Template:Whatsnew by editing it. After saving the template, a link to the new release version page appears in the list and in the category. Edit this page and insert some text into it (you can copy it from an older release page).

Also insert a link to the current roadmap on the bugtracker, you can find out the version_id by looking at the roadmap page.


Forum Announcement

Post thread about new version on the News & Announcements subforum.


Back to Development Phase

After the dust of release settles, it's time to start the next release cycle and return to development phase.

  • Make sure all changes from release branch are also present in trunk, since you will unlikely return to the release branch in future.
  • Switch working copies of both repos back to trunk.
  • Open framework/Licensee.h in the source code repo, and update TDM_VERSION_MAJOR, TDM_VERSION_MINOR, ENGINE_VERSION for the next major release. Commit the change.
  • Go to Manage section of bugtracker, choose The Dark Mod project. Click "Edit" for the just released version, set it to "Released" state. Add new version if not yet present.


Hotfix Release

Ideally, players use the major release for about a year, until the next major release comes out. But in the real world, we often discover some issues which cannot/shouldn't be delayed until the next major release. In such case, it is possible to create a special release, which is historically called "hotfix" release. It is something like a "patch release" from semantic versioning: a new version which is almost the same as the major release, but with a few small fixes added.

The hotfix release is created from the previous release branch. For instance, if 2.09 is the last published major release and 2.10 is currently in development, then hotfix release 2.09a is created from release2.09 branch (in both SVN repos). Review Changelog or SVN commits since the last major release, and choose the changes which should be added in the hotfix. Discuss them with team members. Proper beta phase lasts months, and you don't want to waste much time on hotfix release, so be very careful and critical about the changes you include:

  • Included commits should be done some time ago, so that at least developers have already tested them.
  • Included commits must not break savegame format, i.e. they must not change any calls to idSaveGame/idRestoreGame objects.
  • Included commits should be small and have little risk of breaking something.
  • Do not include commits which provide a new feature: new features are for new major release.

Usually, commits are included in the following cases:

  • Fixing game crashes, race conditions, data corruption. Especially the ones which players bump into.
  • Fixing wrong usage of OpenGL API. Or adding workarounds for bad OpenGL drivers. Same for OpenAL and other machine-dependent stuff.
  • Fixing interaction with outside projects (FM database, DarkRadiant, tdm_installer, etc.) in case it has been broken.

Note that unlike typical convention, we guarantee full savegame compatibility between the hotfix release and its predecessor. The game writes SVN revision into every savegame and checks that it is the same when it loads game, complaining if it does not match. To silence this warning, go to the method RevisionTracker::GetSavegameRevision and make it return the hardcoded number: the revision reported by the original major release. Commit this change to branch (but not to trunk): now all executables created from this branch will use the same revision number when writing and reading savegames. Don't forget to test that savegames created in original release load properly in hotfix release and vice versa.

When everyone agrees on the set of included commits, merge them all to the release branch.

Now do the typical steps for creating a new release:

  1. #Compile Game Binaries
  2. #Build Manifest
  3. #Create Package
  4. #Add Release to zipsync database.
  5. Write #Changelog and Announcement.

Use release branch. Name the added version appropriately in zipsync database (e.g. release209a). Write announcement on public forums with changelog describing the merged commits, and instructions how to get hotfix (like this one).

After a very brief public beta-testing, move the default=1 line from the original release to the hotfix release in tdm_installer.ini. Now everyone who runs tdm_installer without explicitly specifying version will get the hotfix release.

Now do the ordinary finalization steps:

  1. Publish 32-bit executables.
  2. Commit 7zipped Windows PDB files to devel/release/debugging, replacing the files from original release.
  3. Create SVN tags in both repos.
  4. Update website: replace link to 32-bit executables and source code archive.
  5. Write announcements, so that people know they should update.

Of course, you can provide more than one hotfix release for one major release: 2.09a, 2.09b, 2.09c, etc. Also note that the original release is not lost: players can even install it if they want.