Mission Database
All released missions are stored in FM database, which defines what is available in mission downloader in-game.
Requirements
In order for a mission to be included in the database, it must satisfy the following conditions:
- It has gone through beta-testing by several forum members who did not take part in its creation.
- As a rare exceptional case, a mission can be rejected from the database if the majority of beta-testers come to conclusion that it is unplayable or its quality is way too low.
- Does not contain any questionable material, potential intellectual property infridgements. Including:
- using names from the original Thief games
- using assets from other games
- having content which is so bad that it is forbidden even on our forums (link to forum rules)
A very few missions are installed along with the game, like Saint Lucia or Training Mission. These official missions are considered to be part of the game, and are not included in the database.
SVN
Starting from 2021, the FM database is stored in SVN repository (#5551). Only TDM team members have access to this SVN.
Here is the SVN address:
But please don't rush to checkout the whole repo yet!
Checkout everything
Yes, you can simply copy the link above to SVN Checkout and get single working copy with all the FMs. But keep in mind that you will have to download ~10 GB of data and the working copy will take ~20 GB of space. In most cases you don't need everything in order to work with the database.
The typical reasons to checkout the whole repo are:
- You want to run automated search or tests over all released FMs.
- You are regular committer of the FM database, added a lot of new missions and updates, so the investment is worth it.
Checkout as needed
Instead of checking out the whole repo, you can checkout only the few FMs you are going to modify. This is the recommended approach.
With this approach, you need the Repo-browser feature of TortoiseSVN. It allows to look through all the directories and files on remote SVN without checking it out first.
The directory structure of SVN is show on the picture. Most importantly, all information about FM with internal name "qwerty" is stored in fms/qwerty subdirectory of the repo. You can checkout only this directory in order to work with FM, just put this checkout address:
Or find the FM directory in Repo-browser, right-click and select Checkout.
Detailed instructions in the rest of the article assume this approach.
Add New Mission
Before adding new FM, negotiate internal name with FM author. It must be different from names of all existing FMs, consist of only lowercase letters and digits, be rather short (aim for 10-20 letters). Among many words in the mission title, prefer rare words and proper nouns over common words when composing the internal name.
Open SVN in Repo-browser, find fms directory which contains all the FMs. Right-click on it and choose Add folder in the context menu. Then enter the internal name of FM as the name of the new directory and proceed with commit. Now that the directory has been created, you can checkout it: find the directory in Repo-browser, right-click it and Checkout.
The second step is to upload pk4 files.
Create two sub-directories under the mission checkout directory. One named screenshots and another named mission-name.pk4_dir
( replace mission-name with your preferred name )
Unpack all the pk4 files of FM pack into the mission-name.pk4_dir sub-folder of the working copy directory (i.e. checkout directory).
Open the directory in Windows Explorer, right-click the parent folder and select SVN commit.
Select all files with Shift, right-click and select Add.
Also set checkboxes for these files.
Write commit message in the text area above: it should start with internal name of FM in brackets.
When everything is done, hit OK to do the actual commit.
While the pk4 files are already in the repository, they are not yet visible in the database. A mission is only added when its directory contains fminfo.xml file, so now you need to add it. You can take this file from another FM (find it in Repo-browser, right-click, Open with, select text editor), and adjust it for the FM being added. Here is explanation for some fields:
- internalName defines name of the directory and pk4 file.
- title is the name seen by players in-game.
- author is one or several people who made the mission.
- releaseDate shows when the very first version of the mission was added.
- type is multi if mission contains several playable .map files (i.e. is campaign), and single otherwise.
- size is size of the main pk4 file in megabytes displayed to users.
- version is natural number used by in-game downloader to decide whether update is available or not. Starts with 1.
- description contains text displayed in in-game downloader when player inspects mission details.
- mainPack points to the main pk4 file of the mission. Note that the name of file is fully determined by internal name.
- localisationPack points to _l10n.pk4 file if it exists.
Note that XML cannot directly contain some characters, thus they must be escaped:
- Ampersand (&) quotes (" or ') and angle brackets (< or >): reference
- Line break symbol can be inserted as according to reference
Example:
<fminfo>
<mission title="Closemouthed Shadows" releaseDate="2012-01-15" size="2.3" version="2" internalName="closemouthed_shadows" type="single" author="LordSavage, Bikerdude">
<description>Find Dougal's key, enter his place and kill him. Find the golden plate and key he stole from his neighbor. Get some loot including Lord Julian's Scepter. This is the original mission from 2008 (modified to run with TDM Version 1.07). Big thanks to b1k3rdude who has helped me to bring Closemouthed Shadows to TDM 1.07 (and improved some text and visuals).</description>
<mainPack filename="closemouthed_shadows.pk4"/>
<localisationPack filename="closemouthed_shadows_l10n.pk4"/>
</mission>
</fminfo>
When you have created fminfo.xml file, double-check that all properties are correct. Then use SVN to add and commit the file, just like you did with pk4 file.
If you did something wrong (most likely), then you will see an error saying that "Commit blocked by pre-commit hook". A long stacktrace from Python script is included, and the meaningful message should be at the end of it. Typically, it is either XML validation error saying that something in fminfo.xml is wrong, or a message from some custom failed check. You need to fix the errors and try to commit again --- until you manage to commit successfully.
As the last step, create subdirectory named screenshots in the working copy directory. Put screenshots in .jpg or .png format into the directory: they will be displayed in in-game mission downloader. Then add and commit all the screenshot files into the repository, same way as you did for pk4 and xml files.
Update Mission
This section covers the case if mission has already been released, but new version should be uploaded.
First of all, make sure you have up-to-date working copy of the FM directory.
If you already have working copy, do right-click and SVN Update in it in Windows Explorer.
If you don't have it yet, then open Repo-browser, find the directory named by FM's internal name, right-click and select Checkout.
If you don't know the internal name, you can learn it like this: install the FM in the game,
then look what is written in the currentfm.txt file.
Suppose internal name is "qwerty".
If you know the author didn't remove any files from the new version, just extract the contents of the new PK4 to the query.pk4_dir and overwrite the contents.
Then browse the sub-folders for any un-versioned new items ( usually will have a question mark indicator ) and right-click them and select SVN Add
Then update the version number in the FM XML file as well as the file size
Make sure no files or folders have spaces or special characters in their names
Navigate to the parent directory of the querty folder and right-click querty then choose commit
If the mission author removed files or you are unsure, you will need to compare the existing copy to the new version prior to merging
For Windows users the program WinMerge is a good tool, Linux users can use "diff -r folderA, folderB"
Identify additions and removals then overwrite the contents of the query.pk4_dir then perform SVN Add and SVN Delete actions to all relevant items
Then update the FM XML and commit as described above
RabbitVCS Steps
Since TortoiseSVN does not have a Linux version, a good alternative is RabbitVCS.
You will need to install the shell integration script that is compatible with your GUI File Manager
If none exist you will need to install a secondary GUI File manager that is compatible with RabbitVCS integration
Here is a quick summary of the Mission management process on RabbitVCS
1) Create a parent folder such as mission-name_new
2) Inside the folder create a sub-folder named after the pk4 name ( eg if the mission name is elixir.pk4 the folder is "elixir" )
3) Insider the sub-folder will be 3 things:
An fminfo.xml with details about the mission ( see attached example)
A folder named "screenshots"
A folder named mission-name.pk4_dir which contains the unpacked data from the mission pack ( eg elixir.pk4_dir )
4) Right click on the parent folder and perform an SVN "Import" and choose https://svn.thedarkmod.com/project/missions_unpacked/trunk/fms/ as the Repository where you are importing
5) Visit the mission URL after the import has completed https://www.thedarkmod.com/missiondetails/?internalName=<mission-name> ( replace mission-name with whatever the pk4 name )
To update the mission it's best to delete the folder then create a new parent folder mission-name_update then perform an SVN checkout of https://svn.thedarkmod.com/project/missions_unpacked/trunk/fms/<mission-name>
Use "diff -r /path/to/checkout_dir/mission-name /path/to/extracted/new_pk4_folder/mission-name" in the console to identify changes
References
- Thread on developer subforum: https://forums.thedarkmod.com/index.php?/topic/20624-store-missions-archive-in-svn/
- Mission Database Management thread: https://forums.thedarkmod.com/index.php?/topic/11393-mission-archive-administration
