Hot Reload

Glossary

There are several different terms used for various components of this feature. To avoid any further confusion, here they are:

• Hot reload means applying map changes to an already running game without restarting it. The term came from programming world, where it is especially popular in javascript world (see e.g. this). The feature was added to TDM 2.09. Hot reload by itself does not need DarkRadiant or any special connection: in fact, the very first implementation was reloadMap console command.

• Automation is the feature of TDM which allows external programs and scripts to communicate and send commands to the game. It was originally introduced in TDM 2.07 for automated testing of the game using Python scripts, but was later repurposed for other needs, including DR connection.

• Game connection is the feature of DarkRadiant which allows it to communicate with TheDarkMod game, send commands, and fetch back information. On the game side, this communication works through the automation framework described just above, and hence is specific to TheDarkMod. While hot reload is the major functionality of the game connection feature, there are other completely unrelated ones, like camera synchronization for instance. The game connection exists as a TDM-only plugin and was first released in DR 2.9.0.

ReloadMap

First let's consider the reloadMap command, which can be used manually in game console.

Whenever the game loads the .map file from disk, it saves its contents in memory. This data is used for spawning entities when the game starts. While it is not necessary after that, it still remains in memory, which is very convenient for the hot reload feature.

ReloadMap command loads the .map file from disk again, and compares its new contents against the old ones stored in memory. Then all the differences in the .map file are classified into three classes:

1. New entity has been added: it was missing in the old data, but present in the new data.
2. Existing entity has been removed: it was present in the old data, but missing in the new data.
3. Existing entity has been modified: it is present both in old and new data, but the set of its spawnargs or their values have changed.

Then reloadMap acts upon the detected changes. All the added entities are spawned using the new spawnargs, just the same way entities are spawned when a new game starts. All the removed entities are deleted as if script method remove was called for them.

Modified entities is the most complicated case: some of them are respawned (i.e. deleted and spawned again), and some of them are modified directly. There is a hardcoded blacklist of spawnargs (like def_attach or classname), changing any of them immediately results in entity respawn. If none of them are touched by the difference, then the entity is updated directly. I guess not all possible updates are supported properly yet. Here is the list of supported spawnargs: model, skin, origin, rotation.

When all changes are applied to the game, the new state loaded from .map file is saved in memory, while the old state is discarded. Subsequent execution of reloadMap will compute difference against this new state.

Keep in mind that reloadMap matches entities in old data to the entities in new data by their name, i.e. by "name" spawnarg. If you rename an entity, the difference will consist of removal of the old entity + addition of the new entity. This case can be used to force respawn of an entity you are modifying, as well as the (delete + reloadMap + undo + reloadMap) combo.

Please note that the difference is computed on .map files, reloadMap does not look at how the entities are doing in-game right now! It is very important because this was the changes are local: if you don't modify some entity in editor, the hot reload algorithm won't touch it in game. But this approach can also lead to some issues when .map file and actual game state get out of sync. For instance, it can easily happen that an entity which you have just modified in editor has already been removed from the game completely.

Such problems are resolved in the following way. If modified entity does not exist in-game, then it is spawned afresh. If removed entity already does not exist in-game, then the removal is ignored. If added entity already exists in-game, then the addition is ignored with console warning, and the existing entity remains in-game (which can be really confusing).

The hot reload only supports entities. Changes to brushes and patches are not supported and most likely won't be supported, since dmap depends really hard on them. There are some ideas of "dynamic" brushes which behave like some polyhedral models in-game, but it is not clear yet if it is worth the trouble.

DarkRadiant

The "game connection" plugin offers some integration between DarkRadiant and TheDarkMod. As of now, it is fully contained in the "Connection" menu. There are plans to extract it to its own GUI window or toolbar in future.

Here is what you have to do in order to use the plugin:

1. Start TheDarkMod game.
2. Select the FM and start the map you want to work with.
3. Open game console and ensure that com_automation cvar is set to 1 (i.e. automation is enabled), so 'com_automation 1'
4. Start DarkRadiant.
5. Select the FM and open the map --- that same ones that you opened in TDM.
6. Now you are ready use any features from the "Connection" menu.

When you first enable automation in TDM, Windows OS will most likely warn you that the game tries to listen to network. This is how automation works: it opens a socket (which is usually used for communicating over network) and listens for incoming connection. In our case no network is involved: the DR editor should connect directly to the TDM game from the same computer. So if Windows asks you about network permissions, you can safely deny all of them: same-computer connections won't be blocked anyway.

One of the basic features is camera synchronization between 3D preview window in DR and player position in TDM. "Enable camera synchronization" and "Disable camera synchronization" menu items enable/disable automatic continuous synchronization from DM to TDM. That's probably the first thing you should try to check that connection has been established properly: move camera in DR preview window, and the player in TDM should follow. Note that enabling this mode automatically enables noclip, god, and notarget in game.

The menu item "Sync camera back now" puts DR camera to the position where TDM player is located right now (once, not continuously). It can be useful if you decided to explore some stuff in TDM as player, and then want to put both cameras to the place you got into.

Other features, which are not related to hot reload, include:

• "Pause game": toggles g_stopTime cvar.
• "Respawn selected entities": executes respawn command for all entities that you have selected in DR right now.

Hot reload in DR

The most useful feature of DR game connection is hot reload, of course =) There are plenty of ways how it can be used, and you are encouraged to try them all and find which are most useful in which cases. Fundamentally, there are two approaches called: "map reload" and "map update".

The map reload is conceptually the simplest one. When you click on "Reload map from .map file" menu item, DR sends reloadMap console command to TDM, which works exactly as described above. Since TDM reloads data from the .map file, it makes sense to save your current state in DR before using this menu item. To avoid excessive click, you can enable/disable automatic reload with "Enable automatic .map reload on save" and "Disable automatic .map reload on save". When this mode is enabled, DR automatically sends reloadMap command to TDM every time you save the map in DR.

The main problem with the "map reload" mode is that you have to wait before seeing the effect of your changes in TDM. DR has to save the whole map to disk, then TDM has to read the whole map from disk, and compute the difference between the old state and the new state. This is hardly a problem for small maps, but can take dozens of seconds for the big maps.

The map update mode works much faster, because it avoids saving the whole map. Instead, DR looks which entities were modified since last update, writes down their definitions, and sends them over the socket to TDM, without writing them to disk. This results in almost instantaneous update in TDM on map of any size (TODO: was that performance issue in DR fixed?) Internally, the differences are packed into special "reloadmap-diff" automation command (as a mapper, you should not care about it).

In order to use this mode, you have to enable it first. It is enabled/disabled using "Enable map update mode" / "Disable map update mode" menu items. When you enable this mode, it forces a save of your map to disk and issues reloadMap command to TDM. It is done in order to save your sanity, because otherwise it would be really hard to understand which differences get applied when. Thanks to the forced save, the state of entities in DR, in .map file on disk, and in TDM are all synchronized.

Merely having the update mode enabled does not mean that any updates are done automatically. You can issue update at any time by clicking "Update map right now": it would apply the differences since previous update (or since enabling update mode) to TDM. If you are annoyed by clicking this menu after every time, then you can enable auto-update with menu item "Always update map immediately after change". Then DR would issue update after every change you do in DR. Obviously, this enables the fastest hot-reload possible: you simply move/change stuff in DR, and changes get applied to TDM quickly without any further actions.

One really bad thing about current menu-centric GUI is that user cannot see which mode is enabled, and whether the connection is alive at all. Hopefully, it will be fixed in future with new GUI.