Subtitles

Support for subtitles (or closed captions) was added in TDM 2.10.

What the gamer sees

In 2.10+, the player can control subtitle visualization under Settings/Audio/Subtitles, which takes one of these values:

• Story - display only story-related subtitles
• On - display subtitles for all speech
• Off - disable subtitles

"Story" is the default. If "On" is chosen, story and non-story speech subtitles (like for AI "barks") appear.

Each subtitle has an implicit or explicit start and end time relative to its audio clip. To handle multiple overlapping sounds, the text is not interwoven; instead, there are 3 separate, non-overlapping fields, stacked in the lower part of the screen. When a subtitle is shown, it is as white text on a translucent field-rectangle background. A text line that is too long is wrapped to two lines.

One immediate purpose for TDM's subtitles is to provide English captions for English audio. This will assist understanding of accented dialog for those who are hearing-impaired or whose English is less fluent. (In theory, subtitles could be relied upon to play an FM silently, but the lack of audio positional and distance cues, e.g., for footsteps, would be detrimental.)

As for translation, i.e., subtitles shown in other TDM-supported languages, the current system provides an important and usable building block. But further design and implementation work would be needed to allow easy switching among multiple subtitle languages, say, in response to the Settings/Video/Language choice.

Rollout so far

As of 2.11

The "Subtitles" setting affects the CVar "cv_tdm_subtitles", as follows:

• Story --> 1 --- show only story-relevant
• On --> 2 --- show all speech
• Off --> 0 --- hide all

There is an additional cv_tdm_subtitles value, 3. It means "show everything", including additional subtitles for sound effects. No core sound effect subtitles are provided yet.

Subtitles are not generated on the fly. They are pre-encoded into game files (as detailed below), and this will take human effort over time. "Story" subtitles are chiefly associated with custom sound files and typically the responsibility of the FM author. Conversely, subtitles for AI standard speech phrases (as well as standard sound effects) are seen as mainly core resources.

For the FMs included in the standard distribution, "Tears of Saint Lucia" (in 2.10) and "A New Job" (planned in 2.11) have "Story" speech English subtitles.

Non-story speech (as of 2.10) is limited to a dozen phrases of the "Cynic" voice, used by some attacking guards. Since this is part of the core distribution, these English subtitles become available to existing games automatically.

Potential for translations

An FM author could choose to provide story-tagged subtitles in any particular TDM-supported language, e.g., German instead of English. (However, the core speech-tagged barks would still have English subtitles, unless the author over-rode them. That override would be a lot of work and in general not a good idea.)

Presumably, for story-tagged subtitles, the FM author could (with the help of translators) create and distribute multiple different language subtitle files, requiring manual selection and installation. A better, fuller, standardized system is needed in the long term, to dynamically switch both FM-specific story-tagged subtitles and cross-FM core-provided subtitles. Conceivably this might be implemented by multiple language-specific ".subs" files, multiple language markups within .subs, or "#str" internationalization support.

Alternatively, a non-TDM site for a specific non-English audience could host full ports of TDM and its FMs, and create its own subtitle files in the target language (as well as sound files if the site goes as far as "dubbed" translations).

How it works

Subtitles work on a very low level: where the engine manages sound samples and sound channels. This approach allows to reliably add subtitles for sounds of any kind, but has some downsides too: the system does not know anything at all about entities, spawnargs, scripts, and other gameplay stuff. The system allows to assign chunks of text directly to sound samples (which are usually .ogg and .wav files), so that when a sound sample is played, the assigned text is shown on the screen.

Subtitles decls

The assignment of text to sound samples is specified in a new type of decl files. These files must:

1. be in subtitles directory,
2. have .subs extension,
3. contain declarations of type subtitles.

For comparison, materials are another type of decls, which must be in materials directory, in files with .mtr extension, and be of type material.

To avoid any conflicts between core game and missions, FM authors should follow the conventions:

1. Names of subtitles decl files must start with fm_, e.g.: fm_conversations.subs
2. Names of declarations must start with fm_ too, e.g.: fm_paul_intro_convo

Here is an example of fm_conversations.subs contents:

//note: comments work as they normally do

subtitles fm_intro_convo {
    verbosity story
    inline "sound/voices/paul/sound1.ogg" "My name is Paul, and I'm a city guard."
    inline "sound/voices/bjorn/sound2.ogg" "Welcome to Anonymous City Guards! Call me Bjorn."
    inline "sound/voices/paul/sound3.ogg" "I started cutting thieves' throats when I was 17..."
    inline "sound/voices/paul/sound4.ogg" "... and now I cannot stop doing that!"

    verbosity effect
    inline "sound/voices/paul/sound5.ogg" "(starts weeping)"

    verbosity story
    inline "sound/voices/bjorn/sound6.ogg" "Calm down, Paul! We all were in your shoes!"
    inline "sound/voices/bjorn/sound7.ogg" "Let us first watch an educational video about the harm from cutting throats."

    srt "sound/voices/bjorn/sound8_long.ogg" "sound/voices/bjorn/sound8_long.srt"
}

subtitles fm_briefing {
    verbosity story
    srt "fromVideo video/briefing/briefing.mp4" "video/briefing/briefing.srt"
}

displayed text

There are two ways to specify subtitle text: inline and srt-based.

inline command is followed by a name of sound sample and the text to be displayed while it is being played. In this case, the text is written straight in the decl file, and it is displayed for the whole duration of the sound sample. This way is convenient for short sounds, which are the majority of sound samples, including phrases of a conversation.

srt command is followed by a name of sound sample and a path to .srt file. The .srt file format is described e.g. on Wikipedia. The file must be in engine-native encoding (internationalization is not supported yet anyway) and have no BOM mark. It contains a sequence of text messages to show during the sound sample, with timestamps and durations. It is recommended to use common software to create .srt files for sound samples, instead of writing them manually. This way is more flexible but more complicated, and it is only necessary for long sounds, for instance sound sample of a briefing video.

sound sample

In order to specify sound sample, write path to the file the same way you do e.g. in sound shaders.

One complicated case is when you have an FFmpeg-based video with embedded sound. In this case write path to the video file, prepended with fromVideo keyword and space, just like you do in the sound shader. See fm_briefing in the example above.

verbosity

Currently there are three verbosity levels for subtitles:

• story: This level should be applied to all story-related subtitles, everything that player should absolutely notice and understand.
• speech: This is for subtitles of speech which is usually of little interest to the player, or is repeated too often. For instance, regular AI barks (e.g. "I'll get ya!") belong here.
• effect: This level is reserved for non-speech subtitles which have little interest to the player, like "(bang)" or "(ding)", etc.

It is not yet clear how exactly these levels will be used in the long term. For now:

1. By default, players only see story-level subtitles, although they can enable the other two categories in settings, or disable subtitles entirely.
2. It is expected that almost all FM-specific subtitles get into story category. The other two exist mostly for core TDM sounds, which include a lot of AI barks (speech level) and trivial sound effects like arrow hit (effect level).

Every subtitles declaration must start with verbosity command. This command applies to all the subsequent commands, until the other verbosity is met or declaration ends (whichever happens first). For instance, all subtitles have story verbosity level in the example above, except for the "(starts weeping)" text, which has effect verbosity.

include

While the engine parses all declarations that are present in the files, the subtitles system only uses one subtitles decl named tdm_root. Include commands are used in order to put other subtitle decls into it. (The tdm_root decl is in file tdm_root.subs. As of TDM 2.11, this file, and all core non-game-specific tdm_*.subs, can be found in tdm_sound_vocals_decls01.pk4/subtitles/. An fm_root.subs file exists there, whose stub fm_root decl is over-ridden by your FM's fm_root as described next.)

include command has one argument: the name of the subtitles decl to be included. Note that it is the name of the decl, not the name of the file which contains it! Included decl can in turn include other decls.

FM-specific subtitles are always included from the decl named fm_root, which must be in the fm_root.subs file. For instance, the contents of the fm_test.subs file from the example above will have no effect until we add the file fm_root.subs with contents:

/**
 * This file should be overridden in order to provide mission-specific subtitles.
 * When doing so, please follow the conventions:
 *   1) The root decl is called "fm_root" and is located in file "fm_root.subs".
 *   2) All other subtitle decls start with "fm_" prefix and are located in files starting with "fm_".
 */

subtitles fm_root {
    include "fm_intro_convo"
    include "fm_briefing"
    //you can add other includes and add subtitles with "inline"/"srt" commands here
}

Notice that one decl can be used to do any number of subtitle assignments, so it is not necessary to have many decls and decl files. The easiest way to do FM-specific subtitles is to have one fm_root.subs file with one fm_root decl, and specify all your subtitles right there. Splitting subtitles across decls and files is entirely optional and can be used for grouping subtitles. It was added mainly for core TDM sounds: there are several thousands of them, so keeping all subtitles in a single file can become a headache quickly.

Output

The subtitles which should be displayed right now are provided to GUI code as gui::subtitleN variables, where N ranges from 0 to 2. GUI scripts display these variables as non-overlapping text messages. Given a logical screen height of 480, each field is 45 units high. N = 0 is the lowest, at Y-origin 400; 1 at 350; 2 at 300. The responsible code is located in tdm_subtitles_common.gui file, and is already used in several places:

• in the following states of the main menu: briefing, briefing video, and debriefing video
• in the always-on GUI overlay during gameplay

For example, here is how it is used in mainmenu_briefing.gui:

//note: each include of tdm_subtitles_common must have different name prefix!
#define SUBTITLES_NAMEPREFIX Briefing
#include "guis/tdm_subtitles_common.gui"

Usually, you don't need to do anything about it. However, this information can be relevant if you write some custom GUI and want to see subtitles there.

Note: it is not clear yet which kind of customization is necessary for the subtitles GUI. If you have any ideas, please share them on forums.

See also

TODO: "Tears of Saint Lucia" mission as an example?...