Subtitles

From The DarkMod Wiki
Revision as of 09:20, 20 January 2023 by Geep (talk | contribs) (Discuss i18n)
Jump to navigationJump to search

Support for subtitles (or closed captions) was added in TDM 2.10.

What the gamer sees

In 2.10+, the player can control subtitle visualization under Settings/Audio/Subtitles, which takes one of these values:

  • Story - display only story-related subtitles
  • On - display subtitles for all speech
  • Off - disable subtitles

"Story" is the default. If "On" is chosen, story and non-story speech subtitles (like for AI "barks") appear.

Each subtitle has an implicit or explicit start and end time relative to its audio clip. To handle multiple overlapping sounds, the text is not interwoven; instead, there are 3 separate, non-overlapping fields, stacked in the lower part of the screen. When a subtitle is shown, it is as white text on a translucent field-rectangle background. A text line that is too long is wrapped to two lines.

While subtitles in movies are often associated with translations, the near-term purpose for TDM's subtitles is to provide English captions for English audio. This will assist understanding of accented dialog for those who are hearing impaired or whose English is less fluent. In theory, subtitles could be relied upon to play an FM silently, but the lack of audio positional and distance cues, e.g., for footsteps, would be detrimental.


Rollout so far

As of 2.11

The "Subtitles" setting affects the CVar "cv_tdm_subtitles", as follows:

  • Story --> 1 --- show only story-relevant
  • On --> 2 --- show all speech
  • Off --> 0 --- hide all

There is an additional cv_tdm_subtitles value, 3. It means "show everything", including additional subtitles for sound effects. No core sound effect subtitles are provided yet.

Subtitles are not generated on the fly. They are pre-encoded into game files (as detailed below), and this will take human effort over time. "Story" subtitles are chiefly associated with custom sound files and the responsibility of the FM author. Conversely, subtitles for AI standard speech phrases (as well as standard sound effects) are seen as mainly core resources.

For the FMs included in the standard distribution, "Tears of Saint Lucia" (in 2.10) and "A New Job" (planned in 2.11) have "Story" speech subtitles.

Non-story speech (as of 2.10) is limited to a dozen phrases of the "cynic" voice, used by some attacking guards. Since this is part of the core distribution, it becomes available to existing games automatically.

Internationalization using TDM's existing methods (e.g., "#str") would require further extension to handle subtitles. So these potential use cases are not now supported:

  • English audio with non-English subtitles
  • Non-English audio with non-English subtitles
  • Non-English audio with English subtitles

However, a full port to a specific different language on a non-TDM site could replace all audio and subtitle files to likely achieve some such aims.


How it works

Subtitles work on a very low level: where the engine manages sound samples and sound channels. This approach allows to reliably add subtitles for sounds of any kind, but has some downsides too: the system does not know anything at all about entities, spawnargs, scripts, and other gameplay stuff. The system allows to assign chunks of text directly to sound samples (which are usually .ogg and .wav files), so that when a sound sample is played, the assigned text is shown on the screen.


Subtitles decls

The assignment of text to sound samples is specified in a new type of decl files. These files must:

  1. be in subtitles directory,
  2. have .subs extension,
  3. contain declarations of type subtitles.

For comparison, materials are another type of decls, which must be in materials directory, in files with .mtr extension, and be of type material.

To avoid any conflicts between core game and missions, FM authors should follow the conventions:

  1. Names of subtitles decl files must start with fm_, e.g.: fm_conversations.subs
  2. Names of declarations must start with fm_ too, e.g.: fm_paul_intro_convo


Here is an example of fm_conversations.subs contents: 

//note: comments work as they normally do

subtitles fm_intro_convo {
    verbosity story
    inline "sound/voices/paul/sound1.ogg" "My name is Paul, and I'm a city guard."
    inline "sound/voices/bjorn/sound2.ogg" "Welcome to Anonymous City Guards! Call me Bjorn."
    inline "sound/voices/paul/sound3.ogg" "I started cutting thieves' throats when I was 17..."
    inline "sound/voices/paul/sound4.ogg" "... and now I cannot stop doing that!"

    verbosity effect
    inline "sound/voices/paul/sound5.ogg" "(starts weeping)"

    verbosity story
    inline "sound/voices/bjorn/sound6.ogg" "Calm down, Paul! We all were in your shoes!"
    inline "sound/voices/bjorn/sound7.ogg" "Let us first watch an educational video about the harm from cutting throats."

    srt "sound/voices/bjorn/sound8_long.ogg" "sound/voices/bjorn/sound8_long.srt"
}

subtitles fm_briefing {
    verbosity story
    srt "fromVideo video/briefing/briefing.mp4" "video/briefing/briefing.srt"
}

displayed text

There are two ways to specify subtitle text: inline and srt-based.

inline command is followed by a name of sound sample and the text to be displayed while it is being played. In this case, the text is written straight in the decl file, and it is displayed for the whole duration of the sound sample. This way is convenient for short sounds, which are the majority of sound samples, including phrases of a conversation.

srt command is followed by a name of sound sample and a path to .srt file. The .srt file format is described e.g. on Wikipedia. The file must be in engine-native encoding (internationalization is not supported yet anyway) and have no BOM mark. It contains a sequence of text messages to show during the sound sample, with timestamps and durations. It is recommended to use common software to create .srt files for sound samples, instead of writing them manually. This way is more flexible but more complicated, and it is only necessary for long sounds, for instance sound sample of a briefing video.

sound sample

In order to specify sound sample, write path to the file the same way you do e.g. in sound shaders.

One complicated case is when you have an FFmpeg-based video with embedded sound. In this case write path to the video file, prepended with fromVideo keyword and space, just like you do in the sound shader. See fm_briefing in the example above.

verbosity

Currently there are three verbosity levels for subtitles:

  • story: This level should be applied to all story-related subtitles, everything that player should absolutely notice and understand.
  • speech: This is for subtitles of speech which is usually of little interest to the player, or is repeated too often. For instance, regular AI barks (e.g. "I'll get ya!") belong here.
  • effect: This level is reserved for non-speech subtitles which have little interest to the player, like "(bang)" or "(ding)", etc.

It is not yet clear how exactly these levels will be used in the long term. For now:

  1. By default, players only see story-level subtitles, although they can enable the other two categories in settings, or disable subtitles entirely.
  2. It is expected that almost all FM-specific subtitles get into story category. The other two exist mostly for core TDM sounds, which include a lot of AI barks (speech level) and trivial sound effects like arrow hit (effect level).

Every subtitles declaration must start with verbosity command. This command applies to all the subsequent commands, until the other verbosity is met or declaration ends (whichever happens first). For instance, all subtitles have story verbosity level in the example above, except for the "(starts weeping)" text, which has effect verbosity.

include

While the engine parses all declarations that are present in the files, the subtitles system only uses one subtitles decl named tdm_root. Include command is used in order to put other subtitle decls into it.

include command has one argument: the name of the subtitles decl to be included. Note that it is the name of the decl, not the name of the file which contains it! Included decl can in turn include other decls.

FM-specific subtitles are always included from the decl named fm_root, which must be in fm_root.subs file. For instance, the contents of the fm_test.subs file from the example above will have no effect until we add the file fm_root.subs with contents: 

/**
 * This file should be overridden in order to provide mission-specific subtitles.
 * When doing so, please follow the conventions:
 *   1) The root decl is called "fm_root" and is located in file "fm_root.subs".
 *   2) All other subtitle decls start with "fm_" prefix and are located in files starting with "fm_".
 */

subtitles fm_root {
    include "fm_intro_convo"
    include "fm_briefing"
    //you can add other includes and add subtitles with "inline"/"srt" commands here
}

Notice that one decl can be used to do any number of subtitle assignments, so it is not necessary to have many decls and decl files. The easiest way to do FM-specific subtitles is to have one fm_root.subs file with one fm_root decl, and specify all your subtitles right there. Splitting subtitles across decls and files is entirely optional and can be used for grouping subtitles. It was added mainly for core TDM sounds: there are several thousands of them, so keeping all subtitles in single file can become a headache quickly.


Output

The subtitles which should be displayed right now are provided to GUI code as gui::subtitleN variables, where N ranges from 0 to 2. GUI scripts display these variables as non-overlapping text messages. Given a logical screen height of 480, each field is 45 units high. N = 0 is the lowest, at Y-origin 400; 1 at 350; 2 at 300. The responsible code is located in tdm_subtitles_common.gui file, and is already used in several places:

  • in the following states of the main menu: briefing, briefing video, and debriefing video
  • in the always-on GUI overlay during gameplay

For example, here is how it is used in mainmenu_briefing.gui: 

//note: each include of tdm_subtitles_common must have different name prefix!
#define SUBTITLES_NAMEPREFIX Briefing
#include "guis/tdm_subtitles_common.gui"

Usually, you don't need to do anything about it. However, this information can be relevant if you write some custom GUI and want to see subtitles there.

Note: it is not clear yet which kind of customization is necessary for the subtitles GUI. If you have any ideas, please share them on forums.

See also

TODO: "Tears of Saint Lucia" mission as an example?...

Retrieved from "https://wiki.thedarkmod.com/index.php?title=Subtitles&oldid=30546"