Full-Screen Video Cutscenes

By Geep, 2020

THIS PAGE UNDER CONSTRUCTION

Introduction

Suppose you have created a video for a TDM in-game cutscene, using video capture and AV editing software. Your video is compliant with video codecs supported in TDM 2.06+ and recommended for new FMs (e.g., MP4 or AVI). Likewise, TDM-supported sound formats.

You want to play this cutscene back at a particular point in the game. Here are two methods:

1) GUI Message Overlay Method. This method repurposes the standard message entity that overlays the full screen (e.g., as the TDM main menu) to display a video instead. It is the generally-preferred method, because it:

• is the easiest to implement;
• avoids one additional pixel resampling that the other method uses, so is likely to be more performant and the highest possible image quality;
• scales easily to multiple disjoint cutscenes.
• does not require the construction of a separate room (but there are advantages to doing so, as discussed under the next method);

2) Movie Theatre Method. This requires the construction of a separate, isolated movie-theatre-like room (e.g, in the void). You teleport the player there, play the cutscene with the player's view locked to a camera looking at the screen, then teleport back. The isolation has these advantages:

• It provides audio and lighting that is separate from the rest of the map.
• It avoids any interactions with AI while the player has no sight control.
• If you want a black frame around your cutscene (and to not have to burn it in with your video editing software), that's easy.

These methods do not rely on the precisely same software architecture (e.g., same GUIs) that briefing videos do.

Common to Examples of Both Methods

Video and Sound Files

In our examples here, we assume an MP4 and a separate OGG audio file, specifically:

• an MPEG 4 video, 16:9 aspect ratio, called <fm>/video/cutscene_video.mp4
• a separate OGG audio file, stereo, called <fm>/sound/video/cutscene_video.ogg
• both have a length of 88 seconds.

Can an audio track that is embedded in the video be used directly? Probably not at this time. (This is possible with briefing videos, because special programming was done in TDM 2.06 to allow it.)

Sound Shader

Create a sound shader for the OGG, for instance <fm>/sound/cutscene_video.sndshd, with content:

video/cutscene_video { sound/video/cutscene_video.ogg }

GUI and Script Setup

Create the start of a custom GUI, here <fm>/guis/cutscene_video.gui:

windowDef Desktop { // more to come }

Also, add a function to <fm>/maps/<fm>.script to trigger the playback, like:

void dovideo() { // more to come }

GUI Message Overlay Method

Somewhere in your map, create an atdm_guis_message entity, with spawnargs values like:

• name atdm_guis_message_1
• show 87 duration of the video in seconds, minus 1
• gui guis/cutscene_video.gui overrides the standard gui

You can delete any non-default "text" or "lines" spawnargs as irrelevant.

"Show" is the duration of the video in (presumably decimal) seconds, minus 1. This is because the standard code (in scriptobject tdm_gui_message) adds 1 second for a gui-based fade out, but our gui doesn't use a fade out. (And if you set hidden spawnarg "fade_out_time" to 0, it reinterprets it as 1)

Building out the GUI

Fill in cutscene_video.gui with:

windowDef Desktop {

  rect 0 ,0 ,640 ,480
  backcolor 1,1,1,0

  background "video/cutscene_video"; // mp4 shader defined in materials/cutscene_video.mtr file.

  onTime 0
  {

resetCinematics; // reset Video to start

       Desktop::matcolor 1,1,1,1; // show video

localsound "video/cutscene_video"; // shader as defined in sound/cutscene_video.sndshd

  }

}

Be aware that, while this specific code suppresses the appearance of the standard large clock-hand cursor seen in the TDM main menu, small code changes can cause it to re-appear, dead in the center of the screen.

Building Out the Script Function to Trigger the Playback

Fill in your <fm>.script function:

void dovideo(){ sys.trigger($atdm_gui_message_1); //start the cutscene }

This function can be called by any triggering device you prefer. (Or bypass it to trigger atdm_gui_message_1 directly. The value of a script function is more apparent in what follows.)

Optional Player Isolation

It may be a good idea to create an isolated room for the player to reside in during the cutscene. The advantages of this are the same as for separate room used in the Movie Theatre method:

• It provides audio that is separate from the rest of the map.
• It avoids any interactions with AI while the player's in the box.

While you can use the same room design as Movie Theatre, the dimensions are more flexible here, so perhaps your existing blue room in the void could be outfitted for the purpose.

While seeing only the cutscene, the player can still move around, so maybe cover the floor with moss or a carpet to make that quiet. Or pin the player, e.g., in player clip. Also, place some named object (e.g., a rug "start_cutscene_spot") on the floor, to use as a convenient teleport target.

Then alter the doVideo function to teleport the player there, play the cutscene, then teleport to where the player needs to be afterwards:

void dovideo(){ sys.teleport($start_cutscene_spot); sys.trigger($atdm_gui_message_1); //start the cutscene sys.wait(88); // length of video... then teleport away sys.teleport($after_cutscene_spot); }

Movie Theatre Method

Room Setup

Begin by creating the void-sealing room as a cube with an internal dimension of 512 in all 3 directions. Paint the interior dark. (For debugging, it's useful to have a perceptable texture like dark marble, plus a dim private ambient light source.)

Inside this box, near and parallel to one wall, place an internal wall (from a brush converted to func_static) to serve as a screen. Name it "video_wall". Given that we have a 16:9 video, make the size of this wall 512 wide x 288 high. Move it up to be centered vertically inside the room. Select the front face as the screen surface, then apply and fit an entityGUI texture to it.

(Details of how to do that:

• Select one surface as the screen: Control-Shift LMB.
• Using Texture editor/Media, click on the background, search for "entityGUI". When found (under "common/"), select the text, then right-click and "Apply to Selection".
• Using the Surface editor, hit the "Fit" button.

)

We will come back to link the screen to a custom GUI below.

Next, create a func_cameraview entity (which we'll call "func_cameraview_1"), and move it to be centered on the screen, at a perpendicular distance of about 184 units. 90-degree-rotate the camera so its arrow faces the screen. Give it the spawnargs:

• trigger 1
• target video_wall

You may have to rotate the camera futher during debugging the setup. And the distance can be adjusted to taste, knowing that you are trading off edge clipping on various monitors versus black bars. A distance of 200 will provide a black frame (your darkened room walls) around the entire window.

Finally, as discussed with the other method, quiet the player's footfalls and place a teleport target (e.g., "start_cutscene_spot") on the floor under camera.

Build out the GUI

Fill in your custom cutscene_video.gui:

windowDef Desktop {

  rect 0 ,0 ,640 ,480
  backcolor 1,1,1,0
  Desktop::matcolor 0, 0, 0, 1
  background "textures/darkmod/paint_paper/wallpaper_fancy_red"
  // or use "tdm_black" or "black_background"... doesn't much matter because you'll never see it

  background "video/cutscene_video"; // mp4 shader defined in materials/cutscene_video.mtr file.

  onTime 99999
  {

// Delay start time of vid playback

  }

  onTrigger
  {

resetCinematics; // reset Video to start

       Desktop::matcolor 1,1,1,1; // show video

localsound "video/cutscene_video"; // shader as defined in sound/cutscene_video.sndshd

  }

}

Then link your screen entity video_wall to it, by adding the spawnarg "gui guis/cutscene_video.gui".

Trigger the Playback

Fill in you <fm>.script function like:

void dovideo(){ sys.teleport($start_cutscene_spot); entity cam = $func_cameraview_1; cam.activate($player1); //start the camera sys.trigger($video_wall); //start the cutscene sys.wait(88); // length of video cam.activate($player1); //stop the camera sys.teleport($after_cutscene_spot); }

This can be called by any triggering device you prefer.

Discussion

Interrupting or Skipping a CutScene

Support for this is limited. This may be fine, even desirable, particularly with short cutscenes.

With the GUI Message Overlay Method. If you hit ESC, you will return to the main TDM menu. You can subsequently "Resume Mission", and the video will resume where interrupted, but (due to limitations of the localsound procedure) there will be no audio. The GUI shown above does not support skipping with LMB. (Experiments adding a doAction() function seemed to show no response to LMB, whether or not the TDM clock-hand cursor was visible.)

With the Movie Theatre Method. [TO DO]

Multiple Non-Sequential Cutscenes in an FM

With the GUI Message Overlay Method. Give each cutscene it's own atdm_gui_message entity and dovideo function. But they should be able to share an isolation room.

With the Movie Theatre Method. Perhaps some way can be found that doesn't require a separate movie theatre for every cutscene.

Alternative Sound Treatments

Using localsound within the GUI, as above, has the skipping drawback noted, but is otherwise easy to do and provides the expected stereo separation.

Conjecturally, other possible ways to go might be:

• ambient sound through the location system.
• using a narrator's voice with tdm_voice.
• a separate speaker in the isolated room. This is not a great choice. Besides reducing the stereo to mono, the sound source will seem to drift around if the player moves. And the player can move in both methods; even if encased in a playerclip cage around $start_cutscene_spot, rotation still seems possible.