Video Cutscene Creation

By Geep, 2021. The following are generalizations of lessons learned while creating FM Away0.

Introduction

Sometimes, the thrust of your story exceeds what can be accommodated by live play, or even a live-action cutscene, and a canned video cutscene provides a solution. These should be kept as brief and engaging as possible. Expect that creating a cutscene, whether video or live, will take time and care.

Creation of a video cutscene shares some elements with creation of a live-action scene (see Cutscenes), but differs in these regards:

• A live-action cutscene, while it may jump around to various viewpoints, is essentially one long take without pause. A video cutscene can be easily broken up into separate sub-scenes, recorded separately and brought together in editing. If the planned action is complicated, decomposition into sub-scenes will make production easier or even possible.
• With live-action, one or more cameras are used to establish points-of-view (POVs), while the player’s position, hidden, serves as the microphone location. When recording a video cutscene, it is simple to use the player as both the video and audio POV.
• With a video cutscene, you can decide whether a particular effect (e.g., slow motion) is most easily created in the TDM world, or with the screen recorder or video and sound editors. Also, you have more control… AI injecting unwanted commentary at odd times, or looking the wrong way? Leave it on the cutting room floor.
• A downside of video cutscene recording, unless you have specialized screen-capture hardware, is the conflict between TDM and the video recording software for resources (CPU, GPU, I/O). Experiments with software and system settings to find an acceptable compromise can take time. More about this further below.

Creation of a video cutscene likewise tends to differ from a Video Briefing:

• A briefing can take the form of slides with video effects, but the cutscene must seem like a view into the 3D game word.
• Audio in the briefing is typically narration and/or music without close synchronization. A cutscene often has video-sync’d dialog or sound effects.
• With a cutscene, continuity with the live game play before and after it are important, as well as visually signaling to the player when the transition occurs. These are not issues for briefings.

Once created, a video cutscene is deployed into an FM as discussed in Full-Screen Video Cutscenes

The Player as Camera POV

The usual keyboard and mouse controls of player movement become the means for simple zooms, pans, and dollies.

Usually, you don’t want the AI to attack you as camera-man. You can do that with console commands (discussed next), or by setting the AI’s “team” spawnarg to 0, or otherwise adjusting AI Relations (Editing) attributes.

Game Settings

You’ll typically want to hide the light gem: Settings > Gameplay

You can also adjust Settings > Video > Advanced > Player Field of View

TDM Console Commands

As an alternative to the foregoing, you can control these in the console:

• tdm_hud_hide_lightgem 0 (where 1 = show)
• g_fov N, where N = angle of field of view. So g_fov 30 would zoom in to restrict the Field of View (FOV) to 30 degrees. Default = 90. You’ll probably want to shoot some sub-scenes with various FOV, and select the best shots at edit time.

You’ll typically want to turn off any on-screen diagnostics, e.g., com_showFPS 0

Other commands, from Console Useful Controls and Cvars in The Dark Mod, that are particularly apt when video recording:

• notarget. When toggled on, in theory, AI not you know you are present. In practice, they sometimes still seem to see and acknowledge you, but at least won’t attack.
• noclip. When toggled on, let’s your camera travel more freely, beyond the pull of gravity, and without the player footsteps up and down.
• pm_noclipspeed N. The default value of N is 200. Changes the speed of player noclip movement. Customize that to fine-tune zoom in and out.
• timescale N, where N of 1 = normal speed, 2 = double, 0.5 = half speed. Speed up or slow down the overall action.

Zooming Using Scripting

Geeps notes: I haven’t tried this, but include it for completion.

You can get much of the benefit of zooming just by moving the player in or out. But should you need true variable-fov zooming, there are script commands you can apply to $player1, namely getFov, startZoom, endZoom, and resetZoom. Descriptions are in TDM Script Reference. The player can be positioned with setOrigin and setViewAngles.

Controlling the Speed of Animation

Besides the "timescale N" console command, you can alter the speed of a particular AI’s walking by setting "anim_rate_walk" to, say, "0.95" to make it 5% slower. Animation playback speed notes some caveats with this, and hints that other "anim_rate_X", where X is an animation name, might be available.

Another approach is to change the framerate of a particular animation. Create your own private copy an "*.md5amin" animation of interest, and edit it with a text editor; the “framerate” line is near the beginning. (Either give your copy a new name and reference that in your AI’s def, or arrange that your local copy overrides the original by mimicking its folder subpath.)

These techniques can cause truncation or unsynchronization of Conversation-driven audio. Tweaking your FM, recording multiple takes, and video editing, even with simple, free tools, can usually work around problems like this. Presumably more sophisticated video and sound editing software will have a richer set of tools for speed adjustments and synchronization.

Sub-Scenes

If your cutscene is just two immobile AI talking to each other, you probably won’t need sub-scenes. Technical, complexity, and continuity issues will usually dictate when they become desirable and where the splits should be. A small room stuffed with 20 path nodes is likely getting unmanageable.

Sets for Sub-scenes

If your cutscene occurs in, say, a single room, it will be good to create a new FM with just that room, plus any vistas likely to be seen and recorded from within, and any additional external architecture that should be included when viewing into the room. This becomes your “base set”, from which you can clone at least the first sub-scene into another FM, and others as convenient. Be sure to leave everything in the same location on the world grid as your main FM, so you can open any 2 maps in separate DR instances and easily cut/paste items between them, without needing translation.

You should assume that you’ll sometimes have to return to a sub-scene after recording it, make a change, and re-record it. This can be managed in several ways:

• Use your favorite version-control method to restore an earlier sub-scene. Note that when you make a change, you will be effect forking that version.
• Create a separate FM for each sub-scene. Either clone the FM containing the base set, or the predecessor sub-scene, as seems best. (As usual, rename the FM folder. In the “maps” sub-folder, rename the .map, .darkradiant, and .script files, and delete the rest. Tweak the darkmod.txt and startingmap.txt contents.)

In either case, there’s always some adding and pruning of resources to do as you march through the sub-scenes.

Behavior Looping

It is usually desirable to set up a sub-scene’s behavior so that it recurs over and over again, rather than having to be triggered each time by the player. That way, you can concentrate on trying different POVs and FOVs, hot-keying your recording software, etc., to do multiple takes as part of the recording session.

Looping can be done in various ways. It could be as simple as a looped path. If the behavior is driven by a Conversation, having the last action of that call a script function that triggers a start_conversation object can form a loop.

Problems with Screen Recording

Geep comments: I’d wish I’d gotten better hardware before doing Away0, so the cutscenes would have been smooth rather than jerky and looking like budget animation.

If you do a lot of game video recording or streaming, you have probably already invested in and tuned up an appropriate setup, such as:

• High-end CPU, GPU, and solid-state drive
• Specialized hardware (e.g., HDMI recorders by companies like Elgato)
• A separate computer for recording.

For the rest of us, it is unfortunately not unusual to have difficulties recording video, and sometimes a “good enough” compromise is all that can achieved. You’ll probably have to consult forums for those using your particular recording tool; your system will make a difference too.

There are two general types of problems:

• Those, like FPS slow-downs, that you see in-game when recording.
• Those, like dropped frames, that occur during the encoding and output and only show up in the video. These are usually reported in the recording software’s log file.

Both of these can be due to resource contention between TDM and the recording software for CPU, GPU, or disk I/O, and will be more prominent on older or underpowered systems. This can be affected by your recording software’s settings. For example:

• Encoding may be possible with either software (CPU) or hardware (GPU). Try both to see what works best.
• The resolution can be reduced, either on your physical screen, or as a pre-encoding resampling stage in your recording software. This trades off number of pixels to process versus coarser pixel artifacts.

TDM fully uses two threads, plus some minor time is taken by async thread and OpenAL mixing. If you have only two physical cores, then recording software will struggle to find processing time.

That observation was from a forum thread with additional suggestions that might improve output:

• Minimize or disable ambient occlusion. If using SSAO high ( r_ssao 3 ), try setting r_ssao 1 ( Low ) then adjusting the related cvars, e.g., r_ssao_radius 64 or 128. There will be more motion and resolution artifacts, but it might be more playable when recording.
• If you have a marginal number of physical cores, try disabling "Frontend Acceleration", which assigns more threads.
• Try setting high priority to your recording software in Window’s Process Explorer as discussed here.

Problems with Audio Sync on Playback

As reported in Time, frames and ticks, when uncapped FPS is off, the game runs slower by 4%. Unfortunately, the required separate audio file runs at normal speed, so the audio will finish before the visuals. Until there is a better solution, if you require lip-sync, you may have to chop your .ogg file into short segments, such as at the start of each sentence, and synchronize them separately. For more, see bug report 5575.