Full-Screen Video Cutscenes - Revision history

Datiswous: /* TEMPORARY CAUTION (March, 2021) */ added link to bugtracker

2024-05-13T08:26:22Z

TEMPORARY CAUTION (March, 2021): added link to bugtracker

← Older revision		Revision as of 08:26, 13 May 2024
Line 3:		Line 3:
	== TEMPORARY CAUTION (March, 2021) ==		== TEMPORARY CAUTION (March, 2021) ==

	If considering creation of a video cutscene with lipsync or other close synchronization, be sure to consult the bugtracker report "0005575: When Uncapped FPS is Off, Video Cutscene Loses Audio Sync", which includes a limited workaround, also discussed below as Technique 3.		If considering creation of a video cutscene with lipsync or other close synchronization, be sure to consult the bugtracker report "[https://bugs.thedarkmod.com/view.php?id=5575 0005575]: When Uncapped FPS is Off, Video Cutscene Loses Audio Sync", which includes a limited workaround, also discussed below as Technique 3.

	== Introduction ==		== Introduction ==

Geep: Change category tag to {{GUI}}, that includes Editing

2022-11-04T18:04:33Z

Change category tag to {{GUI}}, that includes Editing

← Older revision		Revision as of 18:04, 4 November 2022
Line 341:		Line 341:

	[https://forums.thedarkmod.com/index.php?/topic/10003-so-what-are-you-working-on-right-now/page/329/&tab=comments#comment-444897 Magic Screens or Prey-like portals]		[https://forums.thedarkmod.com/index.php?/topic/10003-so-what-are-you-working-on-right-now/page/329/&tab=comments#comment-444897 Magic Screens or Prey-like portals]
	{{~~editing~~}}		{{GUI}}
	[[Category:Video and cutscenes]]		[[Category:Video and cutscenes]]

Geep: /* Build out the GUI */

2022-01-05T17:42:03Z

Build out the GUI

← Older revision		Revision as of 17:42, 5 January 2022
Line 237:		Line 237:
	{		{
	// commands within event handlers must end with ";", unlike elsewhere		// commands within event handlers must end with ";", unlike elsewhere
	set background "video/cutscene_video"; // "set" required here to change initial value		set "background" "video/cutscene_video"; // "set" required here to change initial value
	resetCinematics; // reset video to start		resetCinematics; // reset video to start
	localsound "video/cutscene_video"; // This works too: localsound "sound/video/cutscene_video.ogg";		localsound "video/cutscene_video"; // This works too: localsound "sound/video/cutscene_video.ogg";

Geep: /* Technique 3 - Preserving Lipsync */ Add tip

2022-01-05T17:40:34Z

Technique 3 - Preserving Lipsync: Add tip

← Older revision		Revision as of 17:40, 5 January 2022
Line 143:		Line 143:

	=== Technique 3 - Preserving Lipsync ===		=== Technique 3 - Preserving Lipsync ===
	Here, you first chop of your audio into segments, each ideally under a dozen seconds, ~~at places~~ where you want to (in-effect) force resync. Record the duration of each segment. Then each segment separately gets the audio treatment of Technique 2. While you will be defining multiple sound shaders, that can all happen within a single .sndshd file. After adding multiple triggers to your map, call them in a script such as:		Here, you first chop of your audio into segments, each ideally under a dozen seconds. Chops are made at dialog pauses, or generally any place where you want to (in-effect) force resync. Record the duration of each segment. Then each segment separately gets the audio treatment of Technique 2. While you will be defining multiple sound shaders, that can all happen within a single .sndshd file. After adding multiple triggers to your map, call them in a script such as:

	void dovideo(){		void dovideo(){
Line 154:		Line 154:
	sys.wait(4.7); // last wait may not be necessary, depending on what else your script does		sys.wait(4.7); // last wait may not be necessary, depending on what else your script does
	}		}

			Tip if planning to use this technique: Make sure when recording your choppable audio track that it contains no music or continuous ambient sounds, because that would become ragged by chopping and re-triggering. Instead, provide any such sound separately, through an ambient or regular speaker in the player isolation room, discussed next.

	=== Optional Player Isolation ===		=== Optional Player Isolation ===

Geep: Correct the GUI sample code. Now 3 techniques to use

2022-01-04T20:36:41Z

Correct the GUI sample code. Now 3 techniques to use

← Older revision		Revision as of 20:36, 4 January 2022
Line 1:		Line 1:
	''By Geep, 2020''		''By Geep, 2020-2022''

	== TEMPORARY CAUTION (March, 2021) ==		== TEMPORARY CAUTION (March, 2021) ==

	If considering creation of a video cutscene with lipsync or other close synchronization, be sure to consult the bugtracker report "0005575: When Uncapped FPS is Off, Video Cutscene Loses Audio Sync", which includes a limited workaround.		If considering creation of a video cutscene with lipsync or other close synchronization, be sure to consult the bugtracker report "0005575: When Uncapped FPS is Off, Video Cutscene Loses Audio Sync", which includes a limited workaround, also discussed below as Technique 3.

	== Introduction ==		== Introduction ==
Line 22:		Line 22:
	* It avoids any interactions with AI while the player has no sight control.		* It avoids any interactions with AI while the player has no sight control.
	Other pluses for this method are:		Other pluses for this method are:
	* It supports "localsound", the quickest-to-implement audio method;
	* If you want a black frame around your cutscene (and to not have to burn it in with your video editing software), that's easy;		* If you want a black frame around your cutscene (and to not have to burn it in with your video editing software), that's easy;
	* It can be extended to hybrid cutscenes with live action in front of a video backdrop.		* It can be extended to hybrid cutscenes with live action in front of a video backdrop.
Line 83:		Line 82:
	}		}

	=== ~~Building~~ out the GUI ===		=== 3 Techniques to Build out the GUI and Trigger Video and Audio ===

			Technique 1 is quickest to implement, but can cause loss of lipsync, depending on which value the gamer sets for "Settings/Video/Advanced/Uncap FPS". So can Technique 2, which might be considered a waystation to Technique 3.

			=== Technique 1: GUI with Localsound ===
			''Jan 2022 Note: GUI code example updated to correct punctuation errors that quietly caused problems. Thanks, stgatilov''

	Fill in cutscene_video.gui with:		Fill in cutscene_video.gui with:
Line 92:		Line 96:
	backcolor 1,1,1,0		backcolor 1,1,1,0
	matcolor 0, 0, 0, 1		matcolor 0, 0, 0, 1
			nocursor 1

	background "video/cutscene_video"; // mp4 shader defined in materials/cutscene_video.mtr file.		background "video/cutscene_video" // mp4 shader defined in materials/cutscene_video.mtr file.

	onTime 0		onTime 0
	{		{
	~~nocursor 1 // This work ONLY in this location, not in Desktop or after reset/matcolor here~~
	resetCinematics; // reset Video to start		resetCinematics; // reset Video to start
	Desktop::matcolor 1,1,1,1; // show video		set "Desktop::matcolor" "1,1,1,1"; // show video
	~~// Doesn't work:~~ localsound "video/cutscene_video"; // shader as defined in sound/cutscene_video.sndshd		localsound "video/cutscene_video"; // shader as defined in sound/cutscene_video.sndshd
	}		}
	}		}

	~~Be aware that, while this specific code~~ suppresses the appearance of the standard large clock-hand cursor seen in the TDM main menu, ~~small code changes can cause it to re-appear, dead in~~ the ~~center of the screen~~.		The "nocursor 1" line suppresses the appearance - dead in the center of the screen - of the standard large clock-hand cursor seen in the TDM main menu.

			=== Technique 2: Trigger the Audio Separately ===
			Use the same GUI as above, except remove the "localsound" line.

	~~=== Handling~~ the ~~Audio ===~~		Then, handle the audio. Someplace in your map, following the [[Script_objects\| tdm_voice]] approach to a narrator's or player's voice, create two objects:
	Someplace in your map, following the [[Script_objects\| tdm_voice]] approach to a narrator's or player's voice, create two objects:
	* an atdm:trigger_voice entity (say, atdm_trigger_voice_1)		* an atdm:trigger_voice entity (say, atdm_trigger_voice_1)
	* an atdm:voice entity (e.g., atdm_voice_1)		* an atdm:voice entity (e.g., atdm_voice_1)
Line 130:		Line 136:

	void dovideo(){		void dovideo(){
	sys.trigger($atdm_gui_message_1); //start the cutscene		sys.trigger($atdm_gui_message_1); //start the cutscene. All you need for Technique 1.
	sys.trigger($adtm_trigger_voice_1); //~~and~~ audio		sys.trigger($adtm_trigger_voice_1); //start the audio, for Technique 2.
	}		}

	~~This~~ function can be called by any triggering device you prefer. The value of a script function (as opposed to a triggering relay) is more apparent in what ~~follows.~~		The dovideo function can be called by any triggering device you prefer. The value of using a script function here (as opposed to a triggering relay) becomes more apparent with Technique 3 or with player isolation described below.

			=== Technique 3 - Preserving Lipsync ===
			Here, you first chop of your audio into segments, each ideally under a dozen seconds, at places where you want to (in-effect) force resync. Record the duration of each segment. Then each segment separately gets the audio treatment of Technique 2. While you will be defining multiple sound shaders, that can all happen within a single .sndshd file. After adding multiple triggers to your map, call them in a script such as:

			void dovideo(){
			sys.trigger($atdm_gui_message_1); //start the cutscene.
			sys.trigger($adtm_trigger_voice_1); //start audio segment 1
			sys.wait(12.3); // duration of segment 1
			sys.trigger($adtm_trigger_voice_2); //start audio segment 2
			sys.wait(8.4); // duration of segment 2
			sys.trigger($adtm_trigger_voice_3); //start audio segment 3
			sys.wait(4.7); // last wait may not be necessary, depending on what else your script does
			}

	=== Optional Player Isolation ===		=== Optional Player Isolation ===

Geep: Add link

2021-11-26T02:41:14Z

Add link

← Older revision		Revision as of 02:41, 26 November 2021
Line 253:		Line 253:

	=== Choosing in Advance to Skip a CutScene ===		=== Choosing in Advance to Skip a CutScene ===
	An alternative to these shortcomings is to offer some sort of switch in-game, so the player can indicate that they'd rather not see the upcoming cutscene, presumably because they've already seen it. For example, in "Away 0: Stolen Heart", there's a wall panel with glowing text that says "Skip Cutscene? Frob Me". If the player clicks that, the text disappears and a script variable is set. When, shortly later, the player arrives at where the cutscene would be triggered, a replacement slide is shown instead. More about that in [[A Full-Screen Slide in Mid-Game]].		An alternative to these shortcomings is to offer some sort of switch in-game, so the player can indicate that they'd rather not see the upcoming cutscene, presumably because they've already seen it. For example, in "Away 0: Stolen Heart", there's a wall panel with glowing text that says "Skip Cutscene? Frob Me". If the player clicks that, the text disappears and a script variable is set. When, shortly later, the player arrives at where the cutscene would be triggered, a replacement slide is shown instead, to maintain story continuity. More about that in [[A Full-Screen Slide in Mid-Game]].

	=== Multiple Non-Sequential Cutscenes in an FM ===		=== Multiple Non-Sequential Cutscenes in an FM ===
Line 300:		Line 300:
	== See Also ==		== See Also ==
	[[Video Cutscene Creation]] gives pluses and minuses of video versus live-action cutscenes, and some useful tips for scene/set preparations and camera control during capture.		[[Video Cutscene Creation]] gives pluses and minuses of video versus live-action cutscenes, and some useful tips for scene/set preparations and camera control during capture.

			[[A Full-Screen Slide in Mid-Game]] shows how to fashion a replacement slide for use when the player wants to skip a video cutscene.

	[[Cutscene video with FFmpeg]] for video codecs supported in TDM 2.06+ and recommended for new FMs, such as MP4 or AVI. Also ideas on video looping.		[[Cutscene video with FFmpeg]] for video codecs supported in TDM 2.06+ and recommended for new FMs, such as MP4 or AVI. Also ideas on video looping.

Geep: /* Discussion */ Added Choosing in Advance to Skip a CutScene

2021-11-26T02:32:50Z

Discussion: Added Choosing in Advance to Skip a CutScene

← Older revision		Revision as of 02:32, 26 November 2021
Line 251:		Line 251:

	''With the Movie Theatre Method.'' If you hit ESC once, you will break out of the camera view, so end up seeing the movie theatre, with the sound still playing. Hitting ESC again kills the audio and brings you do the main menu. "Resume" then returns you to the theatre (but NOT the camera view), and with no sound.		''With the Movie Theatre Method.'' If you hit ESC once, you will break out of the camera view, so end up seeing the movie theatre, with the sound still playing. Hitting ESC again kills the audio and brings you do the main menu. "Resume" then returns you to the theatre (but NOT the camera view), and with no sound.

			=== Choosing in Advance to Skip a CutScene ===
			An alternative to these shortcomings is to offer some sort of switch in-game, so the player can indicate that they'd rather not see the upcoming cutscene, presumably because they've already seen it. For example, in "Away 0: Stolen Heart", there's a wall panel with glowing text that says "Skip Cutscene? Frob Me". If the player clicks that, the text disappears and a script variable is set. When, shortly later, the player arrives at where the cutscene would be triggered, a replacement slide is shown instead. More about that in [[A Full-Screen Slide in Mid-Game]].

	=== Multiple Non-Sequential Cutscenes in an FM ===		=== Multiple Non-Sequential Cutscenes in an FM ===

Geep: /* See Also */ Add link to Video Cutscene Creation

2021-11-24T03:45:05Z

See Also: Add link to Video Cutscene Creation

← Older revision		Revision as of 03:45, 24 November 2021
Line 296:		Line 296:

	== See Also ==		== See Also ==
			[[Video Cutscene Creation]] gives pluses and minuses of video versus live-action cutscenes, and some useful tips for scene/set preparations and camera control during capture.

	[[Cutscene video with FFmpeg]] for video codecs supported in TDM 2.06+ and recommended for new FMs, such as MP4 or AVI. Also ideas on video looping.		[[Cutscene video with FFmpeg]] for video codecs supported in TDM 2.06+ and recommended for new FMs, such as MP4 or AVI. Also ideas on video looping.

Geep: /* Introduction */ reference new Video Custscene Creation page

2021-11-24T03:39:23Z

Introduction: reference new Video Custscene Creation page

← Older revision		Revision as of 03:39, 24 November 2021
Line 7:		Line 7:
	== Introduction ==		== Introduction ==

	Suppose you have created a video for a TDM in-game cutscene, using video capture and AV editing software. Your video is compliant with [[Cutscene_video_with_FFmpeg \| video codecs supported in TDM 2.06+ and recommended for new FMs]] (e.g., MP4 or AVI). Likewise, [[Sound_File_Formats\| TDM-supported sound formats]]. You want to play this cutscene back at a particular point in the game. Here are two methods, both of which necessarily deploy somewhat different software architecture (e.g., GUIs) than that of [[Video Briefing\|briefing videos]].		Suppose you have created a video for a TDM in-game cutscene, using video capture and AV editing software. ([[Video Cutscene Creation]] has thoughts about the why and how of this.) Your video is compliant with [[Cutscene_video_with_FFmpeg \| video codecs supported in TDM 2.06+ and recommended for new FMs]] (e.g., MP4 or AVI). Likewise, [[Sound_File_Formats\| TDM-supported sound formats]]. You want to play this cutscene back at a particular point in the game. Here are two methods, both of which necessarily deploy somewhat different software architecture (e.g., GUIs) than that of [[Video Briefing\|briefing videos]].

	=== GUI Message Overlay Method ===		=== GUI Message Overlay Method ===

Geep at 16:24, 1 April 2021

2021-04-01T16:24:49Z

← Older revision		Revision as of 16:24, 1 April 2021
Line 184:		Line 184:
	{		{
	qer_editorimage textures/editor/video		qer_editorimage textures/editor/video
	noshadows		noshadows
	guiSurf entity		guiSurf entity
	discrete		discrete
	translucent		translucent
	sort "-2"		sort "-2"
	{		{
	videoMap video/cutscene_video.mp4 // Any embedded sound ignored; TDM playback method uses separate sound file.		videoMap video/cutscene_video.mp4 // Any embedded sound ignored; TDM playback method uses separate sound file.
	remoteRenderMap 512 266 // matches ratio of projection surface		remoteRenderMap 512 266 // matches ratio of projection surface
	red 1		red 1
	green 1		green 1
	blue 1		blue 1
	scale 1,1		scale 1,1
	translate 1,1		translate 1,1
	}		}
	}		}

	For more about possible color channel parameterization of this, and color/resolution effects, see ~~Obstrict~~'s video in [~~Security Camera~~].		For more about possible color channel parameterization of this, and color/resolution effects, see Obsttortes's video in [https://forums.thedarkmod.com/index.php?/topic/14394-apples-and-peaches-obsttortes-mapping-and-scripting-thread/page/11/&tab=comments#comment-349960 How to get security camera observation screens working].

	=== Build out the GUI ===		=== Build out the GUI ===
Line 223:		Line 223:

	The "background black" command here is optional; it generates a console warning but does show the screen as black before the movie plays. Or delete the line to have the pre-movie screen invisible.		The "background black" command here is optional; it generates a console warning but does show the screen as black before the movie plays. Or delete the line to have the pre-movie screen invisible.

	~~sound/video/cutscene_video.ogg~~

	Then link your screen entity "video_wall" to this, by adding the spawnarg:		Then link your screen entity "video_wall" to this, by adding the spawnarg: