PROC File Format

By Geep, 2022. Mostly derived from Danile Gibson’s 2014 article, except for Overview and Shadow Volumes.

Overview

While your MAP file defines the geometry of your FM, there are additional ways of re-structuring that information that greatly accelerates gametime processing. The computation of this re-structuring is done in advance, by "dmap", which saves the results to intermediary files in your "<FM>/maps/" folder. These files are then read into memory during game loading (e.g., "map" invocation). If the calculations were not done in advance, but instead during loading, game startup would be very slow.

One of these intermediary files is the PROC file. The structures in a PROC file help speed up rendering, texture-mapping to triangles, and visibility determinations. It contains the following.

• For each named entity defined by primitives (i.e., brushes and patches), all its visible triangles, batched up into surfaces sharing a texture.
• A Binary Space Partitioning (BSP) tree, subdividing the space into convex parts, that are then grouped into "Areas" (i.e., visLeafs).
• For the content of each such Area, all its visible triangles, batched up into surfaces sharing a texture.
• A listing of "Inter Area Portals" (i.e., visPortals), indicating visibility between Areas.
• Shadow Volumes, those that can be precalculated.

Shadow Volumes support stencil shadows. The algorithm for this was made famous by Doom3. At runtime, conceptually, the view the player sees at any moment is rendered in two ways:

1. lights off (except ambient). Everything is shadowed.
2. lights on, with light rays passing through objects.

Then (2) is put in front of (1), with "stencil holes" cut through (2) where the shadows are. The shadow volumes are consulted to define the holes. (TDM also offers shadow mapping; this alternative ignores shadow volumes.)

Not found in the PROC file - but of course defined elsewhere - are:

• a separate representation of each named entity defined by an .ase or .lwo models.
• surface triangles that are invisible or never seen by the player.
• Other precalculations: collision detection (CM file) and pathfinding (AAS file)

The File Format

The List of (Geometric) Models

After a 1-line header, the PROC file begins with a series of named "model" sections, which constitute the bulk of the file. These come in two groups, with a shared format. As we shall see, this format specifies constituent surfaces and their textures, grouped by the latter.

Models for Named Entities with Primitives

All these models are listed first. Each represents (and is named after) a named entity with primitives, like this example:

model { /* name = */ "func_static_2571" /* numSurfaces = */ 3

Typical objects defined by primitives are func_statics (like this example), and brush-derived doors. The surfaces are those of the object's brushes and patches, as discussed below.

Models for Areas (aka visLeafs; Groups of BSP Leaves)

The first line of an Area-model looks like this example:

model { /* name = */ "_area0" /* numSurfaces = */ 40

The first Area-model name is always "_area0", then names are numbered consecutively. These names are referenced by number from the PROC-contained:

• Nodes section, defining the tree of the BSP.
• Section about InterAreaPortals (essentially, visPortals), each of which defines a line-of-sight boundary between two Areas.

During binary space partitioning, an FM's map-defined space is split into a list of convex parts, the leaves of the BSP tree. Information about adjacency/connectivity among leaves is known. Subsequently, using a flood-fill algorithm, sets of connected leaves are grouped into Areas (visLeafs) separated by Inter Area Portals. Each leaf of the BSP tree becomes marked either as part of a particular Area, or as an opaque solid (i.e., unreachable). Each Area-model in the PROC file lists the visible surfaces of worldspawn objects contained therein, as explained further below.

A Model and its Surfaces

Every model consists of one or more "surfaces", where a "surface" is a set of visible triangles sharing a particular material/texture. For an entity with primitives, these are the triangles making up its faces. For an Area-model, these are triangles contained in this Area, independent of which worldspawn object they come from. The overall form is:

model { /* name = */ <name> /* numSurfaces = */ <number of surfaces>
<surface 0>
<surface 1>
...
}

where <number of surfaces> gives the count of <surface i> that follow. Each <surface i> begins a one-line header like this example:

/* surface 2 */ { "textures/darkmod/wood/boards/old_small_grainy" /* numVerts = */ 118 /* numIndexes = */ 198
...
}

The overall form is:

/* surface i */ { <material> /* numVerts = */ <number of vertices>  /* numIndexes = */ <number of indices>
<vertex 0> <vertex 1> ... <vertex k>
<index 0> <index 1> ... <index m>
}

where <material> is a quoted material path-name defined in a core or FM-specific .mtr file. The <number of vertices> and <number of indices> are counts of the respective number of items that follow. The actual lists of vertices and indices as output will contain additional line breaks, after every 6 vertices and every 18 indices, which may impede understanding.

A <vertex ...> here has the format:

( x y z u v nx ny nz )

where all values are double precision floating point and

• x, y and z locates the vertex in the world coordinate system.
• u and v are the texture coordinates, used for mapping the texture maps onto this surface.
• nx, ny and nz represent the surface normal at the vertex location.

(In what comes next, it will help understanding to imagine that there is a line break after every <vertex i>, and that it is preceded by a comment indicating its vertex number, e.g.:

 /* vertex 0 */ ( -8 -8 -8 0.027777778 1 -1 0 0 )
 /* vertex 1 */ ( -8 8 -8 0.013888889 1 -1 0 0 )
 /* vertex 2 */ ( -8 8 8 0.013888889 0.9861111641 -1 0 0 ) 
 /* vertex 3 */ ( -8 -8 8 0.027777778 0.9861111641 -1 0 0 )

)

Finally, an <index ...> is simply an unsigned integer that succinctly indicates a vertex in the surface’s vertex list. Every three consecutive indices denote a triangle.

Inter Area Portals (basically visPortals)

An Inter Area Portal (IAP) is summary information about a visPortal (or occasionally, a portion of a fragmented visPortal). As the name suggests, it defines both a separation between two Areas and the location of the visibility connection between them. IAPs are used for occlusion culling, wherein parts of the FM unseen in the current field of view are left unrendered. See Visportals for more.

Only a single instance of this listing appears in a PROC file. Here’s a few lines of an example:

interAreaPortals { /* numAreas = */ 100 /* numIAP = */ 139
/* interAreaPortal format is: numPoints positiveSideArea negativeSideArea ( point) ... */
/* iap 0 */ 4 1 2 ( 2382 2558 189 ) ( 3226.125 2558 189 ) ( 3226.125 521.5 189 ) ( 2382 521.5 189 ) 
/* iap 1 */ 4 1 3 ( 1846 513 189 ) ( 3228.125 513 189 ) ( 3228.125 -8.5 189 ) ( 1846 -8.5 189 ) 
....
}

In the header, numAreas is the total count of Area-models defined earlier in the file, and numIAPs is the count of portals, each represented by a line of data. For the latter, as the comment indicates, the format is:

<number of points> <positive side area> <negative side area> <point 0> <point 1> ....

where

• <number of points> is a count of the IAP's vertices (that together make up its "winding"). Four is the typical number.
• <positive side area> and <negative side area> are integers referencing the defined Area-models separated by the IAP, e.g., "0" refers to "_area0".
• A <point> is a vertex location in world space, namely, 3 double precision floating point values of format ( x y z ). All the points of a winding appear on one line.

Nodes of the BSP Tree

In the PROC file, the BSP tree structure is defined by a list of nodes. Here’s a partial example:

nodes { /* numNodes = */ 19477
/* node format is: ( planeVector ) positiveChild negativeChild */
/* a child number of 0 is an opaque, solid area */
/* negative child numbers are areas: (-1-child) */
/* node 0 */ ( 1 0 0 0 ) 1 13426
/* node 1 */ ( 1 0 0 -3072 ) 2 110
/* node 2 */ ( 1 0 0 -5120 ) 3 18
...
}

where numNodes is a count of nodes in the BSP tree, each on its own line following. As the comment indicates, a node contains information about its splitting plane and two indices which points to either a child node or a leaf. A leaf is either a reference to an Area or to an "opaque, solid" inaccessible location.

Formally, a node is:

( nx ny nz d ) <positive child> <negative child>

The values of nx , ny and nz (each in range -1 to 1) represent the plane normal (with vector length 1) of the splitting plane. The value of d is the distance of the plane from the world origin. These four double precision floating point values define the plane's position in space.

<positive child> and <negative child> are implemented as integers. If an integer is:

• positive, it's an index pointing to another node in the node list.
• zero, it's a tree leaf with a solid, opaque location.
• negative, it's a tree leaf that references an Area-model. For this case, the actual index is computed as (-1-index). So, for instance, a value of "-6" would refer to "_area5".

Shadow Models – Background about Shadow Volumes

Briefly, to support stencil shadows, a shadow-causing light source shining on a shadow-casting object projects a "shadow volume" on the other side. The projected surfaces of that volume are determined by the object’s "silhouette", the set of vertices at the transition from triangles that face towards the light and away. Also, some shadow volumes must be closed for algorithmic reasons. A closed shadow volume has both:

• a "front cap" (nearest the light source, and typically derived from the silhouette.)
• a "rear cap" (e.g., at the end of a light's range or at some "infinity" set by a system max)

Once created, a shadow volume can be treated as a model object, independent of the object that created it. In some cases, for a particular light source, the shadow volumes of multiple objects overlap, and can be merged. Precalculation of a shadow volume is not always possible, e.g., when either the light source or object is mobile.

Shadow Models - Format for Shadow Volumes

A series of "shadowModel" sections occur, organized by light source, with name of form "_prelight_<name of light>". An example:

shadowModel { /* name = */ "_prelight_light_4"
/* numVerts = */ 4228 /* noCaps = */ 2820 /* noFrontCaps = */ 5250 /* numIndexes = */ 7680 /* planeBits = */ 63
( 479 1841 256 ) ( 516 1815.0402832031 302.1007995605 ) ( 479 1841 270.9410400391 ) ( 516 1815.0402832031 321.5000610352 ) ( 479 1841 256 ) 
( 516 1815.0402832031 302.1007995605 ) ( 468.0658874512 1841 256 ) ( 516 1804.1162109375 321.5000610352 ) ( 479 1841 256 ) ( 516 1815.0402832031 302.1007995605 )
...
32 24 25 32 25 33 758 760 759 760 761 759 762 764 765 762 765 763 
766 768 767 768 769 767 770 772 773 770 773 771 24 42 25 42 43 25 
774 6 777 774 777 775 778 780 779 780 781 779 94 96 95 96 97 95
...

The first header field, "numVerts", gives the number of vertices in vertex list (i.e., the first listing to follow). Each vertex is of form "(x y z)", with an output maximum of 5 vertices per line. (This is similar to the earlier "model" vertex lists, but without the normal or UV information.)

Then come 3 header fields that indicate counts within the indices list, the second listing to follow. As with the models earlier, each index is an integer number pointing to a vertex in the vertex list, and every 3 indices define a triangle of the shadow volume’s surface. Output has a maximum of 18 per line. There are 3 subgroups, which are all run together in the listing, but appear in order:

1. Shadow volumes with no caps, i.e., both front and rear caps omitted.
2. Shadow volumes with front caps omitted.
3. Shadow volumes with front and rear caps.

The border between these is defined by header fields, which are unobviously cumulative:

1. noCaps. Count of Indices with no caps;
2. noFrontCaps. noCaps + count of indices with no front caps.
3. numIndexes. noCaps + noFrontCaps + count of indices with both caps (= total count)

The remaining header field is "planeBits", referring to Shadow Cap Plane Bits. Consider a light’s axis-aligned range box. Some of the range-limit planes may have shadow volume triangles projected onto them. PlaneBits indicates which planes:

Bit 0 = min x; 1 = max x; 2 = min y; 3 = max y; 4 = min z; 5 = max z

PlaneBits is a shortcut to help decide if drawing a rear cap is necessary, given the player’s location.

See Also

About the PROC File Format

Regrettably, the creators of the PROC format never released an official description of it. As mentioned at the outset, Danile Gibson’s 2014 piece on the PROC File Format was the basis for much of this page.

Also helpful was the DMAP section of Fabine Sanglard’s 2012 Doom 3 Source Code Review. This extensive overview is still largely pertinent to TDM engine code today.

Related TDM forum postings about PROC include:

• OrbWeaver’s in 2007.
• SteveL’s in 2015

About BSP

Specifics of how TDM constructs and uses BSP is beyond the scope here; see the reference to Fabien Sanglard Doom 3 Review above.

Among online resources:

• Wikipedia’s "Binary Space Partitioning" article covers the topic well and has a step-by-step example.
• Another thoughtful overview are these slides of a BSP Tutorial (within a PDF). See particularly the "Solid Leaf BSP" section.

Among offline resources:

• General information about BSP is covered in gaming-related CS textbooks, for example, Foley and van Dam, "Computer Graphics: Principles & Practice", now in 3rd edition.
• Fabien Sanglard's book "Game Engine Black Book: DOOM v1.1", (c) 2018, reveals the technology development at id Studios behind early Doom. Available through online vendors as print-on-demand or eBook, ISBN 9781099819773. Among highlights:
  • BSP is explained with diagrams in Chapter 5.
  • The Doom engine initially, before BSP, used simple raycasting inherited from Wolfenstein 3D. This would bog down in one of the more complex Mission 2 rooms. Concurrently, a contract with Nintendo to port Wolfenstein 3D to SNES was in trouble, because that console was even less powerful than the PC. John Carmack, aware of Bell Lab's Bruce Naylor and his presentations about BSP, had him visit and received his reprints. Subsequently, BSP was programmed by id Studios for the (technically easier) Wolfenstein 3D port first, then transferred into the Doom engine.
  • Aside: This book also sheds light on the motivations, skills, and personalities at id Studio at the time. As such, it can be seen as complementary to "Masters of Doom" by David Kushner (Random House, (c) 2003, ISBN 978-0-8129-7215-3), that has less-technical but more sweeping and insightful biographies of John Carmack, John Romero, and other contributors.

About Stencil Shadows

Wikipedia’s article about Shadow volume informed the discussion here, and has considerably more about stencil shadow algorithms, including Doom 3 contributions.

The TDM 2.10 source code and its comments revealed some aspects, e.g., about planeBits.

For a helpful explanation and diagrams of how stencil shadows works (in an OpenGL context), including use of a shadow volume and stencil buffer to define stencil holes, see:

• Tutorial 39: Silhouette Detection
• Tutorial 40: Stencil Shadow Volume

See Shadow within Performance: Essential Must-Knows for TDM shadow volume diagnostic visualization.

Mappers: As described here, you can turn off loading of the prelight/shadow volumes with r_useOptimizedShadows 0. Better do reloadModels after you change this cvar. Subsequently, stencil shadows should recompute without dmap.