Coding in the SDK

...also known as greebo's random notes about coding for The Dark Mod.

This article is constant WIP. When coding in the SDK, there are thousands of little lessons to be learned - some of them include things I wish I would have known earlier, some might be common knowledge. Anyway, I plan include all the miscellaneous things in this article.

General

When the id people designed Doom 3, they roughly divided their code into two parts: the engine and the gameplay code. The engine code is closed source, i.e. we don't have the code, only the compiled binary in the form of DOOM3.exe.

The engine contains all the "important" stuff which id software is making money with, by licensing it to third party companies. This is where the real brainpower is in, so we don't get that part (yet - John Carmack repeatedly stated that he wants to release all the source under the GPL soon).

The gameplay code is fully available. This includes the entities, game logic, AI, physics, weapon and animation code. This leaves a whole lot of headroom for modders like you and me, and we probably should thank id software for that. There are all kinds of mods out there which change the gameplay code to create their own game so to say. Some of them are "mini-mods" (introducing a new weapon or double-jump-mods), some of them are larger, aiming to change the nature of the game and finally there's The Dark Mod, which changed almost everything that can be changed in the first place. The term Total Conversion comes close - when playing TDM, you won't believe it's the same game as Doom 3.

What parts of the code are inaccessible?

All of the code id are earning money with by licensing their engine to others. I don't know all the parts, but these are the ones from the top of my head:

• the rendering engine
• the sound engine
• the collision model manager
• the Map and AAS compilers
• the declaration manager (although we can implement our own parsers)
• the network system
• the keyboard and mouse handlers
• anything platform-specific
• the command system
• the virtual file system
• the CVAR system plus some special CVARs
• the editor (D3Ed), but we have DarkRadiant

What parts of the code are left for us to change?

A whole lot, and in most cases it is sufficient:

• gameplay code (entities, logic)
• physics
• AI
• custom declarations
• custom CVARs
• the scripting system
• the animation code
• plus everything outside the SDK like scripts, defs, materials and models.

Coding Style

This deserves its own article: TDM Coding Style

It's easy to see that the gameplay code has been written by a number of different programmers at id software. It's also easy to see that several of them have strong roots in C, whereas others have used C++ coding paradigms more strongly. At any rate, almost all of those programmers need a heads up that the possibility of placing comments in the code has not been removed from C++ - only a few of them seem to know about comments at all, which is a shame, especially when it comes to the animation code.

Another thing that a newcomer might notice is the mere absence of STL or other standard stuff in the id code. I'm not in the position of judging whether this is actually necessary (which I doubt), but it seems that id rewrote every little low-level library on their own, including strings, lists, hashtables, vectors, etc. Some of the classes are admittedly quite handy, but sometimes it's clear that they did it just because they could. Overall, there is strong NIH smell all about the place, but the code is there and stable already, so we might as well use it for TDM coding.

Something to keep in mind is also the custom memory manager, which (according the comments in the code) is multiple times faster than the one provided by the CRT. As a consequence of this, everything that is allocated in the game DLL must be freed again before the DLL is unloaded, otherwise you'll be running into weird crashes at shutdown. Most classes have Clear() methods in them, and whenever you write custom (global or standalone) classes you need to make sure that these are destroyed in idGameLocal::Shutdown() at the latest.

id software has been using the following coding style:

• class member variables are lowercase, plain notation (without the m_bVar hungarian stuff)
• class names start with the "id" prefix
• class methods are uppercase with mixed case, like this: idDict::MatchPrefix()
• One True Brace Style, i.e. the opening { brace is in the same line as the if ().

STL vs. idLib

The most common idLib member is the idList<> template, which is the counter-part of std::vector<>. Whenever you need a variable-sized array, use idList. This class has some nice convenience functions and supports the definition of granularity, which allows you to specify the minimum size of memory blocks which are allocated by idList. This is useful to prevent idList from calling the new operator each time you push an element into that list.

Another popular member is idStr. This is mostly equivalent with std::string. Both classes provide the well-known c_str() method which can be used to retrieve the const char* pointer from the object. Two gotchas about idStr:

• void idStr::Empty() is an imperative method, i.e. it clears the string. Do not confuse this with bool std::string::empty(), which returns true when the string is empty. If you want to check whether an idStr is empty, use idStr::IsEmpty()...
• idStr has an operator cast to const char*, so you can pass idStr to everything that expects a const char*. Where's the gotcha? Don't expect this to work when passing idStr to functions with variable-sized argument lists, like this

idStr mystr = "test";
idGameLocal::Printf("String is %s", mystr); // this will crash!

The compiler cannot know which operator cast to call for this type of function call, so use the c_str() function instead:

idStr mystr = "test";
idGameLocal::Printf("String is %s", mystr.c_str());

There are more container classes in idLib:

• idLinkList, a bit similar to std::list - I don't use that much, because it's inconvenient.
• idHashTable, ditto
• idDict - this one is useful. The most common use for this is to contain all the spawnargs of an entity.
• idKeyValue - used to define a spawnarg pair
• idStrList - just a shortcut for idList<idStr>

Another thing I had to learn the hard way was using idStr in combination with std::map. Due to the built-in operator cast to const char*, an idStr cannot be used as index in a std::map. The std::map will try to sort the given idStr into the correct place, but the comparison operator is actually invoking the idStr::operator const char*() const which traps the std::map into comparing pointers instead of strings. If you want to use std::map, use std::map<std::string, someothertype>.

Performance Considerations

The code might not win a beauty contest, but most of id's programmers know the ins and outs of how to code a performing game, no doubt about that.

When writing new code and you're worrying about performance, think about how your code is going to be used and more importantly: how often. If your code piece is called when a projectile is hitting a surface, there's no problem with that. If your code is more or less directly hooked into the idAI::Think() routine, think twice before writing slow code.

How to recognise slow code? That's a difficult question, you need to use your brain and read lots of code to get a feeling for this.

Bad Things include:

• Looping over a large amount of entities.
• Querying spawnargs multiple times each frame, converting strings to float (e.g. calling idDict::GetFloat() for every little thing). Read the spawnarg once in your Spawn() method and store it in a local member variable instead.
• Calling the clip/trace/collision code exorbitantly.

Another thing to keep in mind: even if the game performs well in your neat little developer testmap with one light and three entities, things might change when you get it to run a full-grown mission with tons of complicated lighting, geometry, moveables and AI. The 16 ms frametime which appeared to be enormously much on your 2.4 GHz developer beast will melt like ice when the rendering of the scene takes up 13 ms on its own on a 1.4 GHz low-end system, leaving a whopping 3 ms for bringing your entire level to life including physics, animation, player movement and last but not least the lightgem. There goes the FPS below 60 and counting downwards. Don't even hope that future processors will execute your code faster - they won't, given that the processor clock rate of a single core has been almost stagnating over the last few years. The SDK code is using only one core.

How to Debug

There are several techniques to find problems in your code to see why the heck your door is not playing the "close" sound, for instance.

• Use your head

That goes without saying.

• Use your debugger

Run the game in your IDE and use the breakpoints at the right places. Note that you can run both debug and release builds in your debugger - the difference of release builds is that some code lines will not be hit and not all variables are available to the inspector. For physics or animation code, the debugger is of limited use, because either the variables don't make sense to our little human brains or the code is executed just too fast and you can't tell when the bug is happening.

• Use your console

If you don't want to interrupt your game by a breakpoint hit, use the console output commands to write stuff to the console, like this:

gameLocal.Printf("the time is %d\n", gameLocal.time); // will print the game time to the console

• Use debug drawings

For things that happen really fast or mathematical things like vectors and bounding boxes use the excellent debug output features that are available for you:

// this will draw a red line from the AI's head to the player's head, lifetime is 16 msecs.
gameRenderWorld->DebugArrow(colorRed, ai->GetEyePosition(), player->GetEyePosition(), 1, 16);

You can also draw text in the game, but keep in mind that you need to pass the text orientation matrix to the function as well, otherwise the text is not oriented to the player:

// this will draw a white "test" at the "eyePosition", readable to the player, lifetime = 16 msecs
gameRenderWorld->DrawText("test", eyePosition, 0.1f, colorWhite, gameLocal.GetLocalPlayer()->viewAngles.ToMat3(), 16);

When drawing a large amount of these, the game performance will go down the drain, but you don't need to worry about that - it's debug output for your personal use only.

If you happened to write a useful piece of debug drawing output, please don't delete it right away before committing. It might pay off if you take the time and make them optional by using a CVAR, like "tdm_ai_showdebugtext", which defaults to "0". There's a good chance that other developers will want to marry you for this.

• Use the logfile

There are some debugging macros available in the TDM codebase, which you can use extensively.

// Let the logfile know that we've spawned
DM_LOG(LC_AI, LT_INFO)LOGSTRING("AI %s has spawned!\r", name.c_str());

There are several log classes and log levels available, just look at the darkmod.ini, which is also where you can switch them on or off. Examples are:

• LC_AI for AI stuff
• LC_ENTITY
• LC_LOCKPICK
• LC_OBJECTIVES
• LC_STIMRESPONSE
• ...

The loglevels can be used to filter a certain log class for specific events:

• LT_ERROR is the most severe message
• LT_WARNING
• LT_DEBUG
• LT_INFO contains verbose stuff

You can also define new log classes in DarkMod/DarkModGlobals.cpp, if it's actually necessary. Note that performance is not a real issue here, as the code can be stripped with a single #define from the entire codebase, leaving our release code nice and clean. When switched on, each call does take a small amount of time. If you really need to know (like me), one log line usually takes up 12 µsecs, which is rather much, as the log buffer is immediately flushed to disk (otherwise the log line would vanish in case of a crash).