Font Conversion & Repair: Difference between revisions

From The DarkMod Wiki
Jump to navigationJump to search
No edit summary
 
(21 intermediate revisions by 4 users not shown)
Line 1: Line 1:
''written by Fidcal''


These are some rough notes about converting ttf fonts to doom3 and repairing if necessary.


There are two tools which are archived on ftp in case they disappear from the internet download sites I got them:
__TOC__
== Introduction ==
Suppose you wished to add a new font to TDM. You could build it either from scratch, or by modifying an existing font. That building could be done with direct metrics+bitmap editing.


* [http://bloodgate.com/mirrors/tdm/pub/DarkModFonts.zip]ExportFontToDoom
Alternatively, you could start with a new or existing TrueType Font. For fresh creation or modification within the TrueType world, a font editor (e.g., FontForge) would be needed.
* Q3Font
Once the .ttf file is in hand, conversion tools create a draft of the required metrics+bitmap files.


My understanding is that ExportFontToDoom is more suitable than Q3Font for converting. It is what I used to convert a lot of ttf fonts. See notes with that tool as to how use it. But be aware of copyright issues if converting ttf fonts.
With either approach, you should expect font completion to have a "long tail" of further "repair", particularly for non-ASCII characters.
As of 2024, there are four tools of note:


With ExportFontToDoom there were quite a few spacing errors. Q3Font I found a bit puzzling so never did any conversions (it may be better for all I know.) BUT it can convert the dat files to readables for editing and back again so I used Q3Font to fix the errors.
=== For conversions ===
...from a TrueType font to TDM bitmaps (in .tga/.dds files) and metrics (in .dat binary files):
* [[ExportFontToDoom3]]
* [[Q3Font]]


Doom fonts have 3 dat files:
=== For repair of TDM metrics ===
* [[Refont]] - New in 2024
* [[Q3Font]]
* [[Font Patcher]]


* fontImage_12.dat
Conversion and repair tools are briefly described further below. These are used in concert with bitmap editors (e.g., Gimp). Also, TGA-to-DDS image conversion may be done with the bitmap editor, or with separate tools.
* fontImage_24.dat
* fontImage_48.dat


But when repairing I found '24' was the only one being used by our readables generally so I never bothered with '12' and '48'.
== Fidcal's Original Take ==
These are some rough notes ''[from 2009]'' about converting ttf fonts to doom3 and repairing if necessary.


Copy the dat file(s) to the q3font folder and use this syntax to get a readable .fnt file:
My understanding is that ''ExportFontToDoom3'' is more suitable than Q3Font for converting. It is what I used to convert a lot of TTF fonts. See notes with that tool as to how use it. But be aware of copyright issues if converting TTF fonts.


q3font.exe -decompile fontImage_24.dat
With ''ExportFontToDoom3'' there were quite a few spacing errors. ''Q3Font'' I found a bit puzzling so never did any conversions (it may be better for all I know.) '''BUT''' it can convert the dat files to readables for editing and back again so I used Q3Font to fix the errors.


A common problem is this:
''Geep adds: Check out newer Font Patcher and Refont to fix errors too.''


Characters have an actual width but they also have a distance value from where they start being 'written' to the start of the next letter (xskip in the readable .fnt files.)
Doom fonts have multiple dat files for each size. However, only the following are loaded by the code:


The true type font to doom font convertor named ExportFontToDoom3 appears to calculate this based on the character width (my guess.)
* fontImage_12.dat
* fontImage_24.dat
* fontImage_48.dat


If lower case letters have descenders that curl back under the previous letter on the line, eg, f, g, j, p, y (q is OK because it curls forward) then the actual width of the character is greater than the distance from its start point to the start of the next character.
The size 24 is used mostly by readables, while the menu uses 48 (the big headlines and the main menu) and 24 (settings and other headlines).
This means it is not necessary to create other sizes, as these will be ignored, anyway.


So, ExportFontToDoom3 makes xskip too big in the case of the above letters.
''Geep adds: see [[Font Files]] for a fuller, more recent treatment about font file usage. For more details about Fidcal's 2009 improvements to TDM fonts, see [https://forums.thedarkmod.com/index.php?/topic/8265-found-a-way-to-repair-fonts-at-last/#comment-163449 this forum thread].''
Reducing xskip in the .fnt files and restoring them to .dat files cures the problem.


In the readable .fnt files the characters are ASCII numbers.
== Main Tools ==
A to Z are 65 to 90
* [[ExportFontToDoom3]] is, as Fidcal says, the recommended method, if you want to start a Doom3/TDM font project by generating from a TrueType font.
a to z are 97 to 122
* [[Q3Font]] is the traditional Windows utility program to fix Doom3/TDM fonts. Fidcal provides a simple example of using it. (Some of this info used to be here in Font Conversion & Repair; some is an update.)
f = 102
* [[Refont]] is another command-line program, new in 2024, that handles some of Q3font's functionality, but is easier to use. This article includes a detailed comparison.
g = 103
* [[Font Patcher]] details the font_patcher.pl utility, which is a preferred method (over Q3font) to fix and change fonts. Includes links to examples.
j = 106
p = 112
y = 121


Search the net for an ascii code list for others.
== Cautionary Considerations ==
=== Usually convert only once ===
If a font was converted from TrueType and subsequently received tweaks and repairs to its DAT or DDS files (as is very common), then you never want to apply conversion again, because that will throw away all those time-consuming customizations.
=== When changing font metrics in the DAT File ===
General constraints on the values of font metrics in a DAT file are discussed in [[Font Metrics & DAT File Format]].
=== Additional constraints on changing metrics of long-deployed fonts ===
Consider a particular TDM font that has been in use for a while, and likely incorporated into, say, the readables of many FMs. Suppose you'd like to improve a particular character within it, say, by adjusting its DAT metrics. Maybe the glyph is showing stray marks from adjoining characters in its bitmap.


They fnt file can be converted back to .dat files using...
It's generally safe to move the bitmap cell boundaries horizontally a pixel at a time, by changing x-axis corner metrics s & s2 together, keeping imageWidth unchanged. When you do that, generally the "pitch" value (which denotes an xOffset distance from the s edge to the layout origin) needs compensating adjustment, to keep the layout origin the same with respect to the glyph. (Note: if the glyph is abutted against the overall bitmap side boundary, this approach may not be viable; the alternative of bitmap surgery may be needed.)


q3font.exe fontImage_24
More problematic would be changing the xSkip value that controls a character’s horizontal spacing. Why? A readable’s author may have designed a particular text layout and expect word-wraps at particular place. An enlarged xSkip may force the end of a readable’s paragraph off the page. Even a shortened xSkip might interfere, e.g., with "ASCII art", tabular presentation, or bullet-point-style indentations that rely on word-wrap within a paragraph instead of separated sentences. So, either forego such a change, or be prepared for difficult research into where and how the candidate font character is used across FMs, including in translation strings.


You will need to relaunch Doom each adjustment as there is no quick way to reload fonts.
A change to ImageWidth in theory has similar concerns, but in the more limited context of affecting only the first character rendered in any given line. So shrinking it is far less likely to be a problem in practice.


Example: Say you want to reduce the space after letter j
== For More ==
* q3font.exe -decompile fontImage_24.dat to produce a readable .fnt file
* [[I18N]] (Internationalization) introduces consideration for non-English languages.
* In the .fnt file search down for the character header, eg, char 106.
* [[I18N - Charset]] details the TDM-specific character mapping for non-English languages. For instance, if you convert a Russian font, you need to move the character 0xFF to 0xB6, otherwise it will not show in the GUI or readables.
* Within the curly brackets below the header are the various values.
* Look for xSkip.
* Say it has a value of 12. Try reducing it to 9 to reduce the offset.
* Convert back to .dat format, reload Dark Mod and check in-game.


[[Category:Tutorial]]
{{tutorial}} [[Category:Fonts]]

Latest revision as of 15:28, 18 March 2024


Introduction

Suppose you wished to add a new font to TDM. You could build it either from scratch, or by modifying an existing font. That building could be done with direct metrics+bitmap editing.

Alternatively, you could start with a new or existing TrueType Font. For fresh creation or modification within the TrueType world, a font editor (e.g., FontForge) would be needed. Once the .ttf file is in hand, conversion tools create a draft of the required metrics+bitmap files.

With either approach, you should expect font completion to have a "long tail" of further "repair", particularly for non-ASCII characters. As of 2024, there are four tools of note:

For conversions

...from a TrueType font to TDM bitmaps (in .tga/.dds files) and metrics (in .dat binary files):

For repair of TDM metrics

Conversion and repair tools are briefly described further below. These are used in concert with bitmap editors (e.g., Gimp). Also, TGA-to-DDS image conversion may be done with the bitmap editor, or with separate tools.

Fidcal's Original Take

These are some rough notes [from 2009] about converting ttf fonts to doom3 and repairing if necessary.

My understanding is that ExportFontToDoom3 is more suitable than Q3Font for converting. It is what I used to convert a lot of TTF fonts. See notes with that tool as to how use it. But be aware of copyright issues if converting TTF fonts.

With ExportFontToDoom3 there were quite a few spacing errors. Q3Font I found a bit puzzling so never did any conversions (it may be better for all I know.) BUT it can convert the dat files to readables for editing and back again so I used Q3Font to fix the errors.

Geep adds: Check out newer Font Patcher and Refont to fix errors too.

Doom fonts have multiple dat files for each size. However, only the following are loaded by the code:

  • fontImage_12.dat
  • fontImage_24.dat
  • fontImage_48.dat

The size 24 is used mostly by readables, while the menu uses 48 (the big headlines and the main menu) and 24 (settings and other headlines). This means it is not necessary to create other sizes, as these will be ignored, anyway.

Geep adds: see Font Files for a fuller, more recent treatment about font file usage. For more details about Fidcal's 2009 improvements to TDM fonts, see this forum thread.

Main Tools

  • ExportFontToDoom3 is, as Fidcal says, the recommended method, if you want to start a Doom3/TDM font project by generating from a TrueType font.
  • Q3Font is the traditional Windows utility program to fix Doom3/TDM fonts. Fidcal provides a simple example of using it. (Some of this info used to be here in Font Conversion & Repair; some is an update.)
  • Refont is another command-line program, new in 2024, that handles some of Q3font's functionality, but is easier to use. This article includes a detailed comparison.
  • Font Patcher details the font_patcher.pl utility, which is a preferred method (over Q3font) to fix and change fonts. Includes links to examples.

Cautionary Considerations

Usually convert only once

If a font was converted from TrueType and subsequently received tweaks and repairs to its DAT or DDS files (as is very common), then you never want to apply conversion again, because that will throw away all those time-consuming customizations.

When changing font metrics in the DAT File

General constraints on the values of font metrics in a DAT file are discussed in Font Metrics & DAT File Format.

Additional constraints on changing metrics of long-deployed fonts

Consider a particular TDM font that has been in use for a while, and likely incorporated into, say, the readables of many FMs. Suppose you'd like to improve a particular character within it, say, by adjusting its DAT metrics. Maybe the glyph is showing stray marks from adjoining characters in its bitmap.

It's generally safe to move the bitmap cell boundaries horizontally a pixel at a time, by changing x-axis corner metrics s & s2 together, keeping imageWidth unchanged. When you do that, generally the "pitch" value (which denotes an xOffset distance from the s edge to the layout origin) needs compensating adjustment, to keep the layout origin the same with respect to the glyph. (Note: if the glyph is abutted against the overall bitmap side boundary, this approach may not be viable; the alternative of bitmap surgery may be needed.)

More problematic would be changing the xSkip value that controls a character’s horizontal spacing. Why? A readable’s author may have designed a particular text layout and expect word-wraps at particular place. An enlarged xSkip may force the end of a readable’s paragraph off the page. Even a shortened xSkip might interfere, e.g., with "ASCII art", tabular presentation, or bullet-point-style indentations that rely on word-wrap within a paragraph instead of separated sentences. So, either forego such a change, or be prepared for difficult research into where and how the candidate font character is used across FMs, including in translation strings.

A change to ImageWidth in theory has similar concerns, but in the more limited context of affecting only the first character rendered in any given line. So shrinking it is far less likely to be a problem in practice.

For More

  • I18N (Internationalization) introduces consideration for non-English languages.
  • I18N - Charset details the TDM-specific character mapping for non-English languages. For instance, if you convert a Russian font, you need to move the character 0xFF to 0xB6, otherwise it will not show in the GUI or readables.