Font Conversion & Repair: Difference between revisions

From The DarkMod Wiki
Jump to navigationJump to search
No edit summary
Geep (talk | contribs)
 
(24 intermediate revisions by 5 users not shown)
Line 1: Line 1:
''written by Fidcal''


These are some rough notes about converting ttf fonts to doom3 and repairing if necessary.


There are two tools which are archived on ftp in case they disappear from the internet download sites I got them:
__TOC__
== Introduction ==
Suppose you wished to add a new font to TDM. You could build it either from scratch, or by modifying an existing font. That building could be done with direct metrics+bitmap editing.


* ExportFontToDoom
Alternatively, you could start with a new or existing TrueType Font. For fresh creation or modification within the TrueType world, a font editor (e.g., FontForge) would be needed.
* Q3Font
Once the .ttf file is in hand, conversion tools create a draft of the required metrics+bitmap files.


My understanding is that ExportFontToDoom is more suitable than Q3Font for converting. It is what I used to convert a lot of ttf fonts. See notes with that tool as to how use it. But be aware of copyright issues if converting ttf fonts.
With either approach, you should expect font completion to have a "long tail" of further "repair", particularly for non-ASCII characters.
As of 2024, there are four tools of note:


With ExportFontToDoom there were quite a few spacing errors. Q3Font I found a bit puzzling so never did any conversions (it may be better for all I know.) BUT it can convert the dat files to readables for editing and back again so I used Q3Font to fix the errors.
=== For conversions ===
...from a TrueType font to TDM bitmaps (in .tga/.dds files) and metrics (in .dat binary files):
* [[ExportFontToDoom3]]
* [[Q3Font]]


Doom fonts have 3 dat files:
=== For repair of TDM metrics ===
* [[Refont]] - New in 2024
* [[Q3Font]]
* [[Font Patcher]]


* fontImage_12.dat
Conversion and repair tools are briefly described further below. These are used in concert with bitmap editors (e.g., Gimp). Also, TGA-to-DDS image conversion may be done with the bitmap editor, or with separate tools.
* fontImage_24.dat
 
* fontImage_48.dat
== Fidcal's Original Take ==
These are some rough notes ''[from 2009]'' about converting ttf fonts to doom3 and repairing if necessary.


But when repairing I found '24' was the only one being used by our readables generally so I never bothered with '12' and '48'.
My understanding is that ''ExportFontToDoom3'' is more suitable than Q3Font for converting. It is what I used to convert a lot of TTF fonts. See notes with that tool as to how use it. But be aware of copyright issues if converting TTF fonts.


Copy the dat file(s) to the q3font folder and use this syntax to get a readable .fnt file:
With ''ExportFontToDoom3'' there were quite a few spacing errors. ''Q3Font'' I found a bit puzzling so never did any conversions (it may be better for all I know.) '''BUT''' it can convert the dat files to readables for editing and back again so I used Q3Font to fix the errors.


q3font.exe -decompile fontImage_24.dat
''Geep adds: Check out newer Font Patcher and Refont to fix errors too.''


A common problem is this:
Doom fonts have multiple dat files for each size. However, only the following are loaded by the code:


Characters have an actual width but they also have a distance value from where they start being 'written' to the start of the next letter (xskip in the readable .fnt files.)
* fontImage_12.dat
* fontImage_24.dat
* fontImage_48.dat


The true type font to doom font convertor named ExportFontToDoom3 appears to calculate this based on the character width (my guess.)
The size 24 is used mostly by readables, while the menu uses 48 (the big headlines and the main menu) and 24 (settings and other headlines).
This means it is not necessary to create other sizes, as these will be ignored, anyway.


If lower case letters have descenders that curl back under the previous letter on the line, eg, f, g, j, p, y (q is OK because it curls forward) then the actual width of the character is greater than the distance from its start point to the start of the next character.
''Geep adds: see [[Font Files]] for a fuller, more recent treatment about font file usage. For more details about Fidcal's 2009 improvements to TDM fonts, see [https://forums.thedarkmod.com/index.php?/topic/8265-found-a-way-to-repair-fonts-at-last/#comment-163449 this forum thread].''


So, ExportFontToDoom3 makes xskip too big in the case of the above letters.
== Main Tools ==
Reducing xskip in the .fnt files and restoring them to .dat files cures the problem.
* [[ExportFontToDoom3]] is, as Fidcal says, the recommended method, if you want to start a Doom3/TDM font project by generating from a TrueType font.
* [[Q3Font]] is the traditional Windows utility program to fix Doom3/TDM fonts. Fidcal provides a simple example of using it. (Some of this info used to be here in Font Conversion & Repair; some is an update.)
* [[Refont]] is another command-line program, new in 2024, that handles some of Q3font's functionality, but is easier to use. This article includes a detailed comparison.
* [[Font Patcher]] details the font_patcher.pl utility, which is a preferred method (over Q3font) to fix and change fonts. Includes links to examples.


In the readable .fnt files the characters are ASCII numbers.
== Cautionary Considerations ==
A to Z are 65 to 90
=== Usually convert only once ===
a to z are 97 to 122
If a font was converted from TrueType and subsequently received tweaks and repairs to its DAT or DDS files (as is very common), then you never want to apply conversion again, because that will throw away all those time-consuming customizations.
f = 102
=== When changing font metrics in the DAT File ===
g = 103
General constraints on the values of font metrics in a DAT file are discussed in [[Font Metrics & DAT File Format]].
j = 106
=== Additional constraints on changing metrics of long-deployed fonts ===  
p = 112
Consider a particular TDM font that has been in use for a while, and likely incorporated into, say, the readables of many FMs. Suppose you'd like to improve a particular character within it, say, by adjusting its DAT metrics. Maybe the glyph is showing stray marks from adjoining characters in its bitmap.
y = 121


Search the net for an ascii code list for others.
It's generally safe to move the bitmap cell boundaries horizontally a pixel at a time, by changing x-axis corner metrics s & s2 together, keeping imageWidth unchanged. When you do that, generally the "pitch" value (which denotes an xOffset distance from the s edge to the layout origin) needs compensating adjustment, to keep the layout origin the same with respect to the glyph. (Note: if the glyph is abutted against the overall bitmap side boundary, this approach may not be viable; the alternative of bitmap surgery may be needed.)


They fnt file can be converted back to .dat files using...
More problematic would be changing the xSkip value that controls a character’s horizontal spacing. Why? A readable’s author may have designed a particular text layout and expect word-wraps at particular place. An enlarged xSkip may force the end of a readable’s paragraph off the page. Even a shortened xSkip might interfere, e.g., with "ASCII art", tabular presentation, or bullet-point-style indentations that rely on word-wrap within a paragraph instead of separated sentences. So, either forego such a change, or be prepared for difficult research into where and how the candidate font character is used across FMs, including in translation strings.


q3font.exe fontImage_24
A change to ImageWidth in theory has similar concerns, but in the more limited context of affecting only the first character rendered in any given line. So shrinking it is far less likely to be a problem in practice.


You will need to relaunch Doom each adjustment as there is no quick way to reload fonts.
== For More ==
* [[I18N]] (Internationalization) introduces consideration for non-English languages.
* [[I18N - Charset]] details the TDM-specific character mapping for non-English languages. For instance, if you convert a Russian font, you need to move the character 0xFF to 0xB6, otherwise it will not show in the GUI or readables.


[[Category:Tutorial]]
{{tutorial}} [[Category:Fonts]]

Latest revision as of 15:28, 18 March 2024


Introduction

Suppose you wished to add a new font to TDM. You could build it either from scratch, or by modifying an existing font. That building could be done with direct metrics+bitmap editing.

Alternatively, you could start with a new or existing TrueType Font. For fresh creation or modification within the TrueType world, a font editor (e.g., FontForge) would be needed. Once the .ttf file is in hand, conversion tools create a draft of the required metrics+bitmap files.

With either approach, you should expect font completion to have a "long tail" of further "repair", particularly for non-ASCII characters. As of 2024, there are four tools of note:

For conversions

...from a TrueType font to TDM bitmaps (in .tga/.dds files) and metrics (in .dat binary files):

For repair of TDM metrics

Conversion and repair tools are briefly described further below. These are used in concert with bitmap editors (e.g., Gimp). Also, TGA-to-DDS image conversion may be done with the bitmap editor, or with separate tools.

Fidcal's Original Take

These are some rough notes [from 2009] about converting ttf fonts to doom3 and repairing if necessary.

My understanding is that ExportFontToDoom3 is more suitable than Q3Font for converting. It is what I used to convert a lot of TTF fonts. See notes with that tool as to how use it. But be aware of copyright issues if converting TTF fonts.

With ExportFontToDoom3 there were quite a few spacing errors. Q3Font I found a bit puzzling so never did any conversions (it may be better for all I know.) BUT it can convert the dat files to readables for editing and back again so I used Q3Font to fix the errors.

Geep adds: Check out newer Font Patcher and Refont to fix errors too.

Doom fonts have multiple dat files for each size. However, only the following are loaded by the code:

  • fontImage_12.dat
  • fontImage_24.dat
  • fontImage_48.dat

The size 24 is used mostly by readables, while the menu uses 48 (the big headlines and the main menu) and 24 (settings and other headlines). This means it is not necessary to create other sizes, as these will be ignored, anyway.

Geep adds: see Font Files for a fuller, more recent treatment about font file usage. For more details about Fidcal's 2009 improvements to TDM fonts, see this forum thread.

Main Tools

  • ExportFontToDoom3 is, as Fidcal says, the recommended method, if you want to start a Doom3/TDM font project by generating from a TrueType font.
  • Q3Font is the traditional Windows utility program to fix Doom3/TDM fonts. Fidcal provides a simple example of using it. (Some of this info used to be here in Font Conversion & Repair; some is an update.)
  • Refont is another command-line program, new in 2024, that handles some of Q3font's functionality, but is easier to use. This article includes a detailed comparison.
  • Font Patcher details the font_patcher.pl utility, which is a preferred method (over Q3font) to fix and change fonts. Includes links to examples.

Cautionary Considerations

Usually convert only once

If a font was converted from TrueType and subsequently received tweaks and repairs to its DAT or DDS files (as is very common), then you never want to apply conversion again, because that will throw away all those time-consuming customizations.

When changing font metrics in the DAT File

General constraints on the values of font metrics in a DAT file are discussed in Font Metrics & DAT File Format.

Additional constraints on changing metrics of long-deployed fonts

Consider a particular TDM font that has been in use for a while, and likely incorporated into, say, the readables of many FMs. Suppose you'd like to improve a particular character within it, say, by adjusting its DAT metrics. Maybe the glyph is showing stray marks from adjoining characters in its bitmap.

It's generally safe to move the bitmap cell boundaries horizontally a pixel at a time, by changing x-axis corner metrics s & s2 together, keeping imageWidth unchanged. When you do that, generally the "pitch" value (which denotes an xOffset distance from the s edge to the layout origin) needs compensating adjustment, to keep the layout origin the same with respect to the glyph. (Note: if the glyph is abutted against the overall bitmap side boundary, this approach may not be viable; the alternative of bitmap surgery may be needed.)

More problematic would be changing the xSkip value that controls a character’s horizontal spacing. Why? A readable’s author may have designed a particular text layout and expect word-wraps at particular place. An enlarged xSkip may force the end of a readable’s paragraph off the page. Even a shortened xSkip might interfere, e.g., with "ASCII art", tabular presentation, or bullet-point-style indentations that rely on word-wrap within a paragraph instead of separated sentences. So, either forego such a change, or be prepared for difficult research into where and how the candidate font character is used across FMs, including in translation strings.

A change to ImageWidth in theory has similar concerns, but in the more limited context of affecting only the first character rendered in any given line. So shrinking it is far less likely to be a problem in practice.

For More

  • I18N (Internationalization) introduces consideration for non-English languages.
  • I18N - Charset details the TDM-specific character mapping for non-English languages. For instance, if you convert a Russian font, you need to move the character 0xFF to 0xB6, otherwise it will not show in the GUI or readables.