LangPeeker

[PAGE UNDER CONSTRUCTION] by Geep, 2025

Introduction

LangPeeker is a standalone Windows utility program, to help fast deciphering of TDM localizations. You can use it to quickly lookup international-language information about specific #str_<id> strings used in an internationalized FM of interest and/or base TDM (e.g., menus). It could be helpful for a developer reviewing in DR an existing, unfamiliar FM that offers localization.

LangPeeker is a viewer, i.e., no editing. It presents 3 tab panes:

• Search
• Settings
• Coverage

When first used, you go to "Settings" and enter paths to the current FM and/or TDM installation. You may validate these by an "Apply and Test" button, which reports what language files are available to be searched. You get a brief popup report for English, plus an update to the full story for all possible languages under "Coverage".

Then at "Search", you enter a search term, with a language of immediate interest (like "English") selected. Most often, the search term is a specific #str_<id>, and you have constrained the search to just look among #str_<id> values for a match. It's easy to swap languages to see comparable results.

The results are broken into fields, with the string "Contents" field (of most interest) viewable in several forms.

Settings

When configuring LangPeeker for a new project, start with the "Settings" tab page, and fill in one or (as in Figure 1) both of the "Folder" fields.

[Figure 1]

Figure 1. "Settings" with both Folder fields filled in and the checkboxes set. After that, the "Apply and Test" button was pressed, resulting in the popup dialog shown.

If you touch the info icon to the right of the "FM Folder" field, you'll see this text:

- Typically holds full path to <tdm_installed>\fms\<fm>
- Folder should contain <fm>_l10n.pk4 and/or
  unpackaged dev project. (If both, pk4 is ignored.)
- LangPeeker will then look in /strings/fm/ for *.lang files.
- For instance, an 'English' search would consult english.lang or
  the [English] section within all.lang
- If you're just working on TDM base strings, leave this field empty,
  or set it here but skip it by unchecking Search tab's 'FM data'.

Similarly, the info icon for "TDM Folder" gives:

- Typically holds full path to <tdm_installed>
- Folder should contain either tdm_base01.pk4 or
  unpackaged equivalent. (If both, pk4 is ignored.)
- LangPeeker will then look in /strings/ for *.lang files.
- For instance, an 'English' search would consult english.lang or
  the [English] section of all.lang
- If you're just working on FM strings, leave this field empty,
  or set it here but skip it by unchecking Search tab's 'TDM data'.
- But be aware that some FMs reference TDM base strings.

Additional checkboxes are:

• "Search FM before TDM". If you specify both an FM and TDM Folder path, this controls which is searched first.
• "As a data source, prefer all.lang over <languages>.lang".

To support its menu system, the TDM base distribution offers both UTF8 file all.lang, and a full set of <language>.lang files (of various codepage encodings). These two sources are largely redundant, so, when both are present, you must choose which one to search. Considerations are:

• All.lang is considered the "master source", from which the <language>.lang files are generated.
• All.lang is already in UTF8 form, so no conversion is needed.
• Optional comments are more present in all.lang (a 2-edged sword).
• <languages>.lang is what the game actually uses (with a few characters remapped at runtime)

The same considerations could apply to an FM with localizations, but it is rare at this time for the FM's all.lang file to be distributed.

To ensure app consistency and correctness:

• Any changes to the checkboxes, and to a Folder field selected by its "Browse" dialog, will be applied immediately.
• Any change to a Folder field by editing or cut/paste will be applied either when you hit <Enter> or do "Apply and Test".

Generally, this update process will be quiet unless a problem is detected. On the other hand, using "Apply and Test" - whose 'apply' is sometimes harmlessly redundant - pops up additional information, revealing how an 'English' search would be processed. An example is shown in the Figure.

Paths and checkboxes under "Settings" are remembered across sessions (unlike constraints on the "Search" page). Each folder field offers a history of up to 5 recent paths.

Search

[Fig. 2]

Figure 2. A typical successful search, with English results, for the configuration given in Figure 1. The author of the FM's content string in this case has included some explicit linebreaks (as \n; one actually starts the string). These will be treated as linebreaks in the "Multiline..." Results pane. But one resulting line was still too long to view except with horizontal scrolling, so "Word wrap..." was momentarily turned on.

Search Term Entry and Language Choice

Suppose you are examining some aspect of an FM in DR or a text editor, and you start to encounter some #str_<id>s. You can configure LangPeeker for that FM (if not already done). Then, copy/paste or type a given #str_<id> into LangPeeker's search field, followed by <Enter>. Use the "Language" pull-down to choose among all the TDM-supported languages, keeping in mind for FMs, available translations are quite limited.

Search Checkbox Constraints

Usually, you'll want to match only against the #str_<id> field in underlying *.lang files. But you can configure your search to include the string "Contents" field. You might do that if you remember a unique phrase from playing the game, and would like to determine its #str_<id>. The results found will then vary by language. For completeness, the "Comment" field (e.g., a line in all.lang that has "//...") can also be searched.

If in Settings you established both the FM and TDM Folders, you could, for a specific search, quickly limit it to just one or the other, with the Search checkboxes "FM Data" and "TDM Base Data".

Results

For a given language, the matching text line in a .lang file is shown, broken up into 3 fields. The "within" field says which file was the source. If not found, that field will say so, and all other fields will be blank.

Coverage

[Fig. 3].

Figure 3. The "Coverage" page, for the configuration given in Figure 1. While the TDM base provides a full set of language files (e.g., for menus), this FM provides only English and German files.

Given what you configured in Settings, this page gives a quick view of what language resources are available. Two of the data columns present what you would see if you looked within the relevant file directories. The other two data columns present what you know if there was an all.lang file and you surveyed [language] headings within it.

In addition, where redundant data sources are available, symbols indicate which source will be consulted and which ignored.

Implementation Notes

Framework

This was rapid-prototyped in Visual Studio Community 2022 as a standard MS WinForms app, written in C#, and using .Net libraries.

Conversion of Data to UTF8

Search results are shown in UTF8 format. This is the native format for all.lang, so no conversion is needed for data drawn from there. Data from other <language>.lang files are not in UTF8, but in various codepages, mostly iso-8859-x, as detailed elsewhere. The facilities of modern .Net are used to convert to UTF8 after file read.

Persistence

Settings are saved as Properties, structured during development through the VS IDE, and stored in its own MS-specific format in the Properties/Settings.settings xml file.

Known Limitations

• Windows-specific. [Geep sez: Anyone else is welcome to explore possibilities for Linux. Or provide similar functionality within DR.]
• If your machine has only early versions of .Net, codepage conversions will fail.
• Because LangPeeker is oriented towards unique search terms, only the first hit is found; there is no "Next" button (or internals to handle such).
• Unlike Gen Lang Programs, LangPeeker does not do a file pre-scan to remove potentially multiline comments of the form /* ... */. This may lead to an occasional false hit.
• LangPeeker uses standardized conversions, which represent precisely what is in the .lang files, but neglects TDM-code remappings of a few characters.

One can imagine additional capabilities if there was a demand for them (with lots more work to implement), e.g.:

• visually indicate in Search's "Language" dropdown what languages are not available.
• for a given #str_<id>, show where it is used, not just defined.
• have Coverage show, out of items needing translation for a given language, the percent completed.
• have some method of stepping through the needing-translation items.
• during results display, perhaps incorporate some smarts related to the actual TDM bitmap font that deploys the string.
• and of course editing.

Additional comments about internal structure may be found in the source code.

Downloads

[TO DO] [SEE ALSO TO DO] [TEMPLATE TO DO]