Rebuild all lang Program

by Geep, 2026

Introduction

Ordinarily, an FM that has been internationalized has a UTF8 "all.lang" text file, from which "*.lang" files in various 8-bit ISO-8859-x (or Windows) encodings were generated and distributed within the "strings" folder.

Sometimes, an attempt is made to improve translations, or extend them to additional languages. The ideal way is to alter content in all.lang, then regenerate the *.lang files (e.g., with gen_lang_plus). This regeneration could be for every language section specified in all.lang, or more selectively, just the ones for languages newly added or with changed content.

Suppose "all.lang" was not distributed with the FM. The original FM author may be able to provide it, or you can ask about it being in some TDM SVN repo.

If those routes fail, you can fabricate a replacement all.lang from the *.lang files. Note that the latter are stripped of helpful comments, including grouping header lines, often found in the original all.lang. So this approach is driven by necessity.

One way that fabrication can be done is first make a new partial all.lang from the existing english.lang file (which being in ASCII, can be viewed as also UTF8), Then add new language sections in UTF8 form. Regenerate just those *.lang files, while retaining older unchanged *.lang files. Each new UTF8 language section can be -

• typed from scratch, while consulting the ISO-8859-x version.
• possibly generated using some third-party ISO to UTF8 translator tool.

Or, as a convenience, use rebuild_all_lang. When run from within a give folder (e.g., "strings"), it will take every *.lang it finds (starting with english.lang), convert it from that language's ISO to UTF8, and basically glue them together into the required form.

Usage and Example

The Windows command-line program rebuild_all_lang is a simple C++ utility that takes no parameters, and if successful generates "all.lang" in the current directory. Control it by - - deciding in advance which *.lang files are the current directory - if you want the generated file to embed a preamble, you can provide it as a file named "preamble_utf8.txt" in the same directory. This is optional. As an example, the program distribution includes a typical stock preamble.

A typical output to console is:

Running rebuild_all_lang, v. 1.0
Note: Did not find an optional preamble file preamble_utf8.txt
Processing...
English
German
French
Italian
Spanish
Polish
Romanian
Russian
Portuguese
Czech
Hungarian
Slovak
Swedish
Danish
Dutch
Turkish
Catalan
Contents of *.lang files were converted successfully to UTF-8 and merged into new 'all.lang'.

Languages are always processed in the order shown. If a particular *.lang is missing, that is handled quietly, and that section is not added to all.lang.

At the start of all.lang, typical text is...

// Rebuilt all.lang file - created by rebuild_all_lang v. 1.0
// Strings in each section in same order as in *.lang, but converted from various iso-8859s encodings to UTF8.

// Note - did not find an optional file preamble_utf8.txt with preamble comments to place here.

{
[English]
// Sourced from english.lang of 2025-07-16T16:08:26+00:00, whose first line was...
// String table - english (iso-8859-1) - created on 2025-07-16T16:08:26Z with gen_lang_plus v. 1.1
	"#str_fm_airpocket_xd_sheet_appointment_to_service_pg1__title"	"Appointment to Service\n"
	....

Within each language section - like [English] here - some variant of the first two commented lines are always generated. As you see, the "// Sourced from..." line is followed by the first line copied from the *.lang file. In this example, the comments show the external file modification datetime, and then the datetime written at generation time. The latter will not always be present, depending upon creation method.

Details

As required, the generated all.lang text file has Linux linebreaks (e.g., LF instead of CRFL), independent of linebreak form in input files *.lang or preamble_UTF8.txt

If all.lang already exists, you will see...

Running rebuild_all_lang, v. 1.0
File 'all.lang' already exists.
Continue and overwrite it? [y/n]

If you indicate "n" for "no", the program quits with message "Bye".

This program has some minimal file I/O and conversion error checking, with reporting to console. It is not thought necessary to do character-level checking of the UTF8 results against characters supported by TDM's character map... such checking is better done elsewhere. This is a simple program. (One could imagine a more elaborate version that would, say, allow english.lang to be annotated with comments (e.g., to group lines into subsections). And then reorder lines within the non-English language sections to match, embedding those same annotations.)

Downloads

Source code: This C++ v20 program was developed under Windows 11 with VS 2022 Community Edition (updated through 2025).

• Windows executable rebuild_all_lang.exe
• rebuilt_all_lang.zip - source code: a .cpp and .sln file

Example preamble files (rename to preamble_utf8.txt to use):

• preamble_utf8[traditional].txt
• preamble_utf8[modern for FM].txt

See Also

[COMING SOON]