Can someone help me understand how the t-file's work?

[thomassanford]

Here is just a random Name from a l miner name="{20101,11104}". Can someone tell me how to decode this into it's filename and then the page and ID it represents. I am working on an AI translator and it is working great right up to the {XXXXXXXX,XXXXXX} point. So, any help turning this into a filename to open and then a page number and id number to look up it would be awesome. I have searched and searched on the forum. I guess I am not hitting the right keywords. Thanks, Fly safe..

EDIT: - I got it, I think. Correct me if I am wrong please. All the descriptor will be in the file corresponding to the language used like 1044 for English. Then the first number is a <page id="20101" and the second the t id like this: <page id="20101" title="Ships" .. .. <t id="11104">(Magnetar \(Gas\) Vanguard). So {20101,11104} = Magnetar Vanguard. Correct?

[Duncaroos]

Each language file is for a specific language (e.g., "0001-L044.xml" is for English). Language files contain all base game and DLC language data. You can find all language files under the game catalogs. Details on what each language ID is (e.g., L044 = 44, L007 = 7) - can found in the "language.xml" file in the libraries subfolders of the game catalogs.

Each player will have the ability to change their language with whatever is available; this tells the game what language file to lookup (player set language is under the "lang.dat" file under the main X4 installation folder.

Within each language "t" file, you have pages {e.g., 20101 in your example) and a record {e.g., 11104 in your example}.

The only thing you need to worry about for files, is what language you want to update. So english translations, you need to open 0001-L044.xml and all pages and records under each page are all there.

Some quirks:

• Anything bounded by round brackets (text) are translation notes and the game removes them.
• If a text in game is supposed to show the round brackets, a slash is added: "\(" and "\)"

I made an excel VBA code to do my best in getting the text that the game wants to display, but I get this may be too complex to read in a browser without context in Excel:

Code: Select all

Function GetEgoName(SetLanguage As String, LocalNameRef As Variant, LTblRef As Range) As String
'SetLanguage = The language ID (44, 48, 82, 83, etc.)
'LocalNameRef = Value of {page,record} (e.g., {20101,11104})
'LTblRef = Used this to track changes to an Excel Language Table that converted and stored data from the language xml files, for automatic updates. It does nothing in the code itself.
    
    'Skip blank cells (typ for input ware lookup where input ware is blank)
        If LocalNameRef = "" Then
            GetEgoName = ""
            Exit Function
        End If

    'Set Database
        Dim LTbl As ListObject
            Set LTbl = EData_01.ListObjects("RawData_ObjNameData")	'This is the language table I have for my tool.

        Dim Ind_UID As Long, Ind_Name As Long
            Ind_UID = LTbl.ListColumns("{Lan,Page,ID}").Index			'Gets column index for a unique ID for a specific row, by adding the language ID to each parameter so that it can handle multiple IDs.
            Ind_Name = LTbl.ListColumns("Item Value").Index			'The stored record value.

    'Set Variables
        Dim temp_out As String

    'Cleanup Name
        temp_out = TransformNameRefs(SetLanguage, LocalNameRef, LTbl, Ind_UID, Ind_Name)     'Transform embedded references
        temp_out = RemoveNameTips(temp_out)                                                                                         'Remove Bracketed Tips (*) meant for for Egosoft coders/translators
        temp_out = WorksheetFunction.Substitute(WorksheetFunction.Substitute(temp_out, "S¶_¶", "("), "¶_¶S", ")")                   'Return ( ) parts of name.

    GetEgoName = temp_out

End Function


Private Function TransformNameRefs(SetLanguage As String, ByVal tempStr As Variant, LTbl As ListObject, Ind_UID As Long, Ind_Name As Long) As String

    'Set Variables
        Dim a As Long, b As Long
        Dim temp_left As String, temp_LocalNameRef As String, temp_FullName As String, temp_right As String
        Dim Bln_num As Boolean

    'Initialize Loop
        On Error Resume Next
            a = WorksheetFunction.Find("{", tempStr)
            b = WorksheetFunction.Find("}", tempStr)
        Err = 0

    'Loop while string for name still contains sets of {*} for NameReference
        While a + b > 0
            'Split Current tempstr
                If a = 1 Then temp_left = Empty Else temp_left = Left(tempStr, a - 1)
                temp_LocalNameRef = WorksheetFunction.Substitute(Mid(tempStr, a, (b - a) + 1), " ", "")
                If b = 0 Then temp_right = Empty Else temp_right = Right(tempStr, Len(tempStr) - b)

            'Get Replacement String
                On Error Resume Next
                    temp_FullName = Trim(WorksheetFunction.XLookup(FullObjNameRef(SetLanguage, temp_LocalNameRef), LTbl.ListColumns(Ind_UID).DataBodyRange, LTbl.ListColumns(Ind_Name).DataBodyRange))
                If Err <> 0 Then
                    temp_FullName = WorksheetFunction.Substitute(WorksheetFunction.Substitute(temp_LocalNameRef, "{", "["), "}", "]")
                    Err = 0
                End If

            'Recombine String
                temp_FullName = RemoveNameTips(temp_FullName)
                tempStr = Trim(temp_left & temp_FullName & temp_right)
                tempStr = WorksheetFunction.Substitute(tempStr, "  ", " ")

            'Reset Search for next {x}
                a = 0
                b = 0

                On Error Resume Next
                    a = WorksheetFunction.Find("{", tempStr)
                    b = WorksheetFunction.Find("}", tempStr)
                    temp_LocalNameRef = Empty
                    temp_FullName = Empty
                Err = 0
        Wend

    'Output result
        TransformNameRefs = tempStr

End Function

Private Function RemoveNameTips(ByVal tempStr As String) As String

    Dim a As Long, b As Long, a2 As Long
    Dim temp_left As String, temp_right As String, temp_tip As String

    'Definition tips by Egosoft between standard brackets (e.g., ">Name (Definition Tip)<"). Where brackets are part of the name, a delimiter "\" is placed before the bracket.
    'Use "S¶_¶" and "¶_¶S" as new substitutions for ( ) parts of names.
        tempStr = WorksheetFunction.Substitute(WorksheetFunction.Substitute(tempStr, "\(", "S¶_¶"), "\)", "¶_¶S")
    
    'Initialize Loop
        On Error Resume Next
            a = WorksheetFunction.Find("(", tempStr)
            b = WorksheetFunction.Find(")", tempStr)
        Err = 0

    'Loop while string for name still contains sets of (*) for tooltips
        While (a > 0 And b > 0)
            'Reset looped variables
                temp_left = Empty
                temp_tip = Empty
                temp_right = Empty

            'Split Current tempstr away from (*)
                temp_left = Left(tempStr, a - 1)
ChkAgain:       temp_tip = Mid(tempStr, a, b - a + 1)

                'Check embedded (*) - e.g. "ABC (CDE (EFG) HIJ)"
                    a2 = 0
                    a2 = WorksheetFunction.Find("(", temp_tip)
                        If a2 > 1 Then
                            temp_left = temp_left & Left(temp_tip, a2 - 1)
                            a = Len(temp_left) + 1
                            GoTo ChkAgain
                        End If
                    Err = 0

                temp_right = Right(tempStr, Len(tempStr) - b)

            'Combine just trimmed results
                tempStr = Trim(temp_left & temp_right)
                tempStr = WorksheetFunction.Substitute(tempStr, "  ", " ")

            'Reset Search for next (x)
                a = 0
                b = 0

                On Error Resume Next
                    a = WorksheetFunction.Find("(", tempStr)
                    b = WorksheetFunction.Find(")", tempStr)
                Err = 0
        Wend

    'Output result
        RemoveNameTips = tempStr

End Function

For context with this function, my tool in my signature uses this ad nauseum. You just need to unhide some hidden sheets.

[sprIder]

If you have unpacked the .cat and .dat files you'll find a folder with the name "t". In this are some .xml files, the language files. 0001-l044.xml is the English language file, 0001-l049.xml the German one. And so on.

For "{20101,11104}":
20101 is the page id, 11104 is the t id.
-> open your language file, search for "page id="20101" and then scroll down for the t id.

Code: Select all

<t id="11104">(Magnetar \(Gas\) Vanguard){20101,11101} {20111,3201} {20111,1101}</t>

In this case the name uses existing lines of text and refers to them, again with page id and t id. Text in "(...)" will not be shown in game. The name "Magnetar Gas Vanguard" is split into three parts:

1. {20101,11101} "Magnetar"
2. {20111,3201} "Gas"
3. {20111,1101} "Vanguard"

I hope this helps.

[Duncaroos]

EDIT: - I got it, I think. Correct me if I am wrong please. All the descriptor will be in the file corresponding to the language used like 1044 for English. Then the first number is a <page id="20101" and the second the t id like this: <page id="20101" title="Ships" .. .. <t id="11104">(Magnetar \(Gas\) Vanguard). So {20101,11104} = Magnetar Vanguard. Correct?

The official raw content of {20101,11104} is (Magnetar \(Gas\) Vanguard){20101,11101} {20111,3201} {20111,1101}

Note that (Magnetar \(Gas\) Vanguard) is in brackets, so it is a translation note to how it should ideally look like. In-game this is out-right removed.

The game would then lookup:
{20101,11101} = Magnetar
{20111,3201} = \(Gas\) ==> (Gas)
{20111,1101} = Vanguard

(Magnetar \(Gas\) Vanguard){20101,11101} {20111,3201} {20111,1101} ==> Magnetar (Gas) Vanguard

So as you can see, there is a bit of recursiveness, where embedded language records within a language record can exist.

The reason you need to look past the translation note, is for non-english translations the note remains the same (for the most part). E.g., 0001-L082.xml has {20101,11104} as "(Magnetar \(Gas\) Vanguard){20101,11101} {20111,3201} {20111,1101}", which is not helpful and you have to dig out these 3 language parameters to get: 마그네타 \(기체\) 뱅가드 ==> 마그네타 (기체) 뱅가드

[thomassanford]

Thanks all! That was fast. I had responses faster than I could find most of it. As to the point of why, I said translate when I guess I meant trans code them. I need then to be coherent sentences for model training and retrieval augmentation. There was talk in another post about using a small LLM as the ships AI voice so, I have been playing with it a bit as something to direct my project at. Funny to hear it spit out a sentences and in the middle you get '20101 comma 11101' in computer voice. Time for some python replacement magic. Thanks again guys.

[Duncaroos]

Thanks all! That was fast. I had responses faster than I could find most of it. As to the point of why, I said translate when I guess I meant trans code them. I need then to be coherent sentences for model training and retrieval augmentation. There was talk in another post about using a small LLM as the ships AI voice so, I have been playing with it a bit as something to direct my project at. Funny to hear it spit out a sentences and in the middle you get '20101 comma 11101' in computer voice. Time for some python replacement magic. Thanks again guys.

Let me know if you want to get those language records processed. Just need a bit of work to my tool below (removing everything except the language records) and have the function write out what these coherent sentences by removing all the translation notes and {page,record} callouts.

[thomassanford]

Update: Thanks again for getting me going guys.

I got all the substitutions made and rebuilt the XML for the 0001-l044.xml file. So that all entries are plain English with no {xxxxx,xxxxx} references. I do not know if it would actually run in game. That wasn't the point, but it is a good dataset of game responses and such if anyone wants a copy or the script to rip another language.

From the example.. the Magnetar Gas example yielded:

Code: Select all

<t id="11104">(Magnetar \(Gas\) Vanguard)Magnetar \(Gas\) Vanguard</t>

It could use some cleanup still I will trim the redundancy.

FInal Update: File has been deduped and all useless special characters were removed. It is ready for my use anyways, and if any of this may help you let me know.

Code: Select all

<t id="11104">Magnetar Gas Vanguard</t>