X4 Catalog Tool - Extract multiple catalogues at once with filtering

[foxxbl]

With the X4 Catalog Tool, is it possible to extract all catalogues for the main game and all DLCs at the same time (in a one command) using the -include filtering to specify files to extract ?

This is what I currently have - it's cumbersome but it's working OK:

Code: Select all

set WORKDIR=%~dp0..\X4Cat
set X4DIR=D:\SteamLibrary\steamapps\common\X4 Foundations
set TOOLDIR=%~dp0\XRCatTool

echo "Working Directory: %WORKDIR%"
echo "X4 Directory: %X4DIR%"
echo "Tool Directory: %TOOLDIR%"

rmdir /s /q "%WORKDIR%"
mkdir "%WORKDIR%"
mkdir "%WORKDIR%\extensions\ego_dlc_split"
mkdir "%WORKDIR%\extensions\ego_dlc_terran"
mkdir "%WORKDIR%\extensions\ego_dlc_pirate"
mkdir "%WORKDIR%\extensions\ego_dlc_boron"
mkdir "%WORKDIR%\extensions\ego_dlc_timelines"
mkdir "%WORKDIR%\extensions\ego_dlc_mini_01"
mkdir "%WORKDIR%\extensions\ego_dlc_mini_02"

%TOOLDIR%\XRCatTool.exe -in "%X4DIR%\01.cat" -in "%X4DIR%\02.cat" -in "%X4DIR%\03.cat" -in "%X4DIR%\04.cat" -in "%X4DIR%\05.cat" -in "%X4DIR%\06.cat" -in "%X4DIR%\07.cat" -in "%X4DIR%\08.cat" -in "%X4DIR%\09.cat" -out "%WORKDIR%" -include "xml$" "xsd$" "dtd$"
%TOOLDIR%\XRCatTool.exe -in "%X4DIR%\extensions\ego_dlc_split\ext_01.cat" -in "%X4DIR%\extensions\ego_dlc_split\ext_02.cat" -in "%X4DIR%\extensions\ego_dlc_split\ext_03.cat" -out "%WORKDIR%\extensions\ego_dlc_split" -include "xml$" "xsd$" "dtd$"
%TOOLDIR%\XRCatTool.exe -in "%X4DIR%\extensions\ego_dlc_terran\ext_01.cat" -in "%X4DIR%\extensions\ego_dlc_terran\ext_02.cat" -in "%X4DIR%\extensions\ego_dlc_terran\ext_03.cat" -out "%WORKDIR%\extensions\ego_dlc_terran" -include "xml$" "xsd$" "dtd$"
%TOOLDIR%\XRCatTool.exe -in "%X4DIR%\extensions\ego_dlc_pirate\ext_01.cat" -in "%X4DIR%\extensions\ego_dlc_pirate\ext_02.cat" -in "%X4DIR%\extensions\ego_dlc_pirate\ext_03.cat" -out "%WORKDIR%\extensions\ego_dlc_pirate" -include "xml$" "xsd$" "dtd$"
%TOOLDIR%\XRCatTool.exe -in "%X4DIR%\extensions\ego_dlc_boron\ext_01.cat" -in "%X4DIR%\extensions\ego_dlc_boron\ext_02.cat" -in "%X4DIR%\extensions\ego_dlc_boron\ext_03.cat" -out "%WORKDIR%\extensions\ego_dlc_boron" -include "xml$" "xsd$" "dtd$"
%TOOLDIR%\XRCatTool.exe -in "%X4DIR%\extensions\ego_dlc_timelines\ext_01.cat" -in "%X4DIR%\extensions\ego_dlc_timelines\ext_02.cat" -in "%X4DIR%\extensions\ego_dlc_timelines\ext_03.cat" -out "%WORKDIR%\extensions\ego_dlc_timelines" -include "xml$" "xsd$" "dtd$"
%TOOLDIR%\XRCatTool.exe -in "%X4DIR%\extensions\ego_dlc_mini_01\ext_01.cat" -in "%X4DIR%\extensions\ego_dlc_mini_01\ext_02.cat" -in "%X4DIR%\extensions\ego_dlc_mini_01\ext_03.cat" -out "%WORKDIR%\extensions\ego_dlc_mini_01" -include "xml$" "xsd$" "dtd$"
%TOOLDIR%\XRCatTool.exe -in "%X4DIR%\extensions\ego_dlc_mini_02\ext_01.cat" -in "%X4DIR%\extensions\ego_dlc_mini_02\ext_02.cat" -in "%X4DIR%\extensions\ego_dlc_mini_02\ext_03.cat" -out "%WORKDIR%\extensions\ego_dlc_mini_02" -include "xml$" "xsd$" "dtd$"

Thanks!

[alexalsp]

Имя файла: XRCatToolUnpackFiles.bat
Сохранить надо в кодировке как Кирилица OEM 866 для русского.
Рядом положить: XRCatTool.exe

File name: XRCatToolUnpackFiles.bat
Save in Cyrillic OEM 866 encoding for Russian.
Place the file near: XRCatTool.exe

Code: Select all

:: ==============================
:: ALEXALSP
:: ==============================
  @echo off
setlocal enabledelayedexpansion

:: ==============================
:: ЯЗЫК / LANGUAGE
:: ==============================
:: Установите RUS или ENG
:: Set RUS or ENG
set "LANG=RUS"

:: ==============================
:: НАСТРОЙКИ / SETTINGS
:: ==============================
set "TOOL=XRCatTool.exe"
set "GAME_ROOT=d:\Games\Steam\steamapps\common\X4 Foundations\000.DIFF_OUTPUT_OLD_FILES"
set "OUT_ROOT=j:\001. Unpack X4\x8.00 HF1"

:: ==============================
:: ПАРАМЕТРЫ ВКЛ/ВЫКЛ / INCLUDE/EXCLUDE PARAMETERS
:: ==============================
set "USE_INCLUDE_EXTENSIONS=ON"
set "USE_INCLUDE_FOLDERS=OFF"
set "USE_EXCLUDE_FOLDERS=OFF"

set "INCLUDE_EXTENSIONS_LIST=xml$"
set "INCLUDE_FOLDER_LIST=libraries/"
set "EXCLUDE_FOLDER_LIST=assets/ libraries/"

:: ==============================
:: ФОРМИРУЕМ ПАРАМЕТРЫ / BUILD PARAMETERS
:: ==============================
set "INCLUDE_PARAM="
if /I "%USE_INCLUDE_EXTENSIONS%"=="ON" if not "%INCLUDE_EXTENSIONS_LIST%"=="" (
    for %%E in (%INCLUDE_EXTENSIONS_LIST%) do (
        set "INCLUDE_PARAM=!INCLUDE_PARAM! -include "%%E""
    )
)

set "INCLUDE_FOLDER_PARAM="
if /I "%USE_INCLUDE_FOLDERS%"=="ON" if not "%INCLUDE_FOLDER_LIST%"=="" (
    for %%F in (%INCLUDE_FOLDER_LIST%) do (
        set "INCLUDE_FOLDER_PARAM=!INCLUDE_FOLDER_PARAM! -include "%%F""
    )
)

set "EXCLUDE_FOLDER_PARAM="
if /I "%USE_EXCLUDE_FOLDERS%"=="ON" if not "%EXCLUDE_FOLDER_LIST%"=="" (
    for %%X in (%EXCLUDE_FOLDER_LIST%) do (
        set "EXCLUDE_FOLDER_PARAM=!EXCLUDE_FOLDER_PARAM! -exclude "%%X""
    )
)

:: ==============================
:: ФУНКЦИИ ВЫВОДА / ECHO FUNCTIONS
:: ==============================
:: Русские / English messages
set "MSG_UNPACK_MAIN_RUS=== Распаковка основной папки ==="
set "MSG_UNPACK_MAIN_ENG=== Unpacking main game folder ==="

set "MSG_UNPACK_DLC_RUS=== Распаковка всех DLC и модов из extensions ==="
set "MSG_UNPACK_DLC_ENG=== Unpacking all DLC and mods from extensions ==="

set "MSG_DONE_RUS==== ГОТОВО ===="
set "MSG_DONE_ENG==== DONE ===="

:: ==============================
:: 1. РАСПАКОВКА КОРНЕВОЙ ПАПКИ / MAIN GAME UNPACK
:: ==============================
if /I "%LANG%"=="RUS" (echo !MSG_UNPACK_MAIN_RUS!) else (echo !MSG_UNPACK_MAIN_ENG!)
for %%F in ("%GAME_ROOT%\*.cat") do (
    echo %%~nxF | findstr /I "_sig" >nul
    if !errorlevel! == 0 (
        if /I "%LANG%"=="RUS" (echo Пропуск файла %%~nxF ...) else (echo Skipping file %%~nxF ...)
    ) else (
        if /I "%LANG%"=="RUS" (echo Распаковка "%%F" в "%OUT_ROOT%" ...) else (echo Unpacking "%%F" to "%OUT_ROOT%" ...)
        "%TOOL%" -in "%%F" -out "%OUT_ROOT%" !INCLUDE_PARAM! !INCLUDE_FOLDER_PARAM! !EXCLUDE_FOLDER_PARAM!
    )
)

:: ==============================
:: 2. РАСПАКОВКА MOD/DLC / MOD/DLC UNPACK
:: ==============================
if /I "%LANG%"=="RUS" (echo !MSG_UNPACK_DLC_RUS!) else (echo !MSG_UNPACK_DLC_ENG!)
for /r "%GAME_ROOT%\extensions" %%F in (*.cat) do (
    :: --- ПРОВЕРКА НА _sig --- / CHECK FOR _sig
    echo %%~nxF | findstr /I "_sig" >nul
    if !errorlevel! == 0 (
        if /I "%LANG%"=="RUS" (echo Пропуск файла %%~nxF ...) else (echo Skipping file %%~nxF ...)
    ) else (
        :: формируем путь относительно папки extensions / build relative path
        set "REL_PATH=%%F"
        set "REL_PATH=!REL_PATH:%GAME_ROOT%\extensions\=!" 

        :: определяем целевую папку / define target folder
        set "TARGET_DIR=%OUT_ROOT%\extensions\!REL_PATH!"
        for %%I in ("!TARGET_DIR!") do set "TARGET_DIR=%%~dpI"
        if "!TARGET_DIR:~-1!"=="\" set "TARGET_DIR=!TARGET_DIR:~0,-1!"

        if not exist "!TARGET_DIR!" mkdir "!TARGET_DIR!"

        if /I "%LANG%"=="RUS" (echo Распаковка "%%F" в "!TARGET_DIR!" ...) else (echo Unpacking "%%F" to "!TARGET_DIR!" ...)
        "%TOOL%" -in "%%F" -out "!TARGET_DIR!" !INCLUDE_PARAM! !INCLUDE_FOLDER_PARAM! !EXCLUDE_FOLDER_PARAM!
    )
)



:: ==============================
:: УДАЛЕНИЕ ПУСТЫХ ПАПОК / EMPTY FOLDERS REMOVAL
:: ==============================
echo.
if /I "%LANG%"=="RUS" (echo Удаление пустых папок ...) else (echo Empty folders removal ...)

:: Рекурсивно удаляем все пустые папки внутри OUT_ROOT, кроме самого OUT_ROOT
for /f "delims=" %%D in ('dir "%OUT_ROOT%" /ad /s /b ^| sort /r') do (
    rd "%%D" 2>nul
    if not exist "%%D" (
        if /I "%LANG%"=="RUS" (echo Удалена пустая папка: %%D) else (echo Deleted empty folder: %%D)
    )
)

if /I "%LANG%"=="RUS" (echo Удаление завершено.) else (echo Removal completed.)
echo.


:: ==============================
:: ГОТОВО / DONE
:: ==============================
if /I "%LANG%"=="RUS" (echo !MSG_DONE_RUS!) else (echo !MSG_DONE_ENG!)
pause

Added EMPTY FOLDERS REMOVAL

[Dj_FRedy]

Code: Select all

1) "d:\Games\Steam\steamapps\common\X4 Foundations\01.cat" -out "j:\Unpack" -include "xml$" "css$"
2) "d:\Games\Steam\steamapps\common\X4 Foundations\01.cat" -out "j:\Unpack" -include "^index/" "^libraries/"
3) "d:\Games\Steam\steamapps\common\X4 Foundations\01.cat" -out "j:\Unpack" -exclude "^index/" "^libraries/"

As I understand it, only examples 1 and 3 can work together?
I tried using 1 and 2, but they don't work together, or am I doing something wrong?

Hey, I'll answer you here then. Sorry, I didn't want a request thread to turn into a thread full of code replies and such.

The items or folders for the '-include' and '-exclude' parameters can't be the same. '-exclude' is used to exclude items or folders for already included files.

If I want to extract the entire assets folder, but I'm not interested in the files related to the ships, I do it like this:

Code: Select all

  -include "^assets/" -exclude "^assets/units"

for two folders in the same already included directory:

Code: Select all

  -include "^assets/" -exclude "^assets/units" "^assets/wares"

for a specific file in the same already included directory:

Code: Select all

  -include "^assets/" -exclude "^assets/units/size_s/cockpit_invisible.xml"

Adapted to your script for the first example:

Code: Select all

set "USE_INCLUDE_EXTENSIONS=OFF"
set "USE_INCLUDE_FOLDERS=ON"
set "USE_EXCLUDE_FOLDERS=ON"

set "INCLUDE_EXTENSIONS_LIST="
set "INCLUDE_FOLDER_LIST=assets/"
set "EXCLUDE_FOLDER_LIST=assets/units"

It's also important to note what the Readme file says about the '-exclude' parameter:

Even for files that are not included, this can speed up the scanning, e.g.: -exclude "^assets/"

So, if I want to speed up the process, I'll try to exclude all the directories I'm not interested in.
I typically exclude the following directories if I'm not looking for files other than the ones I regularly modify:

Code: Select all

...
set excludeMusicAndVoiceTranslation="^music/" "^voice-l044/" "^voice-l049/"
set excludeCustom="xpl$" "^particles/" "^sfx/" "^textures/"
set excludeFiles=%excludeMusicAndVoiceTranslation% %excludeCustom%
...

[alexalsp]

Basically, it's clear. It's just a shame there's no way to combine 1 and 2, but oh well, that's no big deal )))

Thanks.

[Dj_FRedy]

Yes, you can . The script you provided works perfectly.
Configured as I would to unpack all the script files:

Code: Select all

:: ==============================
:: ПАРАМЕТРЫ ВКЛ/ВЫКЛ / INCLUDE/EXCLUDE PARAMETERS
:: ==============================
set "USE_INCLUDE_EXTENSIONS=ON"
set "USE_INCLUDE_FOLDERS=ON"
set "USE_EXCLUDE_FOLDERS=ON"

set "INCLUDE_EXTENSIONS_LIST=xml$ xsd$ dtd$ lua$ html$ css$ js$ xsl$"
set "INCLUDE_FOLDER_LIST=shadergl/"
set "EXCLUDE_FOLDER_LIST=music/ voice-l044/ voice-l049/ xpl$ particles/ sfx/ textures/"

[alexalsp]

Added EMPTY FOLDERS REMOVAL

viewtopic.php?p=5291532#p5291532

set "USE_INCLUDE_EXTENSIONS=ON"
set "USE_INCLUDE_FOLDERS=ON"
set "USE_EXCLUDE_FOLDERS=ON"

set "INCLUDE_EXTENSIONS_LIST=xml$ xsd$ dtd$ lua$ html$ css$ js$ xsl$"
set "INCLUDE_FOLDER_LIST=shadergl/"
set "EXCLUDE_FOLDER_LIST=music/ voice-l044/ voice-l049/ xpl$ particles/ sfx/ textures/"

It doesn't work ))

Look, do this

set "USE_INCLUDE_EXTENSIONS=ON"
set "USE_INCLUDE_FOLDERS=ON"
set "USE_EXCLUDE_FOLDERS=ON"

set "INCLUDE_EXTENSIONS_LIST=xml$"
set "INCLUDE_FOLDER_LIST=shadergl/"
set "EXCLUDE_FOLDER_LIST=music/ voice-l044/ voice-l049/ xpl$ particles/ sfx/ textures/"

Why isn’t it working? Logically, it should be like this:

Only this folder should be unpacked with the specified extension filter, but in reality, all folders are being unpacked with that type of extension.

Maybe it’s just that the one folder specified unpacked for you , I don’t know.

And one more thing:

It turns out there’s a difference in how you unpack by extension.

Code: Select all

xml$ f$

and

Code: Select all

\.xml$ \.f$

The first option is for extensions that happen to include xml (like *.bobaxml).
The second option is for the exact file extension, specifically *.xml, *.f.

Just in case anyone’s curious

[Dj_FRedy]

...

set "USE_INCLUDE_EXTENSIONS=ON"
set "USE_INCLUDE_FOLDERS=ON"
set "USE_EXCLUDE_FOLDERS=ON"

set "INCLUDE_EXTENSIONS_LIST=xml$"
set "INCLUDE_FOLDER_LIST=shadergl/"
set "EXCLUDE_FOLDER_LIST=music/ voice-l044/ voice-l049/ xpl$ particles/ sfx/ textures/"

Why isn’t it working? Logically, it should be like this:

Only this folder should be unpacked with the specified extension filter, but in reality, all folders are being unpacked with that type of extension.
...

Ok, try this other way for the "-include" parameter "folder/.+\.extension":

Code: Select all

set "USE_INCLUDE_EXTENSIONS=OFF"
set "USE_INCLUDE_FOLDERS=ON"
set "USE_EXCLUDE_FOLDERS=ON"

set "INCLUDE_EXTENSIONS_LIST=xml$"
set "INCLUDE_FOLDER_LIST=shadergl/.+\.xml"
set "EXCLUDE_FOLDER_LIST=music/ voice-l044/ voice-l049/ xpl$ particles/ sfx/ textures/"

You can extrapolate it to the folder or file extension you want and if I understood you correctly, it would work the way you expect based on my tests.

[alexalsp]

Yes, this option works
I thought there were files, folders, and exceptions: files and folders. But I didn't think that files could also be specified in folders (INCLUDE)...

[alexalsp]

One more question.

If it is possible to do this for (Include), then what is the point of dividing it into two parts?

I thought these were two different conditions, but it turns out that everything can be done in one step.
If necessary, by extension, or only by folder, or by folder with extension.

So, there was no point in separating them?

Code: Select all

set "USE_INCLUDE_FOLDERS_AND_FILES=ON"
set "USE_EXCLUDE_FOLDERS=ON"

That's enough, judging by your latest examples?

[Dj_FRedy]

You have answered yourself. But yes, it doesn't matter if you mix folder and extension in the same variable for "include" and "exclude" parameters to work correctly. Ultimately, they are instructions for tracing strings of names in form of lines for files or folders.
But by that rule, shouldn't it be this way?

Code: Select all

set "USE_INCLUDE_FOLDERS_AND_FILES=ON"
set "USE_EXCLUDE_FOLDERS_AND_FILES=ON"

think about it

[alexalsp]

X Catalog Tool GUI for Console XRCatTool

Purpose: A graphical interface wrapper for XRCatTool.exe, designed to unpack game archives (*.cat) with flexible settings.
What for: Simplifies unpacking of Egosoft X-series game files by providing a GUI with configuration saving.

Features:

* Selection of the game folder and the folder for unpacking
* Auto-detects *.cat files (ignores * sig.cat).
* Specify file types to unpack.
* Exclude folders/files by pattern.
* Save and load custom configs.
* Auto-clean empty folders after unpack.
* Built-in XRCatTool.exe check and extraction.
* Process logging.
* Cancel operation option.

System Requirements:

* OS: Windows 7 / 8 / 10 / 11
* .NET Framework 4.6.1 or higher

Just to make unpacking easier, nothing more...

Spoiler

Show

[alexalsp]

Version 1.0.3
Date: 24-03-2026

Added options exclusively for the EXTENSIONS folder:
- Unpack only the EXTENSIONS folder: enable only if you need to unpack data directly into the game folder.
(e.g., you specify the source and destination as the same folder, like c:\X4 to c:\X4).
- Delete cat/dat after unpacking.
- Field for folders to be excluded from processing (all DLC folders by default).

When this option is disabled, unpacking proceeds in normal mode.

[alexalsp]

X Catalog Tool GUI for Console XRCatTool

Version 1.1.0

Pack mode — completely new functionality for creating .cat files from unpacked mod folders.
Switching between modes — radio buttons to select "Unpack" or "Pack" mode with automatic UI switching.
Pack parameters: Select pack type (simple or structural). Ability to specify .cat file name. Subfolder processing (first level or recursive). Fields for pack rules and exclude rules.
Separate folder storage — different paths are saved for unpack and pack modes.

Spoiler

Show

[alexalsp]

Version 1.1.1

- Added "File" button: you can now specify one or more individual files for unpacking, not just a folder.
- Removed catalog tools utility log output due to lack of Cyrillic support.
- Fixed: Added normalization for selected and pasted paths. Previously, a trailing backslash (\) prevented the path from being recognized correctly.
- Path normalization now ensures the utility works correctly regardless of whether there is a trailing slash.

[alexalsp]

1.1.2

Fix unpack