[TOOLS] XML diff and patch for X4: Foundations

The place to discuss scripting and game modifications for X4: Foundations.

Moderators: Scripting / Modding Moderators, Moderators for English X Forum

User avatar
ChemODun
Posts: 414
Joined: Mon, 12. Feb 07, 21:58
x4

[TOOLS] XML diff and patch for X4: Foundations

Post by ChemODun »

Modding tools: XML diff and patch for X4: Foundations

This toolset is a simple XML diff and patch tools for X4: Foundations. It is designed to help modders to compare and patch XML files.

The format of diff XML files is compatible with the appropriate diff.xsd format definition. It is means - you can you this tool to create diff files for any XML files used in game.
Also, you can use appropriate tool to patch XML files with diff files, this action has reason to check how your diff file will be applied to the vanilla XML file. Or better understand, what other modders did in their mods.

Important note

It is highly recommended to use the diff.xsd file to validate the diff XML files. It is especially important when you creating them by XMLDiff.exe.
If the diff.xsd file if it located in the "current" folder it will be used automatically. If you want to use another diff.xsd file, you can specify it with the -x option.

How to use
  • Download the latest release from:
    • GitHub releases page - there is an archive file XMLDiffAndPatch.zip.
    • NexusMods - there is an archive file XMLDiffAndPatch.zip.
  • Extract the archive file to any useful location.
  • Inside will be a folder, named XMLDiffAndPatch with two executables - XMLDiff.exe and XMLPatch.exe.
How to create a diff file

There is a command line help for the XMLDiff tool:

Code: Select all

XMLDiff 0.2.24
Developed by Chem O`Dun

  -o, --original_xml      Required. Path to the original XML file or directory.

  -m, --modified_xml      Required. Path to the modified XML file or directory.

  -d, --diff_xml          Required. Path for the diff XML file or directory.

  -x, --xsd               Path to the diff.xsd schema file.

  -l, --log-to-file       Log level (error, warn, info, debug).

  -a, --append-to-log     (Default: false) Append logs to the existing log file.

  --anywhere-is-allowed   (Default: false) Generate a path using the anywhere '//' construction, if possible. Instead of full path.

  --use-all-attributes    (Default: false) Use all attributes in XPath.

  --help                  Display this help screen.

  --version               Display version information.
Example:

Code: Select all

XMLDiff.exe -o vanilla.xml -m modified.xml -d diff.xml
Example of resulting diff files

There the is example of the diff files created by tool:
  • with add operation:

Code: Select all

    <?xml version="1.0" encoding="utf-8" standalone="yes"?>
    <diff>
    <add sel="//ware[@id=&quot;scanningarrays&quot;]" pos="before">
        <ware id="xenon_psi_emitter_mk1" name="{1972092403, 7002}" description="{1972092403, 7002}" transport="equipment" volume="1" tags="satellite noplayerbuild">
        <price min="845800" average="901420" max="1054580" />
        <production time="60" amount="0" method="default" name="Xenon Psi Emitter" />
        <production time="60" amount="0" method="xenon" name="Xenon Psi Emitter" />
        <production time="60" amount="0" method="terran" name="Xenon Psi Emitter" />
        <component ref="xenon_psi_emitter_macro" amount="0" />
        <use threshold="0" />
        </ware>
    </add>
    </diff>
    
  • with replace operation:

Code: Select all

    <?xml version='1.0' encoding='UTF-8'?>
    <diff>
      <replace sel="//do_if[@value=&quot;@$speak and not this.assignedcontrolled.nextorder and (@$defaultorder.id != 'Patrol') and (@$defaultorder.id != 'ProtectPosition') and (@$defaultorder.id != 'ProtectShip') and (@$defaultorder.id != 'ProtectStation') and (@$defaultorder.id != 'Plunder') and (@$defaultorder.id != 'Police') and (not this.assignedcontrolled.commander or (this.assignedcontrolled.commander == player.occupiedship)) and notification.npc_await_orders.active&quot;]/@value">@$speak and not this.assignedcontrolled.nextorder and (@$defaultorder.id != 'ProtectSector') and (@$defaultorder.id != 'Patrol') and (@$defaultorder.id != 'ProtectPosition') and (@$defaultorder.id != 'ProtectShip') and (@$defaultorder.id != 'ProtectStation') and (@$defaultorder.id != 'Plunder') and (@$defaultorder.id != 'Police') and (not this.assignedcontrolled.commander or (this.assignedcontrolled.commander == player.occupiedship)) and notification.npc_await_orders.active</replace>
    </diff>
    
Path options

Only full path

The --only-full-path option will generate only the full path to the element in the XML file. It is mean - there no // will be in the sel attribute of the add, replace or remove element.

Example:

Code: Select all

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<diff>
<add sel="/wares/ware[@id=&quot;scanningarrays&quot;]" pos="before">
    <ware id="xenon_psi_emitter_mk1" name="{1972092403, 7002}" description="{1972092403, 7002}" transport="equipment" volume="1" tags="satellite noplayerbuild">
    <price min="845800" average="901420" max="1054580" />
    <production time="60" amount="0" method="default" name="Xenon Psi Emitter" />
    <production time="60" amount="0" method="xenon" name="Xenon Psi Emitter" />
    <production time="60" amount="0" method="terran" name="Xenon Psi Emitter" />
    <component ref="xenon_psi_emitter_macro" amount="0" />
    <use threshold="0" />
    </ware>
</add>
</diff>
Use all attributes in XPath

The --use-all-attributes option will generate the sel attribute with all attributes of the element in the XML file. It is mean - there will be all attributes of the element in the sel attribute of the add, replace or remove element.

Example:

Code: Select all

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<diff>
<add sel="/wares/ware[@id=&quot;scanningarrays&quot;][@name=&quot;{20201,3301}&quot;][@description=&quot;{20201,3302}&quot;][@factoryname=&quot;{20201,3304}&quot;][@group=&quot;hightech&quot;][@transport=&quot;container&quot;][@volume=&quot;38&quot;][@tags=&quot;container economy&quot;]" pos="before">
    <ware id="xenon_psi_emitter_mk1" name="{1972092403, 7002}" description="{1972092403, 7002}" transport="equipment" volume="1" tags="satellite noplayerbuild">
    <price min="845800" average="901420" max="1054580" />
    <production time="60" amount="0" method="default" name="Xenon Psi Emitter" />
    <production time="60" amount="0" method="xenon" name="Xenon Psi Emitter" />
    <production time="60" amount="0" method="terran" name="Xenon Psi Emitter" />
    <component ref="xenon_psi_emitter_macro" amount="0" />
    <use threshold="0" />
    </ware>
</add>
</diff>
Defining the position generation for the diff by in-line comment

The pos attribute of the add usually is set to after, taking in account the common logic of how the program is working. But in some cases, you may want to have a diff file with the pos attribute set to before. Mostly it is useful when it working in conjunction with the vscode - XMLDiff with VSCode.
From version 0.2.26, you can define the position of the add element in the diff file by using an in-line comment in the original XML file. The comment should be placed before the element you want to add and should contain the text <!-- pos="before" -->.

How to apply a diff file

There is a command line help for the XMLPatch tool:

Code: Select all

XMLPatch 0.2.20
Developed by Chem O`Dun

  -o, --original_xml     Required. Path to the original XML file or directory.

  -d, --diff_xml         Required. Path to the diff XML file or directory.

  -u, --output_xml       Required. Path for the output XML file or directory.

  -x, --xsd              Path to the diff.xsd schema file.

  -l, --log-to-file      Log level (error, warn, info, debug).

  -a, --append-to-log    (Default: false) Append logs to the existing log file.

  --allow-doubles        (Default: false) Allow doubles in the diff XML. Useful for scripts patching.

  --help                 Display this help screen.

  --version              Display version information.
Example:

Code: Select all

XMLPatch.exe -o vanilla.xml -d diff.xml -u modified.xml
Example of resulting patched XML files

There is an example of the patched XML files created by the tool:
  • with add operation:

Code: Select all

      <ware id="satellite_mk2" name="{20201,20401}" description="{20201,20402}" transport="equipment" volume="1" tags="equipment satellite">
        <price min="44380" average="52215" max="60045"/>
        <production time="60" amount="1" method="default" name="{20206,101}">
          <primary>
            <ware ware="advancedelectronics" amount="5"/>
            <ware ware="energycells" amount="10"/>
            <ware ware="scanningarrays" amount="5"/>
          </primary>
        </production>
        <production time="60" amount="1" method="xenon" name="{20206,601}" tags="noplayerbuild">
          <primary>
            <ware ware="energycells" amount="10"/>
            <ware ware="silicon" amount="1"/>
          </primary>
        </production>
        <component ref="eq_arg_satellite_02_macro"/>
        <use threshold="0"/>
      </ware>
      <ware id="xenon_psi_emitter_mk1" name="{1972092403, 7002}" description="{1972092403, 7002}" transport="equipment" volume="1" tags="satellite noplayerbuild">
        <price min="845800" average="901420" max="1054580"/>
        <production time="60" amount="0" method="default" name="Xenon Psi Emitter"/>
        <production time="60" amount="0" method="xenon" name="Xenon Psi Emitter"/>
        <production time="60" amount="0" method="terran" name="Xenon Psi Emitter"/>
        <component ref="xenon_psi_emitter_macro" amount="0"/>
        <use threshold="0"/>
      </ware>
    
  • with replace operation:

Code: Select all

        <set_to_default_flight_control_model object="this.assignedcontrolled"/>
        <set_value name="$defaultorder" exact="this.assignedcontrolled.defaultorder"/>
        <do_if value="@$speak and not this.assignedcontrolled.nextorder and (@$defaultorder.id != 'Patrol') and (@$defaultorder.id != 'ProtectSector') and (@$defaultorder.id != 'ProtectPosition') and (@$defaultorder.id != 'ProtectShip') and (@$defaultorder.id != 'ProtectStation') and (@$defaultorder.id != 'Plunder') and (@$defaultorder.id != 'Police') and (not this.assignedcontrolled.commander or (this.assignedcontrolled.commander == player.occupiedship)) and notification.npc_await_orders.active">
          <set_value name="$speakline" exact="10304" comment="Awaiting orders."/>
    
If output XML is a directory

If the output XML is a directory, the tool will create a new XML file with the same name as the original XML file in the output directory.
For example, if the original XML file is vanilla.xml and the output directory is output, the tool will create a new XML file output/vanilla.xml.

How to apply a tools to a directories

Applying XMLDiff to directories

You can apply the XMLDiff tool to directories. In this case, the tool will traverse the directory structure and create diff files for XML files with identical names and relative paths. The process is as follows:
  • If all input parameters are directories, the tool will create diff files for all XML files in the directories.
  • The tool will recursively go through the directory structure, using modified files as a "keys" for it
  • For each changed file, the corresponding original XML file in the original directory with the same relative path will be checked.
  • If the original XML file is not found, the operation will be skipped.
  • If the original XML file is found, the diff file will be created in the output directory with the same relative path.
Example:

Code: Select all

XMLDiff.exe -o vanilla_dir -m modified_dir -d diff_dir
Applying XMLPatch to directories

You can apply the XMLPatch tool to directories. In this case, the tool will traverse the directory structure and apply the patch to XML files with identical names and relative paths. The process is as follows:
  • If all input parameters are directories, the tool will apply the patch to all XML files in the directories.
  • The tool will recursively go through the directory structure, using diff files as a "keys" for it
  • For each diff file, the corresponding original XML file in the original directory with the same relative path will be checked.
  • If the original XML file is not found, the operation will be skipped.
  • If the original XML file is found, the diff file will be patched with the original XML file, and a new patched XML file will be created in the output directory with the same relative path.
Example:

Code: Select all

XMLPatch.exe -o vanilla_dir -d diff_dir -u modified_dir
Issues reporting

If you have any issues with the tool, please create an issue in the issues page.
Will be highly appreciated if you will provide a version of used tool and XMLDiff.log or XMLPatch.log file respectively to the tool.
To create such debug file please use the --log-to-file option with debug level.

License

There is a MIT license for this tool. You can find it in the LICENSE file.

Credits

Special thanks to Duncaroos for the patience, testing, and valuable feedback.

Additional links

There is a topic on the EGOSOFT forum, related to this toolset.

Antivirus scanning

Please be aware - each release archive has an appropriate link to the VirusTotal. Follow the link to be sure that the archive is safe.
Last edited by ChemODun on Mon, 16. Jun 25, 12:33, edited 10 times in total.
Multiply entropy by absolute zero

Freedom in space
User avatar
ChemODun
Posts: 414
Joined: Mon, 12. Feb 07, 21:58
x4

Re: [TOOLS] Python based XML Diff and Patch utilities

Post by ChemODun »

Changelog

[0.2.26] - 2025-06-16
  • Improved:
    • XMLDiff: added in-line comments processing for the add elements in the modified XML file. It allows to define the position attribute of the add element in the diff file.
[0.2.25] - 2025-03-31
  • Fixed:
    • XMLDiff: fixed issue missed root element attributes comparison
    • XMLDiff: fixed usage of the removed element path for the further diff operations
    • XMLDiff: fixed incorrect index number when addressing the elements in XPath
    • XMLDiff: fixed identification elements via sibling keyword in XPath
[0.2.24] - 2025-03-17
  • Improved:
    • Both utilities: result folder will be created in recursive processing as it made now for the single file.
    • XMLDiff: sibling keyword usage in XPath for the elements.
    • XMLDiff: XPath generation when the element has child elements, which can unique identify it.
  • Changed:
    • XMLDiff: --only-full-path option replaced by --anywhere-is-allowed. And default behavior is to use full path.
    • Both utilities: log level of console will not be more detailed than for the log file.
    • Both utilities: the unknown options will be ignored.
  • Fixed:
    • XMLDiff: doubling the first attribute in XPath for the elements, if more than one attribute is used.
[0.2.23] - 2025-03-15
  • Fixed:
    • XMLDiff: fixed issue with the last sub element comparison (index out of range)
[0.2.22] - 2025-03-10
  • Improved
    • XMLDiff: The first attribute of elements in XPath will be always added to make a diff more clear.
    • XMLDiff: If one attribute is not enough to define the element, the next one will be added to the XPath, iteratively.
    • XMLDiff: Added the short description of integration with VSCode via RunXMLDiff.bat script and appropriate extension to make a diffs "on the fly" during editing XML files (modified ones).
[0.2.21] - 2025-03-03
  • Fixed
    • XMLDiff: fixed issue with text nodes differing
    • XMLDiff: improved "pos" definition logic
    • XMLDiff: improved logging information
  • Added
    • XMLPatch: added --allow-doubles option, useful for scripts patching
    • XMLPatch: added comments processing, now comments will be added to the resulting XML file
[0.2.20] - 2025-02-27
  • Fixed
    • XMLDiff: wrong attribute selection for the path
    • XMLDiff: fixed issue with not applied --append-to-log option
    • XMLDiff: fixed issue with wrong changed attributes count detection
    • XMLDiff: fixed usage or remove/add instead of replace for elements
    • XMLPatch: skip the diff file without diff elements
  • Improved
    • Both utilities: --log-to-file option now requires a log level (error, warn, info, debug) as a parameter
[0.2.17] - 2025-02-25
  • Fixed
    • Added checks to prevent duplicate elements during addition
[0.2.16] - 2025-02-25
  • Fixed
    • Fixed issue with element replacements
  • Improved
    • Logging information
[0.2.15] - 2025-02-24
  • Fixed
    • Fixed loading the diff.xsd
    • Fixed issue, if resulting file has to be located in current folder
  • Improved
Logging information, especially about wrong sel value. More info logged about processed XML elements.
  • Added
    • Possibility to append into existing debug log
[0.2.14] - 2025-01-17
  • Added
    • First public version coded in C#.
Last edited by ChemODun on Mon, 16. Jun 25, 12:33, edited 4 times in total.
Multiply entropy by absolute zero

Freedom in space
User avatar
Duncaroos
Posts: 715
Joined: Wed, 4. Jan 12, 22:23
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by Duncaroos »

Hi, first - this tool looks great and I believe what I need to help assist in handling mod support for my logistics tool.

One small request - could the execution files pause before closing so I can see any notes/errors/warnings that came out of say patching an XML file? The program window immediately closes after providing all the inputs, and I do not see the new patched file so I assume something went wrong but cannot diagnose.
Playing X4+All_DLC on:
CPU: Ryzen 5 5600X; RAM: 4x8GB DDR4 3200MHz; GPU: GTX 1070 8GB, Driver v536.23, DirectX 12.0; OS: Win10 Home 22H2 (19045.4780); Monitor: Single Acer S232HL 1920x1080

Duncaroo's Empire Logistics Tool (v0.23 Beta) - {{Vanilla Economy - Direct link}} {{Economy Overhaul Mod Version - Direct link}}
User avatar
ChemODun
Posts: 414
Joined: Mon, 12. Feb 07, 21:58
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by ChemODun »

Duncaroos wrote: Tue, 28. Jan 25, 19:07 Hi, first - this tool looks great and I believe what I need to help assist in handling mod support for my logistics tool.

One small request - could the execution files pause before closing so I can see any notes/errors/warnings that came out of say patching an XML file? The program window immediately closes after providing all the inputs, and I do not see the new patched file so I assume something went wrong but cannot diagnose.
Thank you. Understand the issue.
I get used to use console, and when run it from console nothing is closed.
But will think how to do it best way
Multiply entropy by absolute zero

Freedom in space
h2o.Ava
Posts: 349
Joined: Sun, 12. Oct 08, 04:04
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by h2o.Ava »

created an addon for this. it processes all enabled mods and outputs the final modified files:
viewtopic.php?t=469405
User avatar
Duncaroos
Posts: 715
Joined: Wed, 4. Jan 12, 22:23
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by Duncaroos »

ChemODun wrote: Sat, 8. Feb 25, 21:42
Duncaroos wrote: Tue, 28. Jan 25, 19:07 Hi, first - this tool looks great and I believe what I need to help assist in handling mod support for my logistics tool.

One small request - could the execution files pause before closing so I can see any notes/errors/warnings that came out of say patching an XML file? The program window immediately closes after providing all the inputs, and I do not see the new patched file so I assume something went wrong but cannot diagnose.
Thank you. Understand the issue.
I get used to use console, and when run it from console nothing is closed.
But will think how to do it best way
I managed to get it to work using a .bat file per your example; I was trying to use the xml-patch.exe file directly and kept complaining that it could not find the diff.xsd file even though it was in the same directory as xml-patch.exe.
Playing X4+All_DLC on:
CPU: Ryzen 5 5600X; RAM: 4x8GB DDR4 3200MHz; GPU: GTX 1070 8GB, Driver v536.23, DirectX 12.0; OS: Win10 Home 22H2 (19045.4780); Monitor: Single Acer S232HL 1920x1080

Duncaroo's Empire Logistics Tool (v0.23 Beta) - {{Vanilla Economy - Direct link}} {{Economy Overhaul Mod Version - Direct link}}
Venomaus
Posts: 38
Joined: Thu, 20. Dec 18, 22:23
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by Venomaus »

How exactly do you patch a dlc file onto the base game file?
For me it generates a patched xml file but the content doesn't really make sense from what was in the dlc diff.

For example if you patch ego_dlc_split\libraries\wares.xml onto the base game libraries\wares.xml
And you open the output file, I don't even see some wares like cheltmeat being added, also the owner faction="split" nodes are being put underneath the </ware> close tags and stuff?
Am I doing something wrong?
User avatar
ChemODun
Posts: 414
Joined: Mon, 12. Feb 07, 21:58
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by ChemODun »

Venomaus wrote: Sat, 22. Feb 25, 22:01 How exactly do you patch a dlc file onto the base game file?
For me it generates a patched xml file but the content doesn't really make sense from what was in the dlc diff.

For example if you patch ego_dlc_split\libraries\wares.xml onto the base game libraries\wares.xml
And you open the output file, I don't even see some wares like cheltmeat being added, also the owner faction="split" nodes are being put underneath the </ware> close tags and stuff?
Am I doing something wrong?
Let's check the wares.xml.zip, I made it by other version of tool, from x4_XMLDiffAndPatch
Last edited by ChemODun on Sat, 22. Feb 25, 23:53, edited 1 time in total.
Multiply entropy by absolute zero

Freedom in space
User avatar
Duncaroos
Posts: 715
Joined: Wed, 4. Jan 12, 22:23
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by Duncaroos »

Venomaus wrote: Sat, 22. Feb 25, 22:01 How exactly do you patch a dlc file onto the base game file?
For me it generates a patched xml file but the content doesn't really make sense from what was in the dlc diff.

For example if you patch ego_dlc_split\libraries\wares.xml onto the base game libraries\wares.xml
And you open the output file, I don't even see some wares like cheltmeat being added, also the owner faction="split" nodes are being put underneath the </ware> close tags and stuff?
Am I doing something wrong?
I just tried this out using your trial example too, and seems the patching is not working. Some "faction owner" attributes are also not being put underneath the ware and is posted outside the ware itself. Reviewing the .bat output notes there does not seem to be any error, but there is not enough detail to see if it skipped over something (e.g., chelt meat)
Playing X4+All_DLC on:
CPU: Ryzen 5 5600X; RAM: 4x8GB DDR4 3200MHz; GPU: GTX 1070 8GB, Driver v536.23, DirectX 12.0; OS: Win10 Home 22H2 (19045.4780); Monitor: Single Acer S232HL 1920x1080

Duncaroo's Empire Logistics Tool (v0.23 Beta) - {{Vanilla Economy - Direct link}} {{Economy Overhaul Mod Version - Direct link}}
User avatar
ChemODun
Posts: 414
Joined: Mon, 12. Feb 07, 21:58
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by ChemODun »

Duncaroos wrote: Sat, 22. Feb 25, 17:43
ChemODun wrote: Sat, 8. Feb 25, 21:42
Duncaroos wrote: Tue, 28. Jan 25, 19:07 Hi, first - this tool looks great and I believe what I need to help assist in handling mod support for my logistics tool.

One small request - could the execution files pause before closing so I can see any notes/errors/warnings that came out of say patching an XML file? The program window immediately closes after providing all the inputs, and I do not see the new patched file so I assume something went wrong but cannot diagnose.
Thank you. Understand the issue.
I get used to use console, and when run it from console nothing is closed.
But will think how to do it best way
I managed to get it to work using a .bat file per your example; I was trying to use the xml-patch.exe file directly and kept complaining that it could not find the diff.xsd file even though it was in the same directory as xml-patch.exe.
Please try another version, from other repo - x4_XMLDiffAndPatch
Multiply entropy by absolute zero

Freedom in space
User avatar
ChemODun
Posts: 414
Joined: Mon, 12. Feb 07, 21:58
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by ChemODun »

Duncaroos wrote: Sat, 22. Feb 25, 23:50
Venomaus wrote: Sat, 22. Feb 25, 22:01 How exactly do you patch a dlc file onto the base game file?
For me it generates a patched xml file but the content doesn't really make sense from what was in the dlc diff.

For example if you patch ego_dlc_split\libraries\wares.xml onto the base game libraries\wares.xml
And you open the output file, I don't even see some wares like cheltmeat being added, also the owner faction="split" nodes are being put underneath the </ware> close tags and stuff?
Am I doing something wrong?
I just tried this out using your trial example too, and seems the patching is not working. Some "faction owner" attributes are also not being put underneath the ware and is posted outside the ware itself. Reviewing the .bat output notes there does not seem to be any error, but there is not enough detail to see if it skipped over something (e.g., chelt meat)
Please post the more details, on which files it is doesn't work
Previously I posted the result of patch applying - what is wrong with that file?
Multiply entropy by absolute zero

Freedom in space
User avatar
ChemODun
Posts: 414
Joined: Mon, 12. Feb 07, 21:58
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by ChemODun »

Removed links on the old version.
Ready to fix bugs in current one
Multiply entropy by absolute zero

Freedom in space
User avatar
Duncaroos
Posts: 715
Joined: Wed, 4. Jan 12, 22:23
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by Duncaroos »

ChemODun wrote: Sat, 22. Feb 25, 23:53
Duncaroos wrote: Sat, 22. Feb 25, 23:50
Venomaus wrote: Sat, 22. Feb 25, 22:01 How exactly do you patch a dlc file onto the base game file?
For me it generates a patched xml file but the content doesn't really make sense from what was in the dlc diff.

For example if you patch ego_dlc_split\libraries\wares.xml onto the base game libraries\wares.xml
And you open the output file, I don't even see some wares like cheltmeat being added, also the owner faction="split" nodes are being put underneath the </ware> close tags and stuff?
Am I doing something wrong?
I just tried this out using your trial example too, and seems the patching is not working. Some "faction owner" attributes are also not being put underneath the ware and is posted outside the ware itself. Reviewing the .bat output notes there does not seem to be any error, but there is not enough detail to see if it skipped over something (e.g., chelt meat)
Please post the more details, on which files it is doesn't work
Previously I posted the result of patch applying - what is wrong with that file?
Using wares.xml from the base game as original, and split vendetta as the diff wares.xml. I'll post a zip to the files later today after I double check I'm using your latest version
Playing X4+All_DLC on:
CPU: Ryzen 5 5600X; RAM: 4x8GB DDR4 3200MHz; GPU: GTX 1070 8GB, Driver v536.23, DirectX 12.0; OS: Win10 Home 22H2 (19045.4780); Monitor: Single Acer S232HL 1920x1080

Duncaroo's Empire Logistics Tool (v0.23 Beta) - {{Vanilla Economy - Direct link}} {{Economy Overhaul Mod Version - Direct link}}
User avatar
Duncaroos
Posts: 715
Joined: Wed, 4. Jan 12, 22:23
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by Duncaroos »

ChemODun wrote: Sun, 23. Feb 25, 00:14 Removed links on the old version.
Ready to fix bugs in current one
OK i tested to the latest v0.2.14 version from your github.

example files here, with a snapshot of my folder structure on Windows 10: https://drive.google.com/drive/folders/ ... sp=sharing

Looks like the new version fixed the merge issues (haven't gone through it fully, but I see the split wares and previous "faction owner" attributes are back inside the ware element).

One thing I noticed, is if I have all the files in the same directory, I still need to specify the diff.xsd file, and I need to specify the full path of the xml file even though for the test it was going to be in the same directory.

Tryout 1 - no xsd definition, no output path specifed:

Code: Select all

XMLPatch.exe -o "base wares.xml" -d "SV diff wares.xml" -u BaseAndSV_wares.xml -l true

Code: Select all

2025-02-23 08:39:14.9625 Warn diff.xsd file does not exist:  
2025-02-23 08:39:14.9784 Info Processing single trio of files. 
2025-02-23 08:39:14.9784 Error Failed to determine the directory for outputXmlPath. 
Tryout 2 - xsd definition, no output path specified:

Code: Select all

XMLPatch.exe -x diff.xsd -o "base wares.xml" -d "SV diff wares.xml" -u BaseAndSV_wares.xml -l true

Code: Select all

2025-02-23 08:40:16.1238 Info Using diff.xsd path: diff.xsd 
2025-02-23 08:40:16.1581 Info Processing single trio of files. 
2025-02-23 08:40:16.1581 Error Failed to determine the directory for outputXmlPath. 
Tryout 3 - xsd definition, output path specified:

Code: Select all

XMLPatch.exe -x diff.xsd -o "base wares.xml" -d "SV diff wares.xml" -u "E:\Users\Dunc\OneDrive\Hobbies\Games\X4 Foundations\Empire Logistics Tool\Coding\x4_xml_diff_and_patch\BaseAndSV_wares.xml" -l true

Code: Select all

No errors to show
Comment 1: your initial notes indicate if you are having everything in the same directory, file paths are not required (for xsd or xmls, so request this get looked at).

Comment 2: Would be nice if when adding wares to the original xml, it goes into more detail on what the 'ware' or 'element' is:

Code: Select all

2025-02-23 08:41:04.2495 Debug Applying add operation: /wares at append 
2025-02-23 08:41:04.2495 Info Appended new element 'ware' to 'wares'. 
Comment 3: same as comment 2, but for attributes added/changed:

Code: Select all

2025-02-23 08:41:04.2799 Debug Applying add operation: /wares/ware[@id='module_arg_dock_m_01'] at append 
2025-02-23 08:41:04.2956 Info Appended new element 'owner' to 'ware'. 
Playing X4+All_DLC on:
CPU: Ryzen 5 5600X; RAM: 4x8GB DDR4 3200MHz; GPU: GTX 1070 8GB, Driver v536.23, DirectX 12.0; OS: Win10 Home 22H2 (19045.4780); Monitor: Single Acer S232HL 1920x1080

Duncaroo's Empire Logistics Tool (v0.23 Beta) - {{Vanilla Economy - Direct link}} {{Economy Overhaul Mod Version - Direct link}}
User avatar
Duncaroos
Posts: 715
Joined: Wed, 4. Jan 12, 22:23
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by Duncaroos »

Example 2: https://drive.google.com/drive/folders/ ... sp=sharing

The log file is not reporting replace / remove actions, making it difficult to assess why later I am getting the following error lines when running the last diff file ("diff wares mod - SWI.xml", coming from SWI mod v0.85):

Code: Select all

2025-02-23 14:10:49.0006 Warn No nodes found for replace selector: /wares/ware[@id='module_gen_prod_claytronics_01']/production[@method='default']/primary/ware[@ware='claytronics']/@amount
2025-02-23 14:10:49.0006 Warn No nodes found for remove selector: /wares/ware[@id='module_gen_prod_claytronics_01']/production[@method='default']/primary/ware[@ware='hullparts']
2025-02-23 14:10:49.0213 Warn No nodes found for replace selector: /wares/ware[@id='module_gen_prod_hullparts_01']/production[@method='default']/primary/ware[@ware='claytronics']/@amount
2025-02-23 14:10:49.0213 Warn No nodes found for remove selector: /wares/ware[@id='module_gen_prod_hullparts_01']/production[@method='default']/primary/ware[@ware='hullparts']
This is likely something with the SWI mod patch to game version v7.5 not accounting for the change in module construction wares, but it is unclear whether the XMLPatch.exe performed the other necessary replace/remove diff commands, or it just skipped them.
Playing X4+All_DLC on:
CPU: Ryzen 5 5600X; RAM: 4x8GB DDR4 3200MHz; GPU: GTX 1070 8GB, Driver v536.23, DirectX 12.0; OS: Win10 Home 22H2 (19045.4780); Monitor: Single Acer S232HL 1920x1080

Duncaroo's Empire Logistics Tool (v0.23 Beta) - {{Vanilla Economy - Direct link}} {{Economy Overhaul Mod Version - Direct link}}
User avatar
ChemODun
Posts: 414
Joined: Mon, 12. Feb 07, 21:58
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by ChemODun »

Duncaroos wrote: Sun, 23. Feb 25, 14:43
ChemODun wrote: Sun, 23. Feb 25, 00:14 Removed links on the old version.
Ready to fix bugs in current one
OK i tested to the latest v0.2.14 version from your github.

example files here, with a snapshot of my folder structure on Windows 10: https://drive.google.com/drive/folders/ ... sp=sharing

Looks like the new version fixed the merge issues (haven't gone through it fully, but I see the split wares and previous "faction owner" attributes are back inside the ware element).

One thing I noticed, is if I have all the files in the same directory, I still need to specify the diff.xsd file, and I need to specify the full path of the xml file even though for the test it was going to be in the same directory.

Comment 1: your initial notes indicate if you are having everything in the same directory, file paths are not required (for xsd or xmls, so request this get looked at).
Thanks. I get uses to have it in different folders, that's why not checked your "user story" :-)
Will fix it
Duncaroos wrote: Sun, 23. Feb 25, 14:43 Comment 2: Would be nice if when adding wares to the original xml, it goes into more detail on what the 'ware' or 'element' is:

Code: Select all

2025-02-23 08:41:04.2495 Debug Applying add operation: /wares at append 
2025-02-23 08:41:04.2495 Info Appended new element 'ware' to 'wares'. 
Comment 3: same as comment 2, but for attributes added/changed:

Code: Select all

2025-02-23 08:41:04.2799 Debug Applying add operation: /wares/ware[@id='module_arg_dock_m_01'] at append 
2025-02-23 08:41:04.2956 Info Appended new element 'owner' to 'ware'. 
Ok. Fully agree.
Multiply entropy by absolute zero

Freedom in space
User avatar
ChemODun
Posts: 414
Joined: Mon, 12. Feb 07, 21:58
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by ChemODun »

Duncaroos wrote: Sun, 23. Feb 25, 20:14 Example 2: https://drive.google.com/drive/folders/ ... sp=sharing

The log file is not reporting replace / remove actions, making it difficult to assess why later I am getting the following error lines when running the last diff file ("diff wares mod - SWI.xml", coming from SWI mod v0.85):

Code: Select all

2025-02-23 14:10:49.0006 Warn No nodes found for replace selector: /wares/ware[@id='module_gen_prod_claytronics_01']/production[@method='default']/primary/ware[@ware='claytronics']/@amount
2025-02-23 14:10:49.0006 Warn No nodes found for remove selector: /wares/ware[@id='module_gen_prod_claytronics_01']/production[@method='default']/primary/ware[@ware='hullparts']
2025-02-23 14:10:49.0213 Warn No nodes found for replace selector: /wares/ware[@id='module_gen_prod_hullparts_01']/production[@method='default']/primary/ware[@ware='claytronics']/@amount
2025-02-23 14:10:49.0213 Warn No nodes found for remove selector: /wares/ware[@id='module_gen_prod_hullparts_01']/production[@method='default']/primary/ware[@ware='hullparts']
This is likely something with the SWI mod patch to game version v7.5 not accounting for the change in module construction wares, but it is unclear whether the XMLPatch.exe performed the other necessary replace/remove diff commands, or it just skipped them.
It means skipped, but will add this information to the log
Multiply entropy by absolute zero

Freedom in space
User avatar
ChemODun
Posts: 414
Joined: Mon, 12. Feb 07, 21:58
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by ChemODun »

Duncaroos wrote: Sun, 23. Feb 25, 20:14 Example 2: https://drive.google.com/drive/folders/ ... sp=sharing

The log file is not reporting replace / remove actions, making it difficult to assess why later I am getting the following error lines when running the last diff file ("diff wares mod - SWI.xml", coming from SWI mod v0.85):

Code: Select all

2025-02-23 14:10:49.0006 Warn No nodes found for replace selector: /wares/ware[@id='module_gen_prod_claytronics_01']/production[@method='default']/primary/ware[@ware='claytronics']/@amount
2025-02-23 14:10:49.0006 Warn No nodes found for remove selector: /wares/ware[@id='module_gen_prod_claytronics_01']/production[@method='default']/primary/ware[@ware='hullparts']
2025-02-23 14:10:49.0213 Warn No nodes found for replace selector: /wares/ware[@id='module_gen_prod_hullparts_01']/production[@method='default']/primary/ware[@ware='claytronics']/@amount
2025-02-23 14:10:49.0213 Warn No nodes found for remove selector: /wares/ware[@id='module_gen_prod_hullparts_01']/production[@method='default']/primary/ware[@ware='hullparts']
This is likely something with the SWI mod patch to game version v7.5 not accounting for the change in module construction wares, but it is unclear whether the XMLPatch.exe performed the other necessary replace/remove diff commands, or it just skipped them.
Now it's looks like, using quote, due to long lines
2025-02-24 12:58:55.6449 Warn No nodes found for replace selector: /wares/ware[@id='module_gen_prod_claytronics_01']/production[@method='default']/primary/ware[@ware='claytronics']/@amount! Existing only: /wares/ware[@id='module_gen_prod_claytronics_01']/production[@method='default']/primary. Skipping operation.
2025-02-24 12:58:55.6449 Warn No nodes found for remove selector: /wares/ware[@id='module_gen_prod_claytronics_01']/production[@method='default']/primary/ware[@ware='hullparts']! Existing only: /wares/ware[@id='module_gen_prod_claytronics_01']/production[@method='default']/primary. Skipping operation.
2025-02-24 12:58:55.7059 Warn No nodes found for replace selector: /wares/ware[@id='module_gen_prod_hullparts_01']/production[@method='default']/primary/ware[@ware='claytronics']/@amount! Existing only: /wares/ware[@id='module_gen_prod_hullparts_01']/production[@method='default']/primary. Skipping operation.
2025-02-24 12:58:55.7059 Warn No nodes found for remove selector: /wares/ware[@id='module_gen_prod_hullparts_01']/production[@method='default']/primary/ware[@ware='hullparts']! Existing only: /wares/ware[@id='module_gen_prod_hullparts_01']/production[@method='default']/primary. Skipping operation.
Multiply entropy by absolute zero

Freedom in space
User avatar
ChemODun
Posts: 414
Joined: Mon, 12. Feb 07, 21:58
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by ChemODun »

0.2.15 (2025-02-24)
  • Fixed loading the diff.xsd
  • Fixed issue, if resulting file has to be located in current folder
  • Improved logging information, especially about wrong sel value. More info logged about processed XML elements.
  • Added possibility to append into existing debug log
Multiply entropy by absolute zero

Freedom in space
User avatar
Duncaroos
Posts: 715
Joined: Wed, 4. Jan 12, 22:23
x4

Re: [TOOLS] XML diff and patch for X4: Foundations

Post by Duncaroos »

Great! Thanks for the improvements thus far.

Few comments:
  1. Is it possible for the log/executable to record replace / remove actions that were completed?
  2. The append to log command [-a true] seems to not be working. When I ran example 2 with this new command line, the log only shows the first execution and nothing gets appended.

    Code: Select all

    XMLPatch.exe -x diff.xsd -o "Base wares (v7.5).xml" -d "diff wares - SV.xml" -u "{dir}\ware data\Tool\tmp\New folder\tmp_wares.xml"
    XMLPatch.exe -x diff.xsd -o "tmp_wares.xml" -d "diff wares - COH.xml" -u "{dir}\ware data\Tool\tmp\New folder\tmp_wares.xml"
    XMLPatch.exe -x diff.xsd -o "tmp_wares.xml" -d "diff wares - TOA.xml" -u "{dir}\ware data\Tool\tmp\New folder\tmp_wares.xml"
    XMLPatch.exe -x diff.xsd -o "tmp_wares.xml" -d "diff wares - KE.xml" -u "{dir}\ware data\Tool\tmp\New folder\tmp_wares.xml"
    XMLPatch.exe -x diff.xsd -o "tmp_wares.xml" -d "diff wares - TL.xml" -u "{dir}\ware data\Tool\tmp\New folder\tmp_wares.xml"
    XMLPatch.exe -x diff.xsd -o "tmp_wares.xml" -d "diff wares - MINI_01.xml" -u "{dir}\ware data\Tool\tmp\New folder\tmp_wares.xml"
    XMLPatch.exe -x diff.xsd -o "tmp_wares.xml" -d "diff wares mod - SWI.xml" -u "{dir}\ware data\Tool\tmp\New folder\SWIwares_complete.xml"
    Changed to (just adding -l true and -a true commands):

    Code: Select all

    XMLPatch.exe -x diff.xsd -o "Base wares (v7.5).xml" -d "diff wares - SV.xml" -u "{dir}\ware data\Tool\tmp\New folder\tmp_wares.xml" -l true
    XMLPatch.exe -x diff.xsd -o "tmp_wares.xml" -d "diff wares - COH.xml" -u "{dir}\ware data\Tool\tmp\New folder\tmp_wares.xml" -a true
    XMLPatch.exe -x diff.xsd -o "tmp_wares.xml" -d "diff wares - TOA.xml" -u "{dir}\ware data\Tool\tmp\New folder\tmp_wares.xml" -a true
    XMLPatch.exe -x diff.xsd -o "tmp_wares.xml" -d "diff wares - KE.xml" -u "{dir}\ware data\Tool\tmp\New folder\tmp_wares.xml" -a true
    XMLPatch.exe -x diff.xsd -o "tmp_wares.xml" -d "diff wares - TL.xml" -u "{dir}\ware data\Tool\tmp\New folder\tmp_wares.xml" -a true
    XMLPatch.exe -x diff.xsd -o "tmp_wares.xml" -d "diff wares - MINI_01.xml" -u "{dir}\ware data\Tool\tmp\New folder\tmp_wares.xml" -a true
    XMLPatch.exe -x diff.xsd -o "tmp_wares.xml" -d "diff wares mod - SWI.xml" -u "{dir}\ware data\Tool\tmp\New folder\SWIwares_complete.xml" -a true
    *I did check if -x diff.xsd could be removed and it worked great!
    .
  3. I wanted to try out the directory option for XMLPatch, and I came across this error:

    Code: Select all

    Warn Original file does not exist for diff file 'C:\Users\Dunc\Desktop\Example 3\SWI\V0.85\ware data\Tool\diff\diff wares - COH.xml'. Skipping.
    I believe I have all the original, diff, and output directories set correctly, but could you let me know what's the issue? I added a few notes to the .bat binary file to explain my madness.
    https://drive.google.com/drive/folders/ ... sp=sharing
    .
  4. Working more with the SWI mod wares.xml diff, it seems that the patch is not handling replacement of sub-elements correctly:
    Original:

    Code: Select all

    <wares>
      <ware id="module_gen_prod_hullparts_01" name="{20104,11301}" description="{20104,11302}" transport="container" volume="1" tags="module">
        <price min="3340194" average="3929640" max="4519086" />
        <production time="862" amount="1" method="default" name="{20206,101}">
          <primary>
            <ware ware="advancedcomposites" amount="853" />
            <ware ware="energycells" amount="1357" />
            <ware ware="plasmaconductors" amount="1228" />
          </primary>
        </production>
        <research time="10">
          <research>
            <ware ware="research_module_production" />
          </research>
        </research>
        <component ref="prod_gen_hullparts_macro" />
        <restriction licence="station_gen_basic" />
        <owner faction="antigone" />
        <owner faction="argon" />
        <owner faction="holyorder" />
        <owner faction="paranid" />
        <owner faction="teladi" />
        <owner faction="trinity" />
      </ware>
    </wares>
    Diff:

    Code: Select all

    <diff>
      <replace sel="/wares/ware[@id='module_gen_prod_hullparts_01']/production">
          <production time="862" amount="1" method="default" name="{20206,101}">
          <primary>
            <ware ware="claytronics" amount="1200" />
            <ware ware="energycells" amount="2400" />
          </primary>
        </production>
      </replace>
    </diff>
    Expected Result:

    Code: Select all

    <ware id="module_gen_prod_hullparts_01" name="{20104,11301}" description="{20104,11302}" transport="container" volume="1" tags="module">
        <price min="3340194" average="3929640" max="4519086" />
        <production time="862" amount="1" method="default" name="{20206,101}">
          <primary>
            <ware ware="claytronics" amount="1200" />
            <ware ware="energycells" amount="2400" />
          </primary>
        </production>
        ...
    XMLPatch v0.2.15 Result (no change):

    Code: Select all

    <ware id="module_gen_prod_hullparts_01" name="{20104,11301}" description="{20104,11302}" transport="container" volume="1" tags="module">
        <price min="3340194" average="3929640" max="4519086" />
        <production time="862" amount="1" method="default" name="{20206,101}">
          <primary>
            <ware ware="advancedcomposites" amount="853" />
            <ware ware="energycells" amount="1357" />
            <ware ware="plasmaconductors" amount="1228" />
          </primary>
        </production>
        ...
Playing X4+All_DLC on:
CPU: Ryzen 5 5600X; RAM: 4x8GB DDR4 3200MHz; GPU: GTX 1070 8GB, Driver v536.23, DirectX 12.0; OS: Win10 Home 22H2 (19045.4780); Monitor: Single Acer S232HL 1920x1080

Duncaroo's Empire Logistics Tool (v0.23 Beta) - {{Vanilla Economy - Direct link}} {{Economy Overhaul Mod Version - Direct link}}

Return to “X4: Foundations - Scripts and Modding”