[TOOL] X3 Customizer 3.16 (added FL support)

The place to discuss scripting and game modifications for X³: Terran Conflict and X³: Albion Prelude.

Moderators: Moderators for English X Forum, Scripting / Modding Moderators

Post Reply
SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Post by SirNukes » Sat, 23. Dec 17, 23:12

Here are some steps to follow, off the top of my head:

1) Copy the User_Transforms_Example.py file and name it something unique, or name it "User_Transforms.py", which is the default the customizer will look for. I will never put a file by that name on the git repository, so it should be safe against updates.

2) Edit the Set_Path call to change the directory where the X3:AP addon folder is located for your system. (This could point somewhere else, like some test directory, as long as it has the expected folder structure.)

3) You can optionally rename the source_folder from whatever default I have in there.

4) In the AP addon folder, make the subfolder with the source_folder name from (3) and start populating it with the files which are to be modified.

5) The easiest way to pull files, with any other mods applied, is to use the X3 Editor 2 virtual file system (VFS), which can be brought up through plugings->CAT Manager. This will show the game files according to highest priority. (This seems to be a little buggy when extracting director scripts and not giving them their full name, but the text files in the types folder should work fine.)

6) Fill in the files according to the requirements of the transforms you want to run. From the other thread, this would likely be types/Jobs.txt, types/TShips.txt, and types/WareLists.txt. When placing these into the source_folder, use the same directory structure relative to the addon folder, eg. put them in addon/source_folder/types.

7) You can run a quick test at this point: comment out any transforms in your User_Transforms.py file and just leave the Set_Path active (or return right after Set_Path), and run the customizer using X3_Customizer.py, optionally providing the name of your python file if it was named something other than User_Transforms.py. (Use the command line for this; shift-right clicking in the windows folder explorer should have a "open command window here" option to easily put it in the right directory.) The customizer should run succesfully and put copies of the files from the source_folder into the normal addon directory (without any edits applied).

8) With that working, you can start playing around with the different transforms. Syntax errors in transform calls will generally show up as python exceptions. Transforms with missing requirements should print a nice warning message and get skipped.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Post by SirNukes » Wed, 27. Dec 17, 07:14

Updated to 2.17.

The major new addition is Add_More_Factory_Sizes, a transform which will fill out the factory lists from small to XL. Parameters can be used to limit the sizes and types generated, as well as to change how the cost is scaled, either using linear scaling or using scaling similar to solar power plants. Generated factories are added to appropriate shipyards using a generated director script, since I was having trouble figuring out how to do it using standard scripts (due to factories sharing names across races). The director script will fire automatically on loading a save game, and may take a few seconds to complete.

(Note: at time of this upload I have limited playing time with the new factories, though initial tests all looked good.)

With new knowledge on writing director scripts, the Add_Ship_Variants transform was switched to also using a director script, and no longer requires manually calling a script using the in-game script editor to update shipyards. Warning: this will fire on game load, and will freeze the game for a few seconds while it runs. This will only happen once.

If unwanted variants are present in a game, they can be soft removed using Remove_Ship_Variants, which will set their race to None (should stop random generation) and will remove them from shipyards. The variants will be left in the tships file to avoid game crashes and bugs, and any existing ships of those variants will continue to fly around.


Other changes are mostly aimed at script edits. Complex_Cleaner_Bug_Fix will clean out a couple bugs in the 4.09 version of that mod (latest that I am aware of). Complex_Cleaner_Use_Small_Cube is an edit to force selection of the smallest factory cube, which is mainly motivated by my personal preference for a smaller visual footprint.

Fleet_Interceptor_Bug_Fix will clear out a bug I noticed in the fleet logic, where a misspelling of 'interecept' when selecting ships to launch at enemy bombers appears to cause a default M6 to be sent instead.

Change_Sector_Music is a small, generic transform to simplify switching the music for a single sector.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Post by SirNukes » Mon, 1. Jan 18, 11:05

Updated to 2.19. Version 2.18 had some bug fixes for the generated director scripts, notably to ensure they can run in a new game, since I haven't gotten around to editing the universe files to stick new ships and factories on default ware lists.

Version 2.19 adds support for editing game .obj files. This evening I learned how to decompile those and figured out the general behavior of the assembly code, well enough to make some fun edits. New transforms are focused on adjusting SETA behavior, with an extra tweak to change marine counts.

Adjust_Max_Seta will let you unlock the Seta factor, with one argument for your desired cap in the menu. I put a soft limit of 50x on it, since that is the game's debug mode limit, but you can always edit that out if 50x isn't crazy enough for you.

Adjust_Max_Speedup_Rate changes the rate at when the game enters Seta mode. A factor of 2 will halve the time from pressing the key to reaching the full Seta multiplier. In practice, I expect this to be limited by framerate, and estimate it will top out around 15x faster at 60 fps.

Stop_Events_From_Disabling_Seta will stop Seta from turning off in a few cases, selectable in the transform arguments. These include: when a missile is fired at your ship (very annoying when just wanting to traverse a pirate sector), when nearing another object, and when a priority message pops up (eg. police scan notice). (I didn't test that last one because of lazy police, but it should work.)

Set_Max_Marines allows you to change the marine cap on different classes of ships. The practical upper limit is 127 marines (1 byte signed, needs to be positive). The ship class groupings are: TM, TP, M6, capitals (M1, M2, M7, TL), and the Sirokos (or whatever ship is at entry 263 in Tships). Adding marines to other ship types is unlikely at this time, but may be possible if there is a way to do it without changing the number of bytes of assembly code in relevant sections (since changing byte count messes up pointers in the obj file, and I don't want to deal with fixing that).


As an aside, if anyone had any transform requests, feel free to post them. Also, if anyone wanted to help with the tool development, I am open to giving github permissions.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Post by SirNukes » Wed, 3. Jan 18, 05:20

Updated to 2.20. This adds two sound related transforms.

Disable_Combat_Music will stop the game from transitioning to combat music when near enemies or missiles, when a missile is fired at the player, etc. Normal sector music will continue to play uninterrupted.

Remove_Combat_Beep will remove the beep that plays when entering combat mode by nearing an enemy.

As an aside, my main motivation for digging deeper into game modding was a hope to be able to address collision damage issues with the suicidal autopilot. Unfortunately, my tests so far lead me to believe that code might be buried in the exe, which is currently beyond my threshold for modding difficulty. If anyone knows more about changing collision damage, please let me know.

Also, the recent obj related transforms likely will break if applied to a game that has those files modded already, and may break if the game gets patched. It doesn't affect me personally for now, but give me a nudge if you want some more robust transforms to use with LU or similar.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Post by SirNukes » Sun, 7. Jan 18, 02:03

Updated to 2.21.

After more hours spent with the KC than I care to admit to, I have now added the Stop_GoD_From_Removing_Stations option. This will short circuit the function which analyzes factories in a sector and picks ones to delete. This is an experimental transform, though I ran a bunch of tests today (at 127x Seta) to try different code edits before being satisfied that this fix should work.

I wasn't able to fully determine the mechanic for station deletion once the factory analysis is done. It kicks off into intermediate steps to generate Event objects which in turn mess around with scripts. If anything down that path is responsible for behavior other than factory deletion, it may be affected as well. Let me know if you notice any side effects.

As before, this transform will not work if the x3story.obj file has already been changed significantly (enough to cause pointer changes). I am mostly thinking of LU here, which I personally haven't tried yet. Making the transform work on that mod shouldn't be much trouble now that I know what to edit (assuming LU hasn't already changed GoD), so just holler if you want that.


If a save game has already has had a bunch of stations removed, it may be possible to restore them by having a setup script do an info dump of still existing stations, comparing that against x3universe to find what is missing, and copying those stations to a secondary map file to be reloaded. I'm not sure if/when I might get to that.


Other changes in 2.21 include:
- Upping the softcap on Seta speed to 127 from 50, which is the max positive signed byte value. In practice, the game only runs at ~40x speed for me (based on the play timer), but you might do better with a strong CPU.
- Added Disable_Asteroid_Respawn, which should allow permanent removal of unwanted asteroids. I suggest only using this temporarily when knocking out a few asteroids, and not leaving it on long term.

Lone Machete
Posts: 29
Joined: Sun, 3. Dec 17, 06:01
x3ap

Post by Lone Machete » Sun, 7. Jan 18, 10:58

to stop god removement entirely is a bad idea as you can read in the relative topics since number of stations is limited and god is more likely to build up stations in some areas of the map than in others, designing a very poor universe with only one truely "prospering" area while the rest stays more or less empty.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Post by SirNukes » Mon, 8. Jan 18, 00:51

Can you provide links to such topics? I didn't run across any good documentation of GoD behavior in my searches; mostly it was just extrapolations based on individual experiences.

At any rate, the transform is reversible, for players to use as they see fit. Eg. turn off station removal when starting a new game, eventually place m5s at every station the player wants to keep, take a minute to turn station removal back on (save, rerun the customizer without the transform, reload), and let it go to work.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Post by SirNukes » Sun, 15. Apr 18, 03:46

Major update to v3.2.

Support for compiling to a standalone executable has been added, removing the python requirement. The binary release can be found on the github releases tab, as a zip file. Note: this is still a command line tool at the time, so don't be surprised if you double click to launch it and nothing happens. :)

Backwards compatibility may be slightly messy, as the older versions did not keep a log of which files they had written. Consider doing an empty run of the prior version first to undo its changes (using a command script with the correct path but no transforms). Alternatively, do at least one run on the new version before moving files out of the old sources folder.


The internal file management system has been reworked; it can now find source data from the standard game directory or from the cat/dat files, and no longer requires (though still supports) a high priority source folder. Existing files will be renamed (suffixed with .x3c.bak) when they would conflict with modified files; they have their name restored if the modified file is not created on the next run. A json log will be generated stating which files were written (with their sha256 hashes), as well as which renamed, to inform later runs.

The general source code structure of the project has been reorganized in a number of ways, though this is mostly transparent if you don't want to edit source. Various misc refinements have been made. The dependency on the scipy package has been removed, though if that package is available it can be used to create smoother scaling equations for transforms that support them.


As a bonus, an Allow_Valhalla_To_Jump_To_Gates transform has been added. Use at your own risk; I recommend some sort of ship size mod or gate mod to avoid crashes. This requires a KC edit, so currently will not work on mods that edit x3story.obj (eg. LU).

Thanks to akruppa (https://forum.egosoft.com/viewtopic.php?t=361709) for an example on how to decode the cat/dat pairs.

User avatar
Joubarbe
Posts: 4796
Joined: Tue, 31. Oct 06, 12:11
xr

Post by Joubarbe » Sun, 15. Apr 18, 05:14

Nice!

User avatar
RoverTX
Posts: 1426
Joined: Wed, 16. Nov 11, 18:37
x4

Post by RoverTX » Tue, 17. Apr 18, 06:38

Lone Machete wrote:to stop god removement entirely is a bad idea as you can read in the relative topics since number of stations is limited and god is more likely to build up stations in some areas of the map than in others, designing a very poor universe with only one truely "prospering" area while the rest stays more or less empty.
What relative topics? I also have seen people say this alot, but I have never seen GoD actually add in any stations to fill missing gaps. It just seems to cull stations.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Post by SirNukes » Sun, 10. Jun 18, 07:47

Updated to 3.3. This fixes a bug that snuck into Adjust_Weapon_Fire_Rate (thanks andu for catching it), adds some documentation on which transforms have been tested to not work with LU (mostly obj and script edits), and adds more graceful handling of any transform errors.

andu
Posts: 17
Joined: Thu, 21. Jun 07, 23:35
x4

Post by andu » Sun, 10. Jun 18, 21:51

Hi, i've downloaded the latest version and using this config file

Code: Select all

'''
Example for using the Customizer, setting a path to
the X3 directory and running some simple transforms.
'''

# Import all transform functions.
from Transforms import *

Set_Path(
    # Set the path to the X3 installation folder.
    path_to_addon_folder=r'[...]\x3 terran conflict\addon',
    # Optional subfolder with high priority source files to be modified.
    # By default, this is relative to the addon folder.
    #  source_folder='addon'
)

Adjust_Weapon_Fire_Rate(scaling_factor = 4)
It doesn't change the rate of fire.

command used

Code: Select all

X3_Customizer.exe User_Transforms.py

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Post by SirNukes » Sun, 10. Jun 18, 22:32

Can you report what gets printed to the console when you try to run the script? If it works correctly, the printout should look like:
Attempting to run User_Transforms.py
Successfully ran Adjust_Weapon_Fire_Rate
Transforms Complete
If there are pathing problems to the user script or to the game directory, those should print out error messages.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Post by SirNukes » Mon, 11. Jun 18, 10:58

Updated to 3.4. This changes how the obj file edits are carried out to use a dynamic pattern match instead of hard offsets, to make it more robust against different obj file layouts (due to patches or other mods).

Adjust_Max_Speedup_Rate and Stop_GoD_From_Removing_Stations (untested) now apply to LU, along with Adjust_Max_Seta (which redirects to a t file edit if the LU t file found). Most other obj patches don't really map to LU since it hardcodes many of the same changes.

Stop_Events_From_Disabling_Seta has a new option, 'on_frame_input', which will prevent seta from turning off when opening menus.

GageDragon
Posts: 59
Joined: Tue, 26. Apr 16, 01:21
x4

Post by GageDragon » Mon, 11. Jun 18, 22:58

Hi. Trying to figure this out but keep getting path error. I am using xrm and have set the path to the xrm addon folder and it works fine using the example_transforms but when I try using the authors_transform and change it to the same path I get the path error.

Exception: Path to the AP/addon folder appears invalid.
[1372] Failed to execute script X3_Customizer

This is copy straight from both files, as you can see the paths are the same.

example path, Set_Path(
# Set the path to the X3 installation folder.
path_to_addon_folder = r'G:\X3xrm\addon',

author path, elif XRM:
Set_Path(
path_to_x3_folder = r'G:\X3xrm\addon',


Any idea what I am doing wrong other than I am an idiot (this I already know lol). Have tried changing some other lines at the other 2 paths and I get other errors then like indent error or syntax.

edit: or do I need to set it to the base folder and not add on.. hmm I will try that now...

edit2: well that was progress but still gave an error after running alot of things

OSError: Not a gzipped file (b'Q\x86')
[6272] Failed to execute script X3_Customizer

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Post by SirNukes » Mon, 11. Jun 18, 23:25

Interesting. The path thing is as you say, taking either the x3 or addon folder, though I can touch up the error message a little bit. The gzip thing I will have to read up on. The x3 .pck files are gzipped versions of text files; when searching for the file to modify, if the tool finds a pck file then it uses the python gzip package to unpack it for reading/editing. I have never had trouble with it on my end, so I need to learn more about what gzip is doing to get an idea of what is going wrong for you. Since the file loader is outside my normal transform exception handler, the problem stops the entire tool run; that I could touch up as well.

GageDragon
Posts: 59
Joined: Tue, 26. Apr 16, 01:21
x4

Post by GageDragon » Mon, 11. Jun 18, 23:42

A couple things so you dont waste time looking into anything if its all my fault..
I notice when I run the authors_transform it is looking at python in programdata, which I do not have, it's after that the gzip error hits. I also do not have a source folder since I am lazy and have not opened xeditor yet and tried to figure out how to do that lol. If either of those are relevant.

Still it is odd that it partially runs at G:\X3xrm but not at all at G:\X3xrm\addon

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Post by SirNukes » Tue, 12. Jun 18, 01:27

3.4.1 has been pushed with some tweaks including nicer gzip error catching.


Just as a warning, the Authors_Transforms includes some options that are hard to roll back once added to the game if the game is saved, such as the factory and ship variants (which get added to shipyards). I mostly make that file available for extended examples of calling transforms.

For the game path, these should hopefully be functionally equivalent:
path_to_x3_folder = r'G:\X3xrm'
path_to_addon_folder = r'G:\X3xrm\addon'

For the ProgramData/Python thing, it might just be a relic of how the executable is packaged up using PyInstaller. At least, if I rename my python distribution folder, the packaged customizer still runs okay.

For the gzip issue, I tracked it down to the X3 Plugin Manager, which will generate a TWareT.pck file but compresses it with something other than gzip. My current best guess from digging around and looking at some source code is that older X games and tools used to use deflate through zlib, so my current theory is that the plugin manager is using deflate still. I am poking around looking for a working solution now.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Post by SirNukes » Tue, 12. Jun 18, 03:15

3.4.2 is now up, and should support the X3 Plugin Manager generated pck file.

After spending some time going through x3 editor source code looking for how it handles decompression, and then the x2fd.dll source code (thanks doubleshadow for making source available), I found that the plugin manager appears to use an x2 pck format. (My theory about deflate instead of gzip was wrong.) Anyway, the customizer now tries x2 style decompression as a fallback when direct gzip fails.

GageDragon
Posts: 59
Joined: Tue, 26. Apr 16, 01:21
x4

Post by GageDragon » Tue, 12. Jun 18, 03:36

Aha will try 3.4.2 then. Thanks for this.

Was just coming on to say I kinda found a work around for the gzip thing in 3.4.1 by deleting the file created by plugin manager then running customizer and then rerunning plugin manager so it recreates that file. No error then. It might even be better off this way if customizer makes any ware size changes then plugin manager would need to recreate the file anyhow I believe.

Also not sure if I did it right but I grabbed the source and input_script files from the master and just put those into my folder. If that was correct then it is still erroring when I try g:\x3xrm\addon. I do see an x3_customizer_logs folder in addon so I guess it is working correctly regardless.

As far as changes being permanent I dont mind. I plan on starting over once I get this working anyway :D

I noticed in my rummaging through authors_transforms some settings for restoring ammo based weapons. I look forward to breaking my game more trying to figure that out next because I miss ammo weapons lol...

edit: well it still errors with the plugin file in folder as well as the path error, that is of course if I am doing things correctly by grabbing from the master on git and adding the changed files into my folder. Docu and readme both have 3.4.2 in them

edit2: grabbed the compiled version and it runs with no gzip error now but g:\x3xrm\addon now gives error Exception: Path to the AP/addon folder appears invalid(path: G:\X3xrm\addon\addon).. kinda funny but no not really lol

edit3: it is renaming the file to TWareT.pck.x3c.bak, maybe it really should be left alone and just rerun plugin manager after customizer?

Post Reply

Return to “X³: Terran Conflict / Albion Prelude - Scripts and Modding”