[Mod/Patch] X3-Unleashed [RC2] [Aug-06-2010] - X3 Performance gains up to 38%

[Locksley]

Hi will this work on the new 2.6 patch?

No. Well, this is for the 2.1 jobs. After 2.1 there has been updates to the Jobs file and this will undo those changes.

I don't know what those changes are so I can't comment if it would be easy or hard to adjust Jaga's file to become 2.6 compliant.

Cheers!

[Hartzaden]

well i will be following this now.

[Jaga_Telesin]

Just FYI - I am returning to X3:TC for a while, and will be updating Unleashed to be TC v2.1+ compliant (through the current v2.7 and future releases of X3:TC).

I am in the process of gathering old information, data, and feedback, and will be publishing a Release Candidate in the near future. If anyone is interested in helping to test this (a focused test aimed at providing real feedback from new and existing games utilizing Unleashed), I would welcome the help and give credit where due.

Cheers.

[paulwheeler]

FYI - I used Unleashed for a long time in my game and I'm afraid over a long period it eventually killed the economy. I had almost no goods being produced at all (this was with 10 UTs).

Its was interesting to watch it all come back to life once I installed a new jobs file.

[Serial Kicked]

i've tried it too and made my own job file loosely based on yours.

The issue with your job file, imho, is that the amount of traders have been cut down too much (around 50% iirc).

High end traders aren't an issue here as they mostly send the finished goods to "sink" stations (eq, trade station), so killing 50% of them has a no bad impact. But low end traders (energy, food, ore) used to supply most if not all factories are too few. It causes to "economy" to slow down a lot. Instead, I've kept ore/food/energy traders to near vanilla (around 3/4) settings and removed a lot of the quite useless ATF, Aldrin and Khaak ships.
One good think about reduced trade jobs, is that most freighters will have something in their cargo and do something useful instead of going around aimlessly.

I also cut down yaki, pirates, and police jobs a bit. IIRC, you didn't touch those values at all. Instead, i gave the police additional firepower to balance that a bit (not sure if it worked, though).

By the way, removing civilians has no impact at all, they don't exists OOS because of the "destroy out of sector" flag. Speaking of which, i have yet to try using this flag on Khaaks and other military/pirate ships to see if it can help reducing workload. If it work it could be used to cut the amount of "kill.enemies" job files a lot.

That's about all i can think right now.

Cheers,
SK.

[Jaga_Telesin]

Thanks for the feedback guys. I believe the issue with traders to be twofold: first was the number of reductions, second was that their flight speed needs to be increased. A combination of both tweaks should rectify any economy issues over a long-term game. I may also make additional changes to them to try and help increase their efficiency/speed, rather than simply returning their numbers to vanilla levels. Fewer ships moving faster and cluttering up gates less has got to be more efficient for the CPU.

The changes to no civilians does have an impact actually - mostly when they are in-sector. The extra horsepower needed to do flight computations and collision detection is freed up otherwise, and has a noticeable effect. But as originally stated (I think), there will be a "standard" version with reduced civilians, and one with no civilians so that people can choose which they want. I personally prefer the reduced version. Nevertheless, thank you for bringing the Destroy OOS flag to my attention - this may have application elsewhere, and I'll look into it.

I think Yaki and Pirates went untouched, and Police received a small reduction in number (almost not worth mentioning). I always thought the number of rogue ships to be adequate without being too high.

Rest assured that the economy in a long-term game is a high priority in the release candidates. The NPC traders need to be able to keep it alive just enough that stations don't start getting the axe from the God engine, but there also needs to be plenty of headroom for player-run traders to make money.

Cheers.

[Jaga_Telesin]

A few thoughts after having looked at the up-to-date Jobs file entries:

There are some flights that fly at full speed, and don't really need to. Flying faster requires more CPU time to handle flight changes and more frequent collision detection, so I'll look at reducing these to "average" speed. Again, this only matters for flights in-sector, since there is no collision out-of-sector, but it does have an impact.

The traders that were still flying at half (average) speed, were primarily the low-end traders (bio, food, minerals), who are the ones that generally keep the stations producing goods. There were others that can also be sped up, but the majority are the backbone of the economy. Increasing their speed to full (double what it was in Beta1), should help them to increase their trades by at least 80% (allowing some fudge time during a flight for docking/undocking and gate travel).

I believe that simply allowing the traders to fly at their full speed will remedy any economy issues seen in long-term games. I can and will bump their numbers up a touch, but want to see where it goes with a 24hr run of 10x SETA and faster traders. As there is no collision OOS, the only impact to faster flying is IS, and less ships vying for gate time should easily offset the increased speeds.

[Jaga_Telesin]

Another quick note, for those that have witnessed the strange "dot" replacing a flight's true icon (typically on civilian flights) - that seems to be a bug with the Jobs Editor, which can be remedied with either a patch to the program, or hand-edits to the Jobs file after changes are made.

One more mystery solved.

[Serial Kicked]

Yes, the full speed option is a good idea. Even IS i'm pretty sure the impact is very low on the autopilot, maybe except in very roid-heavy sectors. Auto pilot issues mostly concern wings / ships flying in formation. standard, slow traders are less likely to be affected. I don't recall if you can set the amount of rudder optimizations in the job file, but it's worth looking into, the more rudder, the less collisions.

About "destroy oos", yeah i'll be interested in the results. I guess you can't spam it on most military ships without triggering tons of issues and incompatibilities with major scripts. But for ships that never leave a sector (Xenon, Khaaks, Aldrin are good potential targets here) it could be very helpful.

[Jaga_Telesin]

Yes, the full speed option is a good idea. Even IS i'm pretty sure the impact is very low on the autopilot, maybe except in very roid-heavy sectors. Auto pilot issues mostly concern wings / ships flying in formation. standard, slow traders are less likely to be affected. I don't recall if you can set the amount of rudder optimizations in the job file, but it's worth looking into, the more rudder, the less collisions.

About "destroy oos", yeah i'll be interested in the results. I guess you can't spam it on most military ships without triggering tons of issues and incompatibilities with major scripts. But for ships that never leave a sector (Xenon, Khaaks, Aldrin are good potential targets here) it could be very helpful.

That was exactly what I had in mind, and have started working on with the newest RC version. Lots of places with room for improvement, without littering the universe with uber (trader) ships. I have also identified a way to increase trader flight speeds without making it impossible for the player's trader ships to "beat them to good deals".


Update: For mod makers with packages that alter the Jobs.txt and/or JobWings.txt files, I am open to merging Unleashed's changes with your own, for full compatibility between the two. Simply contact me here on the forums via PM, and we can discuss the amount and depth of changes required.


Cheers.

[Jaga_Telesin]

Note: With regards to the "Destroy OOS" flag that can be toggled on flights in the Jobs file.. it is either not working at all, or we don't fully understand how it works. I just fired up a new game (vanilla 2.7.1) with only Unleashed RC1 and Cycrow's Cheat script (so I can watch the universe). After letting it run in SETA for an hour of in-game time, I revealed all sectors and dropped satellites in all. I then proceeded to review stations and ships halfway across the galaxy - and saw Civilian ships of all types flying around in each sector I had a satellite in.

That means ships with the flag (in this case civvies) either appear every time it is *possible* to view them, or they are always flying around in other sectors throughout the entire game. I am inclined to believe the latter, based on the results I received when testing Unleashed's last Beta. In that case, removing civilian flights resulted in a direct increase in FPS, even when not viewing (or having discovered) other sectors.

Fortunately it changes little, since even if it did work it could still only be applied to "ornamental" flights that give flavor to the universe, and have no real purpose. Removing something like a Pirate, Yaki, or Khaak from any sectors that qualify as OOS, would mean visiting ships would have no interaction with them, as they wouldn't exist.

[Serial Kicked]

This command is tricky, if cycrow hasn't pointed me to it i'd still think this flag had no real usage.

The sector map is deceptive on that matter (and many others), ships with this tag are created "on the fly" when it's opened according to the "max ship per sector" (or something like that) flag. However when you check the job list in the script editor you only see around 20 "job.stationtostation" instead of the supposedly 200.

So, they are generated when the player open the map and when he enters a sector with civilians inside. I'm not really sure if it occurs when you use one of your ships to jump in the oos sector, it would make sense as you can target/attack them from OOS, but it raises a few questions like: "what happens if i orders my M6 to attack a civilian i see on the map OOS, and then close the map before the M6 reach the sector?".

And whatever the answer is, it of course needs calculations from the engine that could be put on a better use elsewhere. I'll probably investigate on that matter anyway as i'm really wondering how all of this really work. I'll post results if anything interesting happen.

Fortunately it changes little, since even if it did work it could still only be applied to "ornamental" flights that give flavor to the universe, and have no real purpose. Removing something like a Pirate, Yaki, or Khaak from any sectors that qualify as OOS, would mean visiting ships would have no interaction with them, as they wouldn't exist.

For pirates and yakis you're right, moreover their job is supposed to make them move from a sector to another, and i don't think the engine will like that mixed with the OOS tag. However, for the khaaks, who run a simple attack.all script, being able to remove all of them when OOS could be beneficial, at least if they start existing when a player owned ship jumps in.

[Jaga_Telesin]

So, they are generated when the player open the map and when he enters a sector with civilians inside.

Or from what I can tell, if there is a satellite/station the player owns in that sector, the ships are never destroyed as there is always a player asset there to monitor (and keep them alive) with.

That may be what contributes to massive savegame bloat in well-established games, where players have satellites in a lot of different sectors (that they would not with a new game). The game saves the state of all ships, including those the satellites/stations/etc are keeping alive. In that case, the total quota on civvy ships would definitely impact the performance of the established game.

It would be interesting to see with a long running game with lots of satellites and stations in different sectors, if that number has raised proportionately. I'll bet it has.

[Jaga_Telesin]

Update: Release Candidate 1 is now available for download, link is in the primary post of this thread. All reported bugs and market anomalies have been addressed and fixed.

A new game is not required, but is recommended. Use with an existing game creates orphaned flights that are removed over time by the game's engine naturally.

Enjoy!

[CuriosityKillsTheMADCat]

Thank you .

CKTMC

[Jaga_Telesin]

Welcome, of course CKTMC.

A bit of feedback regarding my own new game using Unleashed (w/Civs): plays extremely smoothly, and can even use 10x SETA without the display slowing to a crawl. On top of that, market numbers are doing better than what they would normally be in Vanilla (number of active and producing/trading factories) by around 1-2%.

I'll probably run some metrics tests in the next few days, depending on whether or not additional changes are made to the mod. We'll probably be merging draffutt's fixes in, and may have additions from the Destroy OOS flag, along with other tweaks (Paranid sectors seem busy, and I need to check on Terran/Aldrin economies).

[Zaixionito]

Welcome, of course CKTMC.

A bit of feedback regarding my own new game using Unleashed (w/Civs): plays extremely smoothly, and can even use 10x SETA without the display slowing to a crawl. On top of that, market numbers are doing better than what they would normally be in Vanilla (number of active and producing/trading factories) by around 1-2%.

I'll probably run some metrics tests in the next few days, depending on whether or not additional changes are made to the mod. We'll probably be merging draffutt's fixes in, and may have additions from the Destroy OOS flag, along with other tweaks (Paranid sectors seem busy, and I need to check on Terran/Aldrin economies).

Yes, thank you. This reduces flights on my game in the thousands. Very useful, and I hope you keep developing it.

[Jaga_Telesin]

Update: A quick update after some recent testing in both live games, and test starts...

It would appear that sometime inbetween 2.1 and 2.71 Egosoft changed the way the game handles orphaned flight cleanup. When adding in a Jobs.txt file that has less flights than the game was started with, you will no longer see nameless ships sitting idle in space - now those ships keep flying and look just like they did before, until their "time is up" (destroyed, recycled, etc). The engine continuously re-evaluates flights based off of Jobs.txt quotas, and simply doesn't spawn new ones if they aren't needed.

What this means is - no cleanup script is needed to remove unnamed flights, and immersion will never be broken because of the Jobs file changing. It is excellent news.

Additionally, starting a game with a standard Jobs.txt file which creates thousands of extra flights, and then switching to an Unleashed Jobs file, will auto-adjust over time. Flights will slowly and naturally reduce until they are near where they should be - performance will increase along with the changes. Tested and confirmed on a new vanilla game, using 10x SETA over 72 in-game hours.

I'm seeing a few civilian flights with dots as their icon in the sector map, which should be easy enough to pin down and correct, and is harmless either way.

The economy in the 72-hour test game is within 3% of normal, satisfying my requirement for a "stable and healthy economy that leaves room for player trading". In my personal game, it is actually healthier than it would be in Vanilla, so I'm pleased with the changes there.

-------------------

Big thank yous go out to Litcube, mark_a_condren, and Serial Kicked - their assistance with testing, scripting, and feedback over the last week has been invaluable.

[Jaga_Telesin]

RC1 Initial Performance Results:

Using nearly identical testing methods (from Beta1's tests), Unleashed RC1 shows a 38% performance gain over Vanilla X3:TC.

[Idea]

Hey Jaga_Telesin

Could you tell me how much is the difference between RC1 and Beta 1 performance gain?