[Mod/Patch] X3-Unleashed [RC2] [Aug-06-2010] - X3 Performance gains up to 38%

The place to discuss scripting and game modifications for X³: Terran Conflict and X³: Albion Prelude.

Moderators: Scripting / Modding Moderators, Moderators for English X Forum

Post Reply
User avatar
Jaga_Telesin
Posts: 430
Joined: Sat, 16. Sep 06, 22:48
x4

Post by Jaga_Telesin » Sun, 25. Jul 10, 20:30

Note: With regards to the "Destroy OOS" flag that can be toggled on flights in the Jobs file.. it is either not working at all, or we don't fully understand how it works. I just fired up a new game (vanilla 2.7.1) with only Unleashed RC1 and Cycrow's Cheat script (so I can watch the universe). After letting it run in SETA for an hour of in-game time, I revealed all sectors and dropped satellites in all. I then proceeded to review stations and ships halfway across the galaxy - and saw Civilian ships of all types flying around in each sector I had a satellite in.

That means ships with the flag (in this case civvies) either appear every time it is *possible* to view them, or they are always flying around in other sectors throughout the entire game. I am inclined to believe the latter, based on the results I received when testing Unleashed's last Beta. In that case, removing civilian flights resulted in a direct increase in FPS, even when not viewing (or having discovered) other sectors.

Fortunately it changes little, since even if it did work it could still only be applied to "ornamental" flights that give flavor to the universe, and have no real purpose. Removing something like a Pirate, Yaki, or Khaak from any sectors that qualify as OOS, would mean visiting ships would have no interaction with them, as they wouldn't exist.

User avatar
Serial Kicked
Posts: 3493
Joined: Fri, 12. Aug 05, 20:46
x3tc

Post by Serial Kicked » Sun, 25. Jul 10, 20:54

This command is tricky, if cycrow hasn't pointed me to it i'd still think this flag had no real usage.

The sector map is deceptive on that matter (and many others), ships with this tag are created "on the fly" when it's opened according to the "max ship per sector" (or something like that) flag. However when you check the job list in the script editor you only see around 20 "job.stationtostation" instead of the supposedly 200.

So, they are generated when the player open the map and when he enters a sector with civilians inside. I'm not really sure if it occurs when you use one of your ships to jump in the oos sector, it would make sense as you can target/attack them from OOS, but it raises a few questions like: "what happens if i orders my M6 to attack a civilian i see on the map OOS, and then close the map before the M6 reach the sector?".

And whatever the answer is, it of course needs calculations from the engine that could be put on a better use elsewhere. I'll probably investigate on that matter anyway as i'm really wondering how all of this really work. I'll post results if anything interesting happen.
Fortunately it changes little, since even if it did work it could still only be applied to "ornamental" flights that give flavor to the universe, and have no real purpose. Removing something like a Pirate, Yaki, or Khaak from any sectors that qualify as OOS, would mean visiting ships would have no interaction with them, as they wouldn't exist.
For pirates and yakis you're right, moreover their job is supposed to make them move from a sector to another, and i don't think the engine will like that mixed with the OOS tag. However, for the khaaks, who run a simple attack.all script, being able to remove all of them when OOS could be beneficial, at least if they start existing when a player owned ship jumps in.
X3:TC/AP Pirate Guild 3 - Yaki Armada 2 - Anarkis Defense System

Anarkis Gaming HQ
Independent Game Development
X3 Scripting and Modding Station

User avatar
Jaga_Telesin
Posts: 430
Joined: Sat, 16. Sep 06, 22:48
x4

Post by Jaga_Telesin » Sun, 25. Jul 10, 21:08

Serial Kicked wrote:So, they are generated when the player open the map and when he enters a sector with civilians inside.
Or from what I can tell, if there is a satellite/station the player owns in that sector, the ships are never destroyed as there is always a player asset there to monitor (and keep them alive) with.

That may be what contributes to massive savegame bloat in well-established games, where players have satellites in a lot of different sectors (that they would not with a new game). The game saves the state of all ships, including those the satellites/stations/etc are keeping alive. In that case, the total quota on civvy ships would definitely impact the performance of the established game.

It would be interesting to see with a long running game with lots of satellites and stations in different sectors, if that number has raised proportionately. I'll bet it has.

User avatar
Jaga_Telesin
Posts: 430
Joined: Sat, 16. Sep 06, 22:48
x4

Post by Jaga_Telesin » Tue, 27. Jul 10, 04:22

Update: Release Candidate 1 is now available for download, link is in the primary post of this thread. All reported bugs and market anomalies have been addressed and fixed.

A new game is not required, but is recommended. Use with an existing game creates orphaned flights that are removed over time by the game's engine naturally.

Enjoy!

CuriosityKillsTheMADCat
Posts: 530
Joined: Thu, 1. Apr 10, 00:19
x4

Post by CuriosityKillsTheMADCat » Wed, 28. Jul 10, 15:27

Thank you :) .

CKTMC
Fähiger Pilot wie DU sein...
"When ships to sail the void between the stars have been invented, there will also be men who come forward to sail those ships."
Johannes Kepler, Somnium, 1634 A.D.

User avatar
Jaga_Telesin
Posts: 430
Joined: Sat, 16. Sep 06, 22:48
x4

Post by Jaga_Telesin » Wed, 28. Jul 10, 17:31

Welcome, of course CKTMC. :D

A bit of feedback regarding my own new game using Unleashed (w/Civs): plays extremely smoothly, and can even use 10x SETA without the display slowing to a crawl. On top of that, market numbers are doing better than what they would normally be in Vanilla (number of active and producing/trading factories) by around 1-2%.

I'll probably run some metrics tests in the next few days, depending on whether or not additional changes are made to the mod. We'll probably be merging draffutt's fixes in, and may have additions from the Destroy OOS flag, along with other tweaks (Paranid sectors seem busy, and I need to check on Terran/Aldrin economies).

Zaixionito
Posts: 158
Joined: Mon, 11. Jan 10, 00:48
x4

Post by Zaixionito » Wed, 28. Jul 10, 17:40

Jaga_Telesin wrote:Welcome, of course CKTMC. :D

A bit of feedback regarding my own new game using Unleashed (w/Civs): plays extremely smoothly, and can even use 10x SETA without the display slowing to a crawl. On top of that, market numbers are doing better than what they would normally be in Vanilla (number of active and producing/trading factories) by around 1-2%.

I'll probably run some metrics tests in the next few days, depending on whether or not additional changes are made to the mod. We'll probably be merging draffutt's fixes in, and may have additions from the Destroy OOS flag, along with other tweaks (Paranid sectors seem busy, and I need to check on Terran/Aldrin economies).
Yes, thank you. This reduces flights on my game in the thousands. Very useful, and I hope you keep developing it.

User avatar
Jaga_Telesin
Posts: 430
Joined: Sat, 16. Sep 06, 22:48
x4

Post by Jaga_Telesin » Sat, 31. Jul 10, 19:38

Update: A quick update after some recent testing in both live games, and test starts...

It would appear that sometime inbetween 2.1 and 2.71 Egosoft changed the way the game handles orphaned flight cleanup. When adding in a Jobs.txt file that has less flights than the game was started with, you will no longer see nameless ships sitting idle in space - now those ships keep flying and look just like they did before, until their "time is up" (destroyed, recycled, etc). The engine continuously re-evaluates flights based off of Jobs.txt quotas, and simply doesn't spawn new ones if they aren't needed.

What this means is - no cleanup script is needed to remove unnamed flights, and immersion will never be broken because of the Jobs file changing. It is excellent news.

Additionally, starting a game with a standard Jobs.txt file which creates thousands of extra flights, and then switching to an Unleashed Jobs file, will auto-adjust over time. Flights will slowly and naturally reduce until they are near where they should be - performance will increase along with the changes. Tested and confirmed on a new vanilla game, using 10x SETA over 72 in-game hours.

I'm seeing a few civilian flights with dots as their icon in the sector map, which should be easy enough to pin down and correct, and is harmless either way.

The economy in the 72-hour test game is within 3% of normal, satisfying my requirement for a "stable and healthy economy that leaves room for player trading". In my personal game, it is actually healthier than it would be in Vanilla, so I'm pleased with the changes there.

-------------------

Big thank yous go out to Litcube, mark_a_condren, and Serial Kicked - their assistance with testing, scripting, and feedback over the last week has been invaluable.
Last edited by Jaga_Telesin on Sat, 31. Jul 10, 22:00, edited 1 time in total.

User avatar
Jaga_Telesin
Posts: 430
Joined: Sat, 16. Sep 06, 22:48
x4

Post by Jaga_Telesin » Sat, 31. Jul 10, 21:04

RC1 Initial Performance Results:

Using nearly identical testing methods (from Beta1's tests), Unleashed RC1 shows a 38% performance gain over Vanilla X3:TC.

User avatar
Idea
Posts: 516
Joined: Fri, 24. Oct 08, 12:14
x3tc

Post by Idea » Sat, 31. Jul 10, 21:31

Hey Jaga_Telesin

Could you tell me how much is the difference between RC1 and Beta 1 performance gain?
Image

User avatar
Jaga_Telesin
Posts: 430
Joined: Sat, 16. Sep 06, 22:48
x4

Post by Jaga_Telesin » Sat, 31. Jul 10, 21:43

Hi Idea,

Beta1's performance tests showed between 29% and 33% gains over vanilla. Even if we take the high end of those results, RC1 still shows significant gains (up to 5%) over Beta1. When RC1 testing is finished, I expect it to perform as good if not better.

More importantly however, Beta1 had some economy problems with games that lasted a long time due to traders being reduced in number but still flying at half-speed. We've fixed that for RC1, so that their numbers aren't reduced quite as much (65% in RC1 vs 45% in Beta1), and so that they now fly up to their ship's potential.

User avatar
Serial Kicked
Posts: 3493
Joined: Fri, 12. Aug 05, 20:46
x3tc

Post by Serial Kicked » Sat, 31. Jul 10, 23:23

So, my findings while playing around with the "Destroy OOS" flag. I was about to send it as a PM, but i guess it would be better put as public knowledge.

First of all, the civilians created with this tag work as follow:

As soon as you look at a sector (via map or jumping in), related ships/jobs are created. The more sector you look at, the more jobs are created. However, once you've closed the galaxy map, ships will start disappearing again (it takes a few minutes). So, we can say that civilians are created and destroyed on the fly. Having a satellite/ship in a sector won't make the civilian appear as long as you don't look at this sector.

About using that on Khaak or Xenon ships that never leave their sector.

They are running this job file: !job.fight.rthr.attackallenemies
(it's basically a call to !fight.attack.enemiesrange, it's important for later).

There's around 380 occurences of this job in vanilla. 80 of them are used by the Khaaks, most of the others are used by Xenons and Yakis. I tagged all the Khaak ships to see what happen.

So, as expected, Khaak ships aren't created until you look at the sector. So it's technically possible for the player to abuse the system a bit, as the ships he make jump there won't be attacked until he open the sector map or give them an order (as most of them require to open the sector map). But i don't see how it could be abused for remote mining purposes or cheaty behaviors like that.

But there's an issue.

Jobs are created on the fly, but they are not destroyed like the civilians. I've found an explanation of this weird behavior (not sure it's the real one, but it's the only one i got).

!job.local.stationtostation is a self sufficient script, it may stack another sub script from time to time (the actual travel from a station to another), but there's long periods (when the ship 'sleep' in the station) where it doesn't.

!job.fight.rthr.attackallenemies, on the other hand, always stack another combat script.

My theory is that the destruction mechanism will only be called if there's no other script on the stack (!fight.attack.enemiesrange).

That's about all there's to say about this tag. I'm not very sure if you'd want to use it or not. There's pros and cons. It's a way to boost the performances a bit more (well, at least until the player goes in the related sectors or until this vanilla job script is altered) but it has its limitations.

Cheers,
SK.
X3:TC/AP Pirate Guild 3 - Yaki Armada 2 - Anarkis Defense System

Anarkis Gaming HQ
Independent Game Development
X3 Scripting and Modding Station

User avatar
Jaga_Telesin
Posts: 430
Joined: Sat, 16. Sep 06, 22:48
x4

Post by Jaga_Telesin » Sun, 1. Aug 10, 00:42

If I understand what you're saying then SK.. it would seem that once created in a sector, Khaak won't "despawn" after a little while the same as civilian flights would, due to the script stack they are running. They may exist in their sector for quite some time before the system decides to remove their flight.

I can think of many ways to abuse the flag actually - tell a miner or explorer to "fly through the gate connected to this sector". Once there, you simply open the Property menu, and issue the associated command to explore/mine/etc, without having to open a sector map. If you never view the sector, local enemies would never spawn, and it would be 100% safe to do whatever the player wanted there as long as they never "peeked". Since the performance gain of toggling the flag on 80(?) flights would be minimal at best, it wouldn't be worth exposing the game to possible loopholes and cheats. Not to mention, scripts that improve the Khaak wouldn't necessarily follow the same rule, so you'd get unpredictable behavior across the board.

Here's the breakdown of flight classifications and how I see them integrating with the universe and as AI entities:
  • Civilians - have no actual role other than "flavor" and adding interest to sectors. Easily destroyed OOS if possible, since they have no interaction with the player, function within the economy, or fight value.
  • Hostiles - (Khaak, Xenon) Have fight value, both IS and OOS, for both the player and the game's AI. Critical to the universe operating with elements of conflict.
  • Military - All the major races, and private corporations (OTAS, Dukes, etc) have a military presence. These ships have fight value, immersion value, and some scripts enable them to react when the player (or other hostiles) attack sector assets. Necessary both IS and OOS for the universe to operate.
  • Other - Aldrin Follower M5 (Terraformer), TL-Idle, Goners, UFOs. The only ships worth addressing in this group are Goners and Idle TLs. Goners (if I remember correctly) can be required to kickoff a particular line of quests (though I don't know if that's IS communications only, or OOS also). Idle TLs can be hired(?) to transport stations for construction, thus are required to exist both IS and OOS (but otherwise have no tangible value either fight or trade). To that end, Idle TLs & Goners may qualify for OOS destruction.
  • Police - Police, Customs, Border Patrol. Scan ships passing through their sectors, have immersion and fight value and interact with ships that impact the economy. Don't qualify for destruction OOS.
  • Rogues - Pirates, Yaki. These are flights valued purely for conflict and immersion. Possible conflict with them needs to happen both IS and OOS regardless of players viewing sectors, partially due to NPC interaction with them as well. Don't qualify for OOS destruction.
  • Traders - the backbone of TC's economy, have to persist at all times to ensure it continues to thrive. Also do not qualify for OOS destruction.
  • Wing flights - any flight that has a leader associated with it, and flies in formation (or docked at it's leader). Since I'm not completely sure how these get spawned with regards to their leader, it may be unwise to attempt to destroy them OOS without also destroying their leader, which isn't something we want to do. Most are military based, or police escorts. As such, I don't believe they qualify for OOS destruction.
That list leaves us with two possibilities - the Idle TL flights, of which there are 20 in Unleashed, and Goners. Idle TLs represent 1/2 of 1% (.005%) of total flights which are not wings, or .003% of all flights period. Their contribution to overall performance would be debatable, despite their large collision boxes (which only affect IS anyhow). I'll have to look more into Goners to know how safe removing them OOS would be.

Thank you very much for your investigation into the Destroy OOS flag. It sounds like it was developed specifically for "ornamental" flights like Civilians that can easily be removed once no one is viewing or tracking them. I am going to re-review all Civvy flights and make sure those that can handle the flag, have it enabled. Thoughts on all of this are welcome of course. :)

User avatar
Serial Kicked
Posts: 3493
Joined: Fri, 12. Aug 05, 20:46
x3tc

Post by Serial Kicked » Sun, 1. Aug 10, 01:57

It's kinda hard to explain it better to non scripters :)
They may exist in their sector for quite some time before the system decides to remove their flight.
The system will never remove the khaaks once they are created (except of course if someone kill them) as the script called is a loop that never give back the hand to the job file.
Since the performance gain of toggling the flag on 80(?) flights would be minimal at best, it wouldn't be worth exposing the game to possible loopholes and cheats.
Well i made my test with 80 ships, but it applies to about 380 flights. It would have been a valid addition if the jobs were created when a player owned ship is in the same sector, but it's not the case.

Other -> TL can be hired. And they are sometimes hijacked by god to transport npc stations iirc, i wouldn't change this setting. Goners could qualify for OOS destruction, but it's the same issue as the Khaaks, the vanilla job script has to be altered or they won't get destroyed.
Thank you very much for your investigation into the Destroy OOS flag. It sounds like it was developed specifically for "ornamental" flights like Civilians that can easily be removed once no one is viewing or tracking them. I am going to re-review all Civvy flights and make sure those that can handle the flag, have it enabled. Thoughts on all of this are welcome of course.
No problem, that's a setting i wanted to investigate anyway even if it's basically a dead end :)

Yes it was designed for that purpose only.

Notice that some/most of jobs will move a ship from a sector to another, and it's not something that should be handled by this flag. As an example, you'll see a TP Liner depart from a station if you're in the same sector, but you'll never see one entering your sector when in route to its destination. That's why i was interested in studying the fight.rthr.attackallenemies job as there's lot of them and they don't move out of sector.
X3:TC/AP Pirate Guild 3 - Yaki Armada 2 - Anarkis Defense System

Anarkis Gaming HQ
Independent Game Development
X3 Scripting and Modding Station

User avatar
Jaga_Telesin
Posts: 430
Joined: Sat, 16. Sep 06, 22:48
x4

Post by Jaga_Telesin » Sun, 1. Aug 10, 08:04

Update: Official RC1 Test Results are in!

The original report of "38% gains" was right on the money. Nine tests were done for both Vanilla and for Unleashed(NoCivs). FPS and percentage increase charts for the results are below.

First, actual FPS results done with little demand on the graphics card. Illustrates how much CPU power is freed up, and how CPU-bound X3:TC is. Data point #1 is right after starting a new game and orienting the ship so that no objects are present. Data point #2 is 50km away from a gate (and any nearby ships), so that rendering demands are nearly nonexistent. Data point #3 is at the same point as #2, but after 30 minutes in-game using SETA so that the game's engine has some time to sort out flights, etc and get into a "stable state".


Image

Note: Due to the low graphical demand of these tests (done deliberately to demonstrate X3:TC's reliance on the CPU), FPS numbers appear rather high. During normal gameplay on the same machine, I witness frame rates between 40 and 150, with a much smoother experience.



For the second chart, the results are translated into performance percentage points. The important data point is the upper-right most one sitting at 138%. It shows what happens after SETA is run to reach a more stable state in the game, and the resulting performance gain.

Image


I'll be updating the primary topic post later, along with updating all of the associated images and data. For now, I'm pleased as pie with the results. :D

Post Reply

Return to “X³: Terran Conflict / Albion Prelude - Scripts and Modding”