[Mod/Patch] X3-Unleashed [RC2] [Aug-06-2010] - X3 Performance gains up to 38%

Jaga_Telesin · Post by **Jaga_Telesin** » Sat, 31. Jul 10, 21:43

Hi Idea,

Beta1's performance tests showed between 29% and 33% gains over vanilla. Even if we take the high end of those results, RC1 still shows significant gains (up to 5%) over Beta1. When RC1 testing is finished, I expect it to perform as good if not better.

More importantly however, Beta1 had some economy problems with games that lasted a long time due to traders being reduced in number but still flying at half-speed. We've fixed that for RC1, so that their numbers aren't reduced quite as much (65% in RC1 vs 45% in Beta1), and so that they now fly up to their ship's potential.

Serial Kicked · Post by **Serial Kicked** » Sat, 31. Jul 10, 23:23

So, my findings while playing around with the "Destroy OOS" flag. I was about to send it as a PM, but i guess it would be better put as public knowledge.

First of all, the civilians created with this tag work as follow:

As soon as you look at a sector (via map or jumping in), related ships/jobs are created. The more sector you look at, the more jobs are created. However, once you've closed the galaxy map, ships will start disappearing again (it takes a few minutes). So, we can say that civilians are created and destroyed on the fly. Having a satellite/ship in a sector won't make the civilian appear as long as you don't look at this sector.

About using that on Khaak or Xenon ships that never leave their sector.

They are running this job file: !job.fight.rthr.attackallenemies
(it's basically a call to !fight.attack.enemiesrange, it's important for later).

There's around 380 occurences of this job in vanilla. 80 of them are used by the Khaaks, most of the others are used by Xenons and Yakis. I tagged all the Khaak ships to see what happen.

So, as expected, Khaak ships aren't created until you look at the sector. So it's technically possible for the player to abuse the system a bit, as the ships he make jump there won't be attacked until he open the sector map or give them an order (as most of them require to open the sector map). But i don't see how it could be abused for remote mining purposes or cheaty behaviors like that.

But there's an issue.

Jobs are created on the fly, but they are not destroyed like the civilians. I've found an explanation of this weird behavior (not sure it's the real one, but it's the only one i got).

!job.local.stationtostation is a self sufficient script, it may stack another sub script from time to time (the actual travel from a station to another), but there's long periods (when the ship 'sleep' in the station) where it doesn't.

!job.fight.rthr.attackallenemies, on the other hand, always stack another combat script.

My theory is that the destruction mechanism will only be called if there's no other script on the stack (!fight.attack.enemiesrange).

That's about all there's to say about this tag. I'm not very sure if you'd want to use it or not. There's pros and cons. It's a way to boost the performances a bit more (well, at least until the player goes in the related sectors or until this vanilla job script is altered) but it has its limitations.

Cheers,
SK.

Jaga_Telesin · Post by **Jaga_Telesin** » Sun, 1. Aug 10, 00:42

If I understand what you're saying then SK.. it would seem that once created in a sector, Khaak won't "despawn" after a little while the same as civilian flights would, due to the script stack they are running. They may exist in their sector for quite some time before the system decides to remove their flight.

I can think of many ways to abuse the flag actually - tell a miner or explorer to "fly through the gate connected to this sector". Once there, you simply open the Property menu, and issue the associated command to explore/mine/etc, without having to open a sector map. If you never view the sector, local enemies would never spawn, and it would be 100% safe to do whatever the player wanted there as long as they never "peeked". Since the performance gain of toggling the flag on 80(?) flights would be minimal at best, it wouldn't be worth exposing the game to possible loopholes and cheats. Not to mention, scripts that improve the Khaak wouldn't necessarily follow the same rule, so you'd get unpredictable behavior across the board.

Here's the breakdown of flight classifications and how I see them integrating with the universe and as AI entities:

Civilians - have no actual role other than "flavor" and adding interest to sectors. Easily destroyed OOS if possible, since they have no interaction with the player, function within the economy, or fight value.
Hostiles - (Khaak, Xenon) Have fight value, both IS and OOS, for both the player and the game's AI. Critical to the universe operating with elements of conflict.
Military - All the major races, and private corporations (OTAS, Dukes, etc) have a military presence. These ships have fight value, immersion value, and some scripts enable them to react when the player (or other hostiles) attack sector assets. Necessary both IS and OOS for the universe to operate.
Other - Aldrin Follower M5 (Terraformer), TL-Idle, Goners, UFOs. The only ships worth addressing in this group are Goners and Idle TLs. Goners (if I remember correctly) can be required to kickoff a particular line of quests (though I don't know if that's IS communications only, or OOS also). Idle TLs can be hired(?) to transport stations for construction, thus are required to exist both IS and OOS (but otherwise have no tangible value either fight or trade). To that end, Idle TLs & Goners may qualify for OOS destruction.
Police - Police, Customs, Border Patrol. Scan ships passing through their sectors, have immersion and fight value and interact with ships that impact the economy. Don't qualify for destruction OOS.
Rogues - Pirates, Yaki. These are flights valued purely for conflict and immersion. Possible conflict with them needs to happen both IS and OOS regardless of players viewing sectors, partially due to NPC interaction with them as well. Don't qualify for OOS destruction.
Traders - the backbone of TC's economy, have to persist at all times to ensure it continues to thrive. Also do not qualify for OOS destruction.
Wing flights - any flight that has a leader associated with it, and flies in formation (or docked at it's leader). Since I'm not completely sure how these get spawned with regards to their leader, it may be unwise to attempt to destroy them OOS without also destroying their leader, which isn't something we want to do. Most are military based, or police escorts. As such, I don't believe they qualify for OOS destruction.

That list leaves us with two possibilities - the Idle TL flights, of which there are 20 in Unleashed, and Goners. Idle TLs represent 1/2 of 1% (.005%) of total flights which are not wings, or .003% of all flights period. Their contribution to overall performance would be debatable, despite their large collision boxes (which only affect IS anyhow). I'll have to look more into Goners to know how safe removing them OOS would be.

Thank you very much for your investigation into the Destroy OOS flag. It sounds like it was developed specifically for "ornamental" flights like Civilians that can easily be removed once no one is viewing or tracking them. I am going to re-review all Civvy flights and make sure those that can handle the flag, have it enabled. Thoughts on all of this are welcome of course.

Serial Kicked · Post by **Serial Kicked** » Sun, 1. Aug 10, 01:57

It's kinda hard to explain it better to non scripters

They may exist in their sector for quite some time before the system decides to remove their flight.

The system will never remove the khaaks once they are created (except of course if someone kill them) as the script called is a loop that never give back the hand to the job file.

Since the performance gain of toggling the flag on 80(?) flights would be minimal at best, it wouldn't be worth exposing the game to possible loopholes and cheats.

Well i made my test with 80 ships, but it applies to about 380 flights. It would have been a valid addition if the jobs were created when a player owned ship is in the same sector, but it's not the case.

Other -> TL can be hired. And they are sometimes hijacked by god to transport npc stations iirc, i wouldn't change this setting. Goners could qualify for OOS destruction, but it's the same issue as the Khaaks, the vanilla job script has to be altered or they won't get destroyed.

Thank you very much for your investigation into the Destroy OOS flag. It sounds like it was developed specifically for "ornamental" flights like Civilians that can easily be removed once no one is viewing or tracking them. I am going to re-review all Civvy flights and make sure those that can handle the flag, have it enabled. Thoughts on all of this are welcome of course.

No problem, that's a setting i wanted to investigate anyway even if it's basically a dead end

Yes it was designed for that purpose only.

Notice that some/most of jobs will move a ship from a sector to another, and it's not something that should be handled by this flag. As an example, you'll see a TP Liner depart from a station if you're in the same sector, but you'll never see one entering your sector when in route to its destination. That's why i was interested in studying the fight.rthr.attackallenemies job as there's lot of them and they don't move out of sector.

Jaga_Telesin · Post by **Jaga_Telesin** » Sun, 1. Aug 10, 08:04

Update: Official RC1 Test Results are in!

The original report of "38% gains" was right on the money. Nine tests were done for both Vanilla and for Unleashed(NoCivs). FPS and percentage increase charts for the results are below.

First, actual FPS results done with little demand on the graphics card. Illustrates how much CPU power is freed up, and how CPU-bound X3:TC is. Data point #1 is right after starting a new game and orienting the ship so that no objects are present. Data point #2 is 50km away from a gate (and any nearby ships), so that rendering demands are nearly nonexistent. Data point #3 is at the same point as #2, but after 30 minutes in-game using SETA so that the game's engine has some time to sort out flights, etc and get into a "stable state".

[ external image ]

Note: Due to the low graphical demand of these tests (done deliberately to demonstrate X3:TC's reliance on the CPU), FPS numbers appear rather high. During normal gameplay on the same machine, I witness frame rates between 40 and 150, with a much smoother experience.

For the second chart, the results are translated into performance percentage points. The important data point is the upper-right most one sitting at 138%. It shows what happens after SETA is run to reach a more stable state in the game, and the resulting performance gain.

[ external image ]

I'll be updating the primary topic post later, along with updating all of the associated images and data. For now, I'm pleased as pie with the results.

Jaga_Telesin · Post by **Jaga_Telesin** » Mon, 2. Aug 10, 21:19

Update: RC2 is in the works, estimated available near the end of the week, Aug 6th-8th. Some fixes as discussed with Draffutt have been merged, as well as tweaks to Terran Traders to boost certain portions of their economy (Water, MRE, Ice, etc).

Remaining things to-do, prior to v1.0 release:
- Check Aldrin economy in an established game
- Check overall economy stability over extreme duration (300+ game hours)
- Spot check availability of wares in universe with extreme duration game
- Check known "busy" sectors for frame rates in an established game

Note: It is perfectly safe to use Unleashed and update to a newer version later, as doing so is completely seamless. No restart required - just replace the current Unleashed cat/dat pair in your X3TC directory with the newer ones, and the X3TC engine will adjust flights automatically.

Jaga_Telesin · Post by **Jaga_Telesin** » Thu, 5. Aug 10, 02:43

Quick note for those staying on top of the mod's status:

Due to the overwhelming number of hours that long-term games require for each market stability test (24+ on average), I went and setup a dedicated PC with 2.7.1 on it to help speed along the results, and allow me to perform more frequent and accurate tweaks to Unleashed. While I can get some information from my personal game while I play it, I really need better baselines with associated timestamps - this second PC can do that for me.

Additionally, vkerinav and I are discussing merging Unleashed with two other popular Mod's job files (that he's created), so that more people can take advantage of Unleashed's performance gains.

Requiemfang · Post by **Requiemfang** » Thu, 5. Aug 10, 12:56

I'm already starting to test this out by putting it into my current game, sure I'll lose some of the jobs being done by vkerinav jobs from SRM but at the moment I'm more concerned with FPS and CPU cycles

Jaga_Telesin · Post by **Jaga_Telesin** » Thu, 5. Aug 10, 23:36

Compatibility update: Verified compatible with the latest X3:TC patch from Egosoft (in Level3+ testing, unavailable to the public currently). Those of you with access to the L3+ forums and the latest patch can freely use X3-Unleashed without fear of file conflicts.

Unleashed is also compatible with every public patch of X3:TC released by Egosoft, from 2.1 through 2.7.1.

Jaga_Telesin · Post by **Jaga_Telesin** » Sat, 7. Aug 10, 01:10

RC2 Update: Going to post RC2 this weekend for people to play with. Tossing about some changes that vkerinav and I have been discussing, in an attempt to unclutter Terran traffic a bit - they may or may not make it into RC2.

Some interesting market vs performance information: For those people interested a bit in how X3:TC market performance works and how Unleashed modifies it, I'm going to toss out some very interesting findings I've been able to gather over the last week..

Vanilla X3:TC has 3,400 traders flying around. They represent 30% of all ships in the universe (including wings and docked ships). Basically, there is a 1 in 3 chance when you see a ship that it will be a trader, making them one of the largest groups of flights (second only to wingmen).
X3-Unleashed accomplishes the same with around two thirds that many traders (2,375), and does it more efficiently. Tweaks to trader's engine and rudder tunings, allowing them to fly faster than "cruising speeds", and small increases to jump range all enhance their performance.
The economy and market for stations in an Unleashed game is as healthy (or healthier than) one in a Vanilla game. Testing on multiple machines and across some very long time spans indicate that gaps in the market are handled more quickly with traders who can respond in a timely manner, where vanilla traders cannot. This helps stations that may otherwise be removed from the universe to continue to exist and trade.
Total number of stations due to changes from the GOD engine in X3 depend on two things: trader efficiency, and available CPU cycles from your computer. The GOD engine will typically try to fill out as many stations as your traders can handle - stations that go idle for too long get the axe. Stations that are too busy, or where product is lacking in certain sectors, can prompt the engine to create new stations in those areas.
On slower machines running SETA over long time periods, the number of stations is lowered by the GOD engine, perhaps in an effort to balance the workload on the CPU. It doesn't matter in this instance how many traders are flying around - the market will balance itself against the speed of your machine and available CPU cycles. For that reason, running SETA on slower machines can actually impact the X3 market negatively over time.

Because of these findings, I would recommend that anyone running a machine which isn't top-notch, to refrain from using really high levels of SETA for long periods of time. It might be tempting to let your fleet trade all night in SETA 10x while you sleep, but it can devastate the global economy if your machine can't handle the workload.

Using Unleashed can help alleviate the stress that SETA puts on the CPU and free up more CPU cycles to help traders do their thing, especially when combined with it's more efficient traders.

supahfly · Post by **supahfly** » Sat, 7. Aug 10, 01:54

all this sounds really nice. Can't wait to install the SRM compatible version.

Jaga_Telesin · Post by **Jaga_Telesin** » Sat, 7. Aug 10, 05:10

New Package: Release Candidate 2 [RC2] is available for download. If you are currently using RC1 or an earlier version, you can simply replace the old cat/dat pair with the new.

A download link is in the main post of this topic.

Requiemfang · Post by **Requiemfang** » Sat, 7. Aug 10, 07:08

Hmmm... interesting

vkerinav · Post by **vkerinav** » Sat, 7. Aug 10, 17:02

SRM Jobs 3.1, including an Unleashed version.

Jaga_Telesin · Post by **Jaga_Telesin** » Sat, 7. Aug 10, 18:16

That's great, thank you for doing the merge vkerinav!

Requiemfang · Post by **Requiemfang** » Sat, 7. Aug 10, 19:58

vkerinav wrote:SRM Jobs 3.1, including an Unleashed version.

swipes, just what I was waiting for

CasualT · Post by **CasualT** » Tue, 10. Aug 10, 03:59

This sounds like a great mod here. Too bad Egosoft doesn't put such attention to detail into their own game

Does this mod work with Improved Races? That mod runs like a pig on my PC (Intel i5-750), but I enjoy the excitement it brings to X3. I'd love to limit the amount of non-military traffic to give me back some CPU cycles.

Jaga_Telesin · Post by **Jaga_Telesin** » Tue, 10. Aug 10, 04:57

I don't think IR has it's own Jobs file, so it should be compatible. However, be forewarned at the large number of ships IR will add to your game. Unleashed can only do so much to offset them, after which your performance will still be worse than Vanilla TC.

TEKing66 · Post by **TEKing66** » Tue, 10. Aug 10, 05:07

I'm running the X3-Unleashed no civs version and I have to say I have noticed a BIG improvement. Before adding this Aldrin was a slide show. After adding it I can actually fly through Aldrin like it was any other sector.
Not to mention the other sectors are also improved. All those civilians running around were really bogging things up, and in truth I do not miss them.

I highly recommend this mod to anyone and everyone.

Requiemfang · Post by **Requiemfang** » Tue, 10. Aug 10, 05:16

Jaga_Telesin wrote:I don't think IR has it's own Jobs file, so it should be compatible. However, be forewarned at the large number of ships IR will add to your game. Unleashed can only do so much to offset them, after which your performance will still be worse than Vanilla TC.

it does have a job file that is in the addon pack

egosoft.com

egosoft.com

[Mod/Patch] X3-Unleashed [RC2] [Aug-06-2010] - X3 Performance gains up to 38%