many script instances slow down SETA (spaceflies, Anarkis/Pirate Guild)

The place to discuss scripting and game modifications for X³: Terran Conflict and X³: Albion Prelude.

Moderators: Moderators for English X Forum, Scripting / Modding Moderators

User avatar
Aldebaran_Prime
Posts: 1102
Joined: Sat, 20. Feb 10, 18:47

Post by Aldebaran_Prime » Fri, 8. Jun 18, 00:33

|K.O.S.H. wrote:Yes, I stopped all global scripts.

I have an eye on the instances, the number does not increse anymore.
I patched the two (spacefly) scripts, so that i can kill any new instances, but he old ones remain :/

If there was a way to restart running scripts with a newer version, my problem would be solved.

Or a command to get all objects, that run a certain script.
please, may you publish the code here? I would include this into my mod to prevent new players to run into this problem
Image

User avatar
Aldebaran_Prime
Posts: 1102
Joined: Sat, 20. Feb 10, 18:47

Post by Aldebaran_Prime » Tue, 12. Jun 18, 22:37

Hi all, because nobody posted here an answer, maybe somebody can check if the following approach works as I thought and it seems to.

The idea is, not to restart the spacefly script, if the version changes, but to destroy the whole spacefly if the version of its script changes.
This is possible without too many disadvantages for the player, because spaceflys are destroyed and re-created normally, if the player changes a sector. So latest after an update and one sector change the typical amout of spaceflys are back in sector.

I put the following code at the top of the !ship.cmd.idle.spacefly-script:

Code: Select all

* set local variable to current script version
$CurrentSpaceflyVersion = get script version
* get last saved script version from global variable and save in local variable
$LastSpaceflyVersion = get global variable: name= 'idle.spacefly.version'
* set golbal variable to current version
set global variable: name='idle.spacefly.version' value =$CurrentSpaceflyVersion

*now compare current version with version of global variable
if not $LastSpaceflyVersion ==$CurrentSpaceflyVersion 
 write to player logbook 'Debug: ship.cmd.idle.spacefly: different Script versions- I will kill myself'
* destroys the object where the sript is runnung on... only the next instance of a spacefly will survive 
[THIS] -> destroy object: killer=null, show no explosion=0
return null
end
...
If the script version is not identically with the last saved version of this script, all spaceflys should kill themselfs, including their script instances

The same method may be used to examine an other global variable which can be manually set to "TRUE" to kill all running instances of spaceflys in the universe

Code: Select all

...
*now examine global kill-variable 
$KillSpaceFly = get global variable: name= 'KillSpaceFly'

if  $KillSpaceFly == [TRUE]
 write to player logbook 'Debug: ship.cmd.idle.spacefly:  KillSpaceFly == TRUE - I will kill myself'
* destroys the object where the sript is runnung on... as long  als KillSpaceFly ist true, no spacefly will survive
[THIS] -> destroy object: killer=null, show no explosion=0
return null
end
...

If we put this code in the spacefly-scripts we are prepared for future inflation of spaceflys.

what dou you think? Are there better approaches out there in x-universe?
Image

Cycrow
Moderator (Script&Mod)
Moderator (Script&Mod)
Posts: 20534
Joined: Mon, 15. Nov 04, 00:26

Post by Cycrow » Wed, 13. Jun 18, 10:32

Theres actually a proper command to check for new versions, which would be better to use, as you wont need to use global variables

something like, if is new version of script available.

as the problem seems to be that spaceflys are in the null sector, you could simply put a check in for the current sector, if its null, then you could just destruct them as well.

one other thing, remember the followers as well, as spaceflies are in swarms

SirNukes
Posts: 225
Joined: Sat, 31. Mar 07, 23:44

Post by SirNukes » Wed, 13. Jun 18, 19:03

Killing the script on version change doesn't seem like it would have any advantage over rebooting the script on version change, unless you want to use a version change as an explicit hack to end all running copies. It would be cleaner to do a proper reboot and then just adding other tweaks to the new script to end it under some conditions.

One idea to prevent immortal spaceflies would be to add a lifetime to the spacefly script: after 2 hours (or whatever), the script can kill the spacefly off and end. Followers that then swap to the leader script would kick off a new timer and eventually die as well.

Though one worry is that the spaceflies might be unkillable. A couple of times in game I have run across immortal ships that could not be killed by any method I tried, including direct script commands or docking them and then killing their dock. The '!ship.cmd.idle.spacefly' script should probably be given its own timeout, since such spaceflies only swap to the leader script if the leader stops existing.

User avatar
Aldebaran_Prime
Posts: 1102
Joined: Sat, 20. Feb 10, 18:47

Post by Aldebaran_Prime » Wed, 20. Jun 18, 00:10

unfortunately, I was not able to get the

Code: Select all

If is a new script version available 
...
end
version to running. So for the moment I stay to the old fashioned way with the global script version variable. But if somebody has some code snipplets for me to do it right, I will try it again.

I tested further, and found, that spaceflys NOT always killed by sector management, if the player leaves the sector.
I found this, because of some debugging messages of my modified spacefly scripts are writing into the player log now. And after back an forth changing of the sector, some old instances of spacefly scripts were still running.
I also found, that my dectector script (see below) showed me some spaceflys at position 0,0,0 - but I was not able to hibernate them at this position with the ion-disruptor. At other detected positions it worked correctly.

I modified now the scripts:
!ship.cmd.idle.spacefly and !ship.cmd.follow.spacefly ...
That they will kill themselves and their spaceflys if a new script version is available OR a global variable $KillSpaceFly was set to TRUE by another script.
Therefore I wrote 2 additional scripts: ...
1. aaKillSpaceFly
2. aaEnableSpaceFly
... which will either set the $KillSpaceFly to TRUE or FALSE

At the moment these scripts needed to be started manually in script editor to kill spaceflys - but it works! Also for the 0,0,0 spaceflys.
For sure with the known restriction, that instances of older spaceflys which are running in the nowhere are still not killable.

As a bonus add-on, the leading spaceflys will now show their positions in the map temporarily, if the player ship is equipped with a spacefly collector and the distance between player ship and spacefly swarm is below 10.000 m

These scripts can be downloaded by anyone interested in here: https://1drv.ms/u/s%21AleWkrAhnZL_gfA0JJHW1CfzEg5jLA

I think, I will puplish this as a separate "Advanced Spacefly MOD" soon, too.
Hope, this may help others to find a final solution for the zombie-spaceflys which slow down our game.
Image

SirNukes
Posts: 225
Joined: Sat, 31. Mar 07, 23:44

Post by SirNukes » Wed, 20. Jun 18, 00:50

Aldebaran_Prime wrote:But if somebody has some code snipplets for me to do it right, I will try it again.
I looked around and found a couple stock scripts that restart themselves on version change:
!job.special.tpliners
!station.shieldstock.terran

SirNukes
Posts: 225
Joined: Sat, 31. Mar 07, 23:44

Post by SirNukes » Wed, 20. Jun 18, 06:15

I took another look at this, now that I have a bit more experience with the obj assembly. After running through a few ideas, one notable aspect of the spacefly scripts is they appear to be the only ones that use the 'is disabled' command (based on a quick grepping). So, I edited the jump table in the script execution code to go to the 'destruct' code when 'is disabled' is called.

In testing, it was very fun to see the 3k+ spacefly scripts (on a 10 day old game) die off in chunks. However, the list of save games got messed up somehow and appears to be listing the directory contents from when I made the save months ago, so something weird got messed up. Other bad news is that I didn't notice any notable framerate boost after the spacefly deaths.

Swapping to the Return command (instead of Destruct) was tried, but only works on idle spaceflies since the followers are running escort commands (and so never hit the 'is disabled' line).

Maybe it is unsafe to call destruct on [THIS] in a script? At any rate, I can play around a bit more and see if a solution works without side effects.

Cycrow
Moderator (Script&Mod)
Moderator (Script&Mod)
Posts: 20534
Joined: Mon, 15. Nov 04, 00:26

Post by Cycrow » Wed, 20. Jun 18, 10:34

the "is disabled" command is mainly for when spaceflys have been stunned and can be collected

User avatar
Aldebaran_Prime
Posts: 1102
Joined: Sat, 20. Feb 10, 18:47

Post by Aldebaran_Prime » Wed, 20. Jun 18, 14:13

SirNukes wrote:...
However, the list of save games got messed up somehow and appears to be listing the directory contents from when I made the save months ago, so something weird got messed up.
Yes, I recognized this effect, too. But just the actual savegame isn't shown anymore
SirNukes wrote:...
Other bad news is that I didn't notice any notable framerate boost after the spacefly deaths...
I got between 5 an 10 FPS more, if I deleted 12.000 Spaceflys
SirNukes wrote:...
Maybe it is unsafe to call destruct on [THIS] in a script? At any rate, I can play around a bit more and see if a solution works without side effects.
yes, the side effect, that the current savegane isn't shown anymore is strange - I' m curious, if you'll find something. In the meantime I'll try the correct script-update/restart mechanism
Image

SirNukes
Posts: 225
Joined: Sat, 31. Mar 07, 23:44

Post by SirNukes » Thu, 21. Jun 18, 02:19

Aldebaran_Prime wrote: I got between 5 an 10 FPS more, if I deleted 12.000 Spaceflys
This motivated me to recheck, this time measuring practical SETA speedup (game minutes passed during a couple real minutes).

Tested save:
XRM, 10 days in, 134 factories, 200 ships.
15k scripts active, 3.3k of them spaceflies (241 idle, rest followers).
Testing in Harmony of Perpetuity, far in the lower left pointing at the planet.
SETA set to 5000% (my normal 1500% was capping out) for 2 real minutes, starting from a new game minute.

With spacefly scripts running: 9 game minutes elapsed (450%).
With spacefly scripts killed off: 59 game minutes elapsed (2950%).

6.5x speedup from killing the spaceflies. Wow.

Here I was thinking your problem was for people with 10 year old saves, so I'm a little shocked at the results.


In other news, I tried a couple more tricks, refining the assembly edit to do a proper SelfDestruct inside the "is disabled" command handling code, followed by either passing to the "return" or "is docked" commands for cleanup (the latter fills in return args for the following conditional branch). These had the same effect as redirecting the jump table, though, with the saved game list getting messed up. If I only redirect to the "return" code this problem doesn't crop up.

There are vanilla scripts that call "[THIS]->destruct:..." followed by a return, for what that's worth. Maybe I need to look into where the saved game list is built to get an idea of what is going on.

But, if I patch the code and destroy the spaceflies, save the game into a known okay spot, remove the code patch, and reload, things seemed to be working fine again. I am just worried that there might be a subtle bug introduced somewhere.

Cycrow
Moderator (Script&Mod)
Moderator (Script&Mod)
Posts: 20534
Joined: Mon, 15. Nov 04, 00:26

Post by Cycrow » Thu, 21. Jun 18, 10:34

SirNukes wrote: There are vanilla scripts that call "[THIS]->destruct:..." followed by a return, for what that's worth. Maybe I need to look into where the saved game list is built to get an idea of what is going on.
Generally anything after the destruct command will be ignored, the script editor does automatically add return statements at the end though, so all scripts have a return, even if they never get to it

SirNukes
Posts: 225
Joined: Sat, 31. Mar 07, 23:44

Post by SirNukes » Fri, 22. Jun 18, 03:27

Thanks, good to know about the script engine.

I tried a couple other code edits, using JumpOutOfExistence instead of SelfDestruct to save on bytes and be able to slot in a spacefly class code check (in case "is disabled" was being called on anything else somehow), but with the same outcome (dead spaceflies, old save game list).

From those tests and Cycrow's tips, I figured the problem probably isn't me, it's somewhere else. For some reason, the save/load system seems to be making spaceflies and then not liking it when those spaceflies get killed off.

That led to discovering that new spacefly swarms are created whenever I load a game. Over multiple saves and loads, they accumulate.


Rolling the game back to plain vanilla doesn't see this problem. Further, applying my spacefly-killer patch doesn't seem to cause the vanilla save game list to go stale, either.

Testing some select mods, I have found so far that Improved Races 2.0 and probably XRM itself spawn these accumulating swarms. My main XRM game makes 2 swarms per load with IR, 1 swarm per load without IR, where the XRM swarm is added during that 1-2 second hiccup XRM has a few seconds after loading a game (running some delayed setup scripts, I guess).

It would be curious to track down where those mods are messing up, and if the problem is limited to spaceflies or if they are littering the null sector with other ships too.

User avatar
Aldebaran_Prime
Posts: 1102
Joined: Sat, 20. Feb 10, 18:47

Post by Aldebaran_Prime » Fri, 22. Jun 18, 12:18

I just tested the "[THIS]-> destruct: show no explosion=0" instead of the "...destroy..." it works, too, but with the same side-effect on showing savegames.

I also use the Improved Races 2.0 in my MOD/Savegame - I try to have a look on this scripts, too.

I searched yesterday longer for a solution of the still non-deletable spaceflys. I found on page 72 of the MSCIHandbook the box "Technical Tidbit" - there is written, that each object should have a ID which is linked to the sector. Therefore the player-ship needs to be in this sector. This can be done by a loop and automatically placing the player ship in all existing sectors and then looping throug all sector objects.
And in this way, also the Zombie-spaceflys may be found.
I also found a old thread of a X2 SpaceFly detector, which uses this method and where are some code snipplets can be taken from:
https://forum.egosoft.com/viewtopic.php?p=887928#887928

Code: Select all

$this.sector = [THIS]-> get sector
$player.sector = [PLAYERSHIP]-> get sector

if $this.sector != $player.sector
return null
end

$sectorID = [PLAYERSHIP]-> get SectorObject ID

$flies = array alloc: size=0

while [TRUE]
dec $sectorID
$object = get object from SectorObject $sectorID
$class = $object-> get object class

* Break out of the loop if you encounter an existng object that is NOT a Space Fly
if $object != null AND $class != [Raumfliege]
break
end

if $object-> exists
append $object to array $flies
end

end

$size = size of array $flies

* Return null if no flies where found.
if $size == 0
return null
end

return $flies
but
$sectorID = [PLAYERSHIP]-> get SectorObject ID

and
$object = get object from SectorObject $sectorID

Does not work anymore in the X3 script engine
Are there newer functions, which do the same?
Last edited by Aldebaran_Prime on Fri, 22. Jun 18, 13:47, edited 1 time in total.
Image

Cycrow
Moderator (Script&Mod)
Moderator (Script&Mod)
Posts: 20534
Joined: Mon, 15. Nov 04, 00:26

Post by Cycrow » Fri, 22. Jun 18, 12:45

The get SectorObject ID command was replaced with, is in active sector.

The other sector object commands were removed due to issues with using those commands. They more often than not broke the game

SirNukes
Posts: 225
Joined: Sat, 31. Mar 07, 23:44

Post by SirNukes » Fri, 22. Jun 18, 21:11

Well, I am happy to report that after looking into the problematic mods, things fell into place pretty quickly and the rest of my guesses were on spot.

In short, the fundamental problem is either bad documentation or bad behavior of the "create ship" command (depending on which angle you look at it). When a spacefly is requested, the script engine redirects from Create to CreateSwarm, so a whole flock of spaceflies are created and their scripts started. The documentation doesn't mention this, and modders do not expect this behavior.
X2-Illuminatus wrote:What could prove difficult is when the spaceflies were created in sector null. Some scripts use this area to create objects in a safe place, before they are used later, or to read their properties upfront for which you need the actual object. If, for example, a script was looping through all TShips entries and forgot to clean up afterwards, this may explain why there are Spaceflies you cannot find in the normal universe.
This is spot on, except that the mods do not exactly forget to clean up. They destroy the single spacefly they meant to create, but not the unexpected extra swarm.


Anyway, after noticing this and figuring the bad saved games list was due to a mod's spacefly getting killed off before it was finished analyzing it, I tweaked the "is disabled" autokill to have a 10 second delay. This allows the mod to do its business, avoiding the saved games list problem. It still doesn't make sense that a script trying to get information on a dead spacefly will mess up the saved games list, but maybe that can be written off as flakiness in the X3 engine.

To prevent new accumulation of spaceflies, I also edited the "create ship" command to behave the way modders would expect, making only a single spacefly with no running script. With this change, Improved Races and XRM no longer leave excess spaceflies lying around. This should also work for any other mod out there with the same problem (in vanilla, XRM, or LU).


These fixes are available in my customization tool (see sig), v3.5, under Kill_Spaceflies and Prevent_Accidental_Spacefly_Swarms. I will update that thread after adding a little polish (3.5.1 with documentation cleanup and such), but github was updated last night with the main changes if anyone wants to try it out.

The MSCI documentation and threads for affected mods should probably be updated, if somebody wants to take care of that. I feel spammy doing it myself.
Author of X3 Customizer

Post Reply

Return to “X³: Terran Conflict / Albion Prelude - Scripts and Modding”