Expensive Vanilla Scripts

[Litcube]

I've been running some tests all morning trying to improve script performance on my late game. I've tried various benchmarking methods, and I've found that a simple FRAPS while running SETA 1000% provides the most sensitive results.

In my benchmark environment, an empty sector with the HUD on, I get ~120 while in SETA. I wish we could get a standardized script benchmark, but I don't think that's likely. With different hardware, we're going to get different results.

By knocking off all of my home made scripts, it only jumps by about 20 FPS. When I start knocking off vanilla scripts, I get some massive jumps.

The most expensive script I've found so far is the move.follow.template, which is called by the wing attack same and protect ship. These are constantly running on all docked ships, even if they're idle. While sitting in a carrier doing nothing, these ships cost big resources. By stopping all move.follow.template ship tasks, I get a 90 FPS boost on said benchmark. This is more prevalent now than before, as a lot of us are using different job packs which increase the amount of escorts in a military wing. Contrary to common sense, out of all the military ships in your Universe, the lead ship, an M1 for example, is the least expensive; it's mostly just moving from one point to another, during which time the game engine has control of it, and not running any script commands. Performance is most happy when the engine has control, such as a wait, a move, an idle, etc.

A rewrite of this particular script won't help. I think what we need is a new method of handling carrier compliments. The initiation of action should come from the carrier itself, and not an object in the hangar. In systems, it's widely accepted that polling for an event is always more expensive than initiating it when required (java/.net delegates comes to mind).

Any thoughts?

[HotSake]

I've been considering frameworks for a few script rewrites, and I keep coming back to a "dispatcher" model, where a single script handles monitoring and direction for many subordinate objects, starting scripts on them when appropriate. For example: Lucike's trade scripts have each trader searching for trades, then polling every other trader to coordinate. If you run a lot of independent traders, that's a lot of duplicated effort. What I'd like to see is a single dispatcher running as an AL plugin with all traders subscribed to it. The dispatcher is the only script searching for holes in the economy and sending traders on runs. No need for each trader to find a hole or for each trader to poll the others.

So yes, I think a similar structure for carrier wings (or any arrangement of ships with a clear hierarchy) would be an improvement. Have you noticed any other especially hungry scripts during your benchmarking?

[Gazz]

It's been a while since I dug into move.follow.template but I don't think it actively scans for enemy ships while docked.
However, while "protecting", no doubt the most common task in a carrier force, it's job is to sit tight until the carrier is under attack.

That the docked ships wait until after the carrier has been attacked is part of their uselessness OOS. Often enough the carrier dies from the first shot.

Another part of this job (argument only ever used for AI ships, AFAIK) is to attack the leader's target.

Soooo... the only events that would change the status of the docked fighter are the carrier being attacked or attacking something.
If you attach some kind of management script to the carrier, it could
- freeze all docked formation ships that run "protect"
- wake them up if one of the 2 conditions is met.

The freezing would be a simple task 0 interrupt script with just these 2 loop conditions.
It can have a 230457234 ms timer.
To unfreeze, kick it out with a higher priority interrupt script with 0 duration.
The unfreeze script terminates, the freeze script checks it's conditions and... showtime!


Of course, this system can be improved in many ways, depending on how much brain sweat you'd like to invest.
move.follow.template (after docking) could start the management script on the carrier if it's currently not running.

The management script doesn't have to do much, either.
Watch for being attacked. Watch for me attacking. Repeat.

As an added bonus, the carrier can be set to being attacked by hostile ships in the vicinity which intend to attack it but haven't fired, yet.
(scan for ships, look at their attack target)
That would let carriers do their bloody jobs OOS instead of trying to run the opposition out of laser energy by getting destroyed in the fastest possible way.

One ship doing such a check? Not an issue.



That's a rather small-scale change to hack the current situation.
A real "dispatcher" structure should be more effective.

[jlehtone]

Another part of this job (argument only ever used for AI ships, AFAIK) is to attack the leader's target.

Does player issuing "Attack my target" pass that argument?

[Litcube]

I've been considering frameworks for a few script rewrites, and I keep coming back to a "dispatcher" model, where a single script handles monitoring and direction for many subordinate objects, starting scripts on them when appropriate. For example: Lucike's trade scripts have each trader searching for trades, then polling every other trader to coordinate. If you run a lot of independent traders, that's a lot of duplicated effort. What I'd like to see is a single dispatcher running as an AL plugin with all traders subscribed to it. The dispatcher is the only script searching for holes in the economy and sending traders on runs. No need for each trader to find a hole or for each trader to poll the others.

I've done this with my own trader scripts, but not for holes. Equally expensive routines are station locks and viable sectors. I have two global scripts running that compile viable sectors (player property present, known sector), every two minutes, and another station lock script that traders can check against the system to make sure you're not doubling up on buying/selling. I've rewritten the UT script to use these commands. No performance difference in my benchmark with 35 UTs at level 20 running.

So yes, I think a similar structure for carrier wings (or any arrangement of ships with a clear hierarchy) would be an improvement. Have you noticed any other especially hungry scripts during your benchmarking?

No, not yet. If you or anyone else finds any, please share!

It's been a while since I dug into move.follow.template but I don't think it actively scans for enemy ships while docked.

It does. Every ~30 seconds. So you can have 60 ships in a single carrier doing the same check for the same purpose at the same time for a ~1% chance of engaging an event. Awesome.

Your interrupt idea is brilliant. It's pretty simple and non-intrusive. Question is, where to throw that check?

Edit: Oh. At the beginning of the move.template.follow, as you say. Run it on the carrier, task 100+, as a monitor. Hm.

[Jack08]

perhaps a rewrite of the move.follow.template script in a Finite State Machine format would yield better results

Litcube i sent you a file a while ago called "EventBasedTask.xml", it contains the code for the XTL Finite state machine - feel free to post it here if theres any merit to this suggestion

Ive written a few scripts recently using this model and what Ive noticed is they provide far better performance at less resource drain because of there dynamic nature

[bobxii]

The initiation of action should come from the carrier itself, and not an object in the hangar. In systems, it's widely accepted that polling for an event is always more expensive than initiating it when required (java/.net delegates comes to mind).

Any thoughts?

I had a similar thought while talking about X:R:
http://forum.egosoft.com/viewtopic.php? ... t=#3589298

The gist of it is that we can improve performance by changing the scripting paradigm for fleets (obviously carriers are included in this), specifically by adopting a hierarchal approach to minimize idle load and thus allow for more complex scripting in the flagship's script instance. (This is precisely what hotsake suggests)

The relevant portion:

Use this hierarchal system everywhere to minimize physical CPU cycles. Because the group is assigned to a single script for the majority of its existence, we can implement higher-order logic functions that are only for use in a small number of script instances (the ones controlling fleets), while saving tons of CPU time because the actual instances running on the ships aren't doing anything but waiting for commands from the higher-up.

Also note that it is an immediately obvious that the next logical extension would be to introduce "intelligent" behavior into the command scripts:

----Fleet Engagements: Instead of the current method of simply sending everything out to attack and resulting in a fleet spread out based upon ship maximum speeds (Xenon incursions from a gate are a great example of this problem), the AI should be grouped under a commanding AI that can implement higher-order logic to increase its effectiveness. Examples:

1) Keep the fleet in close proximity to one another, meaning leash fighters to larger vessels and larger vessels to the flagship. This increases its deadliness by virtue of combined arms (a modern military concept).

2) Acquire and kill targets in order of threat. Again, to increase the deadliness of the fleet by not allowing its members to die unnecessarily.

3) Delegate tasks to the fleet members most able to accomplish them. For example, we use light scout-class fighters to chase down missiles aimed at our high-value targets.

[Gazz]

It's pretty simple and non-intrusive. Question is, where to throw that check?

Edit: Oh. At the beginning of the move.template.follow, as you say. Run it on the carrier, task 100+, as a monitor. Hm.

Possible alternative: Secondary signal_docked.

Attach to "small ship" or "fighter".

May not trigger at the creation of new AI wings or MD enemy fleets when they are created docked.



Possible alternative: Tap into the command

See !init.ship.globalscriptmap.std.xml

Write small launcher scripts.
Attach them to COMMAND_PROTECT and other apropriate commands.
Use them to fire up your watcher / manager script on the leader / mothership,
THEN call the "real" !ship.cmd.protect.std or whatever it was.

Bonus:
100% non-invasive.

Use a regular setup script to change the command attachment.

Delete the seup script: functionality is back to vanilla on the next reload. (not counting active scripts, of course)

Doesn't catch cases where !ship.cmd.protect.std is called by name.
The engine quite often calls by command, though. No idea if Jobs.txt does.

Also note that it is an immediately obvious that the next logical extension would be to introduce "intelligent" behavior into the command scripts:

Yes. Then we call it CODEA. =P

[bobxii]

Isn't the dispatcher model already being used in ADS (in a sense, at least)?

[Litcube]

Attach them to COMMAND_PROTECT and other apropriate commands.
Use them to fire up your watcher / manager script on the leader / mothership,
THEN call the "real" !ship.cmd.protect.std or whatever it was.

The real !ship.cmd.protect is really just

Code: Select all

= [THIS]->call script '!fight.attack.object' : victim=$attacker  follow=null  onlyShields=null 

Doesn't catch cases where !ship.cmd.protect.std is called by name.

Are you sure? [THIS]->set command: {COMMAND_PROTECT} target=$target target2=null par1=null par2=null is run in command protect.

I like this idea the best so far.

[Gazz]

I'm not sure because I haven't tested it.
It does work for turret scripts. They are assigned to AI ships by command.

Some script may reference the script directly. Maybe Jobs does. I dunno.

There's a good chance it would work and it doesn't require hacking a single vanilla script...

The real !ship.cmd.protect is really just

I didn't say that I'm the first person to ever think of launcher scripts. =P

[s9ilent]

As mentioned above I don't think that this is a particularly bad script, just that many instances of the script are active at once.

I think a problem with a lot of the stock scripts (and lots of my scripts too ) is that they use polling, instead of signaling. i.e. All ships run the same identical script that do the same checks.

Ideally, there should be very little logic on the escorts, and most of it should be done by the leader, and signaled to the escorts. As there is a very high correlation (~90%?) between the results of all the checks

[Litcube]

Yup. It needs a drastic turn in another direction.

My brain is melting trying to decipher the follow.template, so I think I'm going to do the following:

Make new scripts:

Cmd.Wing.AttackSame: Attack same enemies while enemies in $range of leader, then revert to move.template.follow
Cmd.Wing.Protect: Loop for -'>get attacker', and attack it if so, while enemies in $range of leader, then revert to move.template.follow

Rewrite the move.template.follow to do only these:

If the Lib.Gen.ManageFollowers isn't running on task 565 of the leader, run it.

If this can dock at it's leader, then dock, set homebase.

Divert attack same or protect or follow the appropriate script:
-Lib.Cmd.Follow (one liner, follow command)
-Lib.Cmd.AttackSame (loop =wait for 193921391823 ms)
-Lib.Cmd.ProtectShip (loop =wait for 193921391823 ms)





Meanwhile, at the base of the Lib.Gen.ManageFollowers:


While ForeverAndEver

- If I have no followers or owned ships, return null.
- Scan for enemies (one liner). If enemies, gosub Combat
- Wait

End


COMBAT:

Loop through followers:
- Undock
- If running task attack same,run Cmd.Wing.AttackSame
- If running task protect only, run Cmd.Wing.Protect

loop while enemies in $range
= wait 10 sec


END SUB

[bobxii]

Another method would be to have each carrier keep a list of "active" followers and have the followers run no script while in idle mode.

[Litcube]

Another method would be to have each carrier keep a list of "active" followers and have the followers run no script while in idle mode.

That's dangerous. Updating that list would be expensive, and removing the followers problematic.

[s9ilent]

Hmm managing followers from a separate tasks does sound interesting. Thou it can get messy as the the order can be replaced. You might want to change your while foreverandever to while validOrder (and get the valid order "somehow")


(I so wish scripting was multi thread able

Also w.r.t lib.cmd.attacksame (and those sort of scripts)
I would consider making them take an argument (e.g. victim etc.)
And you can do something like if victim == null -> wait till infinity
Else do work
This way there is one being used to do the action.

(So other scripts can do, is lib.cmd.attack on stack of scripts on task 0)



(Also your wait to infinity should be if leader is carrier and can dock -> dock, wait infinity, else follow for infinity)

[Gazz]

Cmd.Wing.Protect: Loop for -'>get attacker', and attack it if so, while enemies in $range of leader, then revert to move.template.follow

This sounds a lot like the existing attack.enemies.and.land. (paraphrasing)

[bobxii]

Another method would be to have each carrier keep a list of "active" followers and have the followers run no script while in idle mode.

That's dangerous. Updating that list would be expensive, and removing the followers problematic.

True enough. Though it'd only have to update/check when an action was to be taken, yes? Even then, the only way a ship in a hangar would disappear outside of the script itself would be via other script events, that should be rare (though may not be).

[LV]

I'd not be surprised if many vanilla scripts were heavy

Can't remember who originally stumbled upon this treasure but i recall getting a massive boost with rrf when i re-wrote the fight attack object script as that checked for many things a gazzilion times a second

You'll never strip everything out due to hardcoding but i'd say it's worthwhile re-writing because i didn't notice too much difference when i pulled many checks out and used much longer waits

[bobxii]

Any chance that such improvements would be signed? (If not for vanilla reasons, then at least to improve mod deployment so that the community doesn't have to constantly reinvent the wheel - example being the "What I want in ..." thread that has solutions done by existing mods)