[7.0 HF1] Extreme late game FPS drops with high asset and ship counts

[Talinoth]

Not a bug or complaint specifically, more a benchmark for your perusal. Hopefully you can play around and see what might be impacting performance in endgame X4 saves.

With medium to low graphic settings, the game is silky smooth (60+ FPS) in Timelines and in savegames with player ship counts below 1000 or so. After that, the game becomes increasingly laggy - now I have <5 FPS, especially with the map open and this savegame is essentially lost to lag. The lag is almost identical no matter how much action is happening on screen - it's mainly CPU bound and to do with too many entities in the universe. Hopefully, this save might help you identify a cause of that lag.
PC Specs:
- CPU: R5 3600
- Video card: GTX 1660
- RAM: 16 Ghz
- Memory speed: 2.6 Ghz (if I recall correctly)
- Installed on an SSD (don't think it's an M2 though)

Link: https://www.mediafire.com/file/36t4appt ... ml.gz/file
- Vanilla, unmodded save.
- Game first started in 4.0, with Cradle of Humanity DLC release. 2y+ old save.
- 11d 06h 43m game time
- Net Worth 146B
- Ships owned: 16,991
- Stations owned: 218
- All DLC unlocked.
- All sectors revealed fully with Advanced Satellites (yes I've tried disabling them all at once, it doesn't help much).
- NPC economies mature and well supplied. TEL especially have a lot of ships. Xenon have been eliminated except for the northwest cell in Emperor's Pride.

I wonder if this is normal performance slowdown, or something less benign?
- I noticed I have a lot of ships with failed orders (orange symbol on map) - could it be that those ships are actually spamming checks to sell mined ore/a cargo hold of 30 E Cells and lagging the game?
- There's a vast amount of fighter craft in this game - mine and the AI's. Position Defense (carrier command) fighters move dynamically, and so do the vast Teladi hordes on patrol in the eastern sectors.
- Xenon got whacked by me and the friendly factions - now there's no predator in the ecosystem to slow down growth in ship numbers.

Would also appreciate some input - would turfing my carrier fleets and replacing them with destroyers increase my FPS massively? I was thinking of replacing my standard 1 Tokyo/50 fighter/10 corvette/1 Honshu/2 Osaka fleet with 1 Asgard and 4 Syns in every sector.

[chew-ie]

I loaded your save and got ~1-7 fps on the map as well as ~4-10 fps with a closed map on my i7-9700k | 32 GB | 4070ti super 16GB on max settings in 4K UHD.

The moment I hit "pause" I have the usual fps I'm playing with (~50-60fps map; 50-120fps no map). I guess the amount of things going on RE your ships is way too much. Your report queue also seems to be busy all the time (there is always someone yelling about completed orders etc.) But some dev might have more fun & will get a better idea when looking into that.

On a sidenote: I tried to select all ships & delete the orders, but X4 died after me selecting 3600+ ships on the map and trying to fool around.



Anyway, impressive work you did there

[CBJ]

Thanks for the savegame. We will add it to our "crazy saves" collection, which we use to test performance and look for optimisations. There's no guarantee that we can actually improve it much, of course, as optimisation isn't a magic bullet that can simply take away the overheads of running scripts on tens of thousands of objects, for example, but it's always useful to see where to look first for any improvements we can make.

[Talinoth]

Thanks for the savegame. We will add it to our "crazy saves" collection, which we use to test performance and look for optimisations. There's no guarantee that we can actually improve it much, of course, as optimisation isn't a magic bullet that can simply take away the overheads of running scripts on tens of thousands of objects, for example, but it's always useful to see where to look first for any improvements we can make.

Thank you! That's all I wanted really. There's clearly a few mistakes I made that are causing slowdowns - helping you figure out exactly how the game can break might provide some inspiration of some kind?

For example, a way of alerting the player that they have ships that haven't actually successfully completed an action in over 3 hours and are constantly trying to search for new trades and are failing. That kind of thing.

I used brute force instead of cunning to solve many logistical problems - Vanilla X4 leaves you with fairly limited ship commands for traders (especially without good Pilot levels) so I built a lot of big Trade Stations. If the Trade Stations couldn't trade enough stuff quick enough (aka "I can't saturate the NPC faction economy yet"), I added yet more trade ships. A big culprit for the lag I think would be the Trade Stations with 200+ assigned traders - either a bunch of ships are left idle, or a bunch of ships are doing tiny trades that don't use up even 10% of their cargo space despite lots of available big orders they could be fulfilling.
- I also assigned often way too many miners and traders to stations in retrospect. Instead of carefully scaling trader and miner numbers, I just look at the Logical Overview, and say "If the Silicon bar isn't full, it needs 5 more L miners", and "When in doubt, add 20 Baldrics and 10 Okinawas". I'll be much more careful with making efficient economic ship assignments in the future.

Another culprit could be my carrier groups. I dealt with Venture invaders and the current Xenon

Spoiler

Show

/Kha'ak Crisis event

by putting a carrier + fighter group in half the sectors in the game and counting. I use Position Defense a lot - a wonderful command, but the individually moving fighters could be creating a lot of lag...

I loaded your save and got ~1-7 fps on the map as well as ~4-10 fps with a closed map on my i7-9700k | 32 GB | 4070ti super 16GB on max settings in 4K UHD.

The moment I hit "pause" I have the usual fps I'm playing with (~50-60fps map; 50-120fps no map). I guess the amount of things going on RE your ships is way too much. Your report queue also seems to be busy all the time (there is always someone yelling about completed orders etc.) But some dev might have more fun & will get a better idea when looking into that.

On a sidenote: I tried to select all ships & delete the orders, but X4 died after me selecting 3600+ ships on the map and trying to fool around.



Anyway, impressive work you did there

Nice! If you have good FPS on the map while paused, that's actually playable by my standards. Shows there might be some potential if/when I return to this save later on with a better computer (especially a hefty single-thread speed CPU).