The game uses CPU resources inefficiently

[Sprengmeister]

you don´t need to dismantle it, just don´t build it bigger and don´t go near it unless you can live with low FPS.
Also btw in 10 Years time you probably won´t play X4 but rather X5 or X6
So for all other Players: if the game can´t multitheard your Station -> Multithread your Station, literally.

[Baconnaise]

Also btw in 10 Years time you probably won´t play X4 but rather X5 or X6

If Ego released X5 or X6 today or in 2025, they would be ready in ten. I hope X6 has corps myself. I bought X3/XR/X4 regardless at launch ayyy

[Ehli]

You can't use the amount of "threads" task manager lists for this kind of information, though. Most programs use an elastic thread pool for blocking IO (e.g. file and socket operations) and a worker thread pool fixed to the amount of CPU cores. Depending on the async architecture a broker thread taking care of the message bus between threads is also present, but no sane developer wants that one to use a lot CPU.

The blocking IO pool is there to e.g. write some stuff to file and wait for it to complete, without blocking a precious worker thread and thus lowering the CPU utilization. In a perfect world the threads of your worker thread pools are constantly calculating stuff, while the broker is stealing some context switches with start/finish messages between the worker threads and the blocking IO threads.

But alas, there's never a perfect world like that: it's typically very hard to get big, meaningful work done by the entire worker thread pool, because the units of work either too small (and thus more chitchat/synchronization) or because the worker tasks themselves are not CPU bound, but memory IO bound. As a result, even though having 8 threads in your worker thread pool it might be that the average utilization is only 2 core worth of work.

Remembering some old threads with CBJ's reactions I think he once mentioned X4 is very much memory IO heavy, so the CPI (cycles per instructions) will not be great due to having to wait for memory. Using a profiler you can spot these kinds of issues by seeing long waits when de-referencing a pointer, especially when the address is not in the CPU cache. Fetching from uncached memory (memory which has not been access recently) can easily take 200 cycles.

TLDR - you can't really analyze a lot using task manager, nor use the reported thread count. It'd be easier to hang the program in e.g. Intel VTune, but without symbols/sources.. heh. You can get some impressions by adding up the core utilization, but even making claims it only uses "x" threads is difficult due to above issues. You'd need a developer to explain you, which I think was done a little by CBJ That information is worth a lot more than any external observations, even while being a developer yourself.

As for comparing it with other games, well, it's been said already - the amount of real-time calculated entities is staggering compared to the AAA games mentioned here. Factorio might be the only one which is potentially comparable, but only a developer from either/both of those games could answer that.

[Zanthra]

The fact that the save performs well when not near this megastation indicates that this issue really has little to do with the economy simulation, as that simulation has to be calculated regardless of proximity. I don't think that the X4 economy simulation is really that CPU expensive. From what I have seen, the economy updates every 5 seconds or so, only on those 5 second ticks do trades complete or automated orders update, and between those 5 second ticks, I don't think any CPU is being spent on the X4 economy calculation.

As for other game comparisons. Distant Worlds 2 is probably the most similar comparison to the X4 economy simulation, although DW2 can dwarf X4 in the number of ships and systems involved, but that's not surprising as it's a pure 4X game.

Whatever does cause the slowdown near the station has to be something that is only handled when in proximity to the station. If I had to give a wild guess, it would have to do with calculating fine pathfinding and collisions for ships near all the station modules.

[Shark22]

Unfortunately or fortunately the problem is not in the ships. I checked it on my station with 1400+ modules and 134 mining ships. I sent all the ships to the neighboring sector and flew closer to the station. With or without ships, the performance is the same. I still think that the problem is not in the number of station modules, but in how much the station produces resources and makes trade deals. Perhaps when approaching the station, the game starts to use a different method of calculating the economy, which sharply reduces productivity. I can't find another reason why even inside the station the FPS is 3-5 times lower than usual. Perhaps I will check this by creating 2 stations in one place, but 1 will be without production modules.

[CBJ]

It has nothing to do with production. As one of the previous posters said, one of the factors is physics. Another is the fact that the game has more work to do preparing large things to render (even if sometimes they turn out not to be visible - this is one of the things it has to work out). And yet another is that when you get close to a station, the game has to deal with doing calculations associated with nearby docking bays and the things on them.

[Shark22]

It has nothing to do with production. As one of the previous posters said, one of the factors is physics. Another is the fact that the game has more work to do preparing large things to render (even if sometimes they turn out not to be visible - this is one of the things it has to work out). And yet another is that when you get close to a station, the game has to deal with doing calculations associated with nearby docking bays and the things on them.

Yes, you are right. I have done several tests on stations with production modules and defense platforms. I also checked it on my station. In the new game I placed it in the same place as in my save and the difference was only 5 FPS. Most likely, this is due to the difference in the scale of the economy of the new and old games. But despite this, I still found an element that significantly reduces performance. It turned out that the turrets and shields on the modules are to blame.
I have done several comparisons on the same stations:
- A station with 480 Energy Panels and several modules holding them together. With a full set of turrets and shields, I had a little over 30 FPS. Without them, the FPS became higher than 70.
- A station with 160 disk defense platforms and 800 bridge defense platforms. With turrets 10-15 FPS, without turrets stable 60 FPS.
Considering that my Mega Station has 610 defense modules, maybe there could be 40 FPS next to it instead of 5-10 if there were no turrets and shields on it.
It's pretty sad that because of elements that I don't even see on my huge station, the performance drops so much.

[Shark22]

By the way, the situation is similar with the ships. I teleported to the bridge of my aircraft carrier, where 130 ships were docked. When the ships were docked, I had 65-70 FPS. After they flew away, the FPS increased to 85-100. This is a pretty big difference.

[Alan Phipps]

That seems rather logical as, in IS high attention, every turret group will be looking individually to detect targets in range and line-of-sight coverage.

If any valid potential target is detected then each turret will turn and start tracking even if it never engages. It will further be computing likely hit probabilities to see if there would be any point in firing (eg slow bullet plasma turrets usually not engaging fast and nimble crossing targets).

With many station turret groups, all that would add up to a lot of additional 'real-time' computation to maintain. I expect there would be some similar IS high attention increase in computation for ship turrets and for defending ships and drones in deciding whether to launch and engage too.

[Shark22]

Yes, that makes sense. Now I'm trying to figure out what exactly is affecting performance close to my station (like any other, but not that much). Performance drops off significantly at 80km. My station probably has over 50,000 turrets and shields. Drawing that many things certainly affects performance, but does it happen that far away from the station? Or is it all about the turret physics?
You wrote that turrets do the calculations in real time. Does that only apply when the turrets find a potential target to shoot at, or does it apply all the time, even if there are no targets?
I also did another test with a carrier. As I wrote before, it has 130 ships, but only 30 M-class of them have turrets. carrierhas 21 S-docks, so I first sent 21 ships to the adjacent sector so that there were none in the outer docks. This gave a small FPS boost. After that I did the same with the remaining 79, after which the FPS increased even more. So some calculations are done even for ships in the inner docks, which, in fact, can do nothing until they are given the order. For ships in the outer docks, I understand that there are physics calculations so that they stand still and fly together with the ship, but what is calculated for ships in the inner docks?

[Shark22]

Well, I made some more comparisons.
Based on what you wrote, I assumed that the attack radius of the turrets could affect the complexity of the calculations. Surprisingly, this turned out to be not the case. I checked this in 3 variants of the station, each of which consisted of 200 earth disk defense platforms:
1. Laser turrets 60-40 FPS
2. Pulse turrets 80-55 FPS
3. Argon plasma turrets 60-80 FPS (60 near the station)
In each case, I flew away from the station in the same direction for 100 km, and then approached it. Of course, as I approached, the FPS dropped (except for the 3rd case, I don't know why)
Next, I decided to check whether the physics of the turrets really seriously affects performance. For this, I installed the same 200 disk platforms, but only with shields. In this case, the FPS was 75-80. (by the way, without shields, the FPS was exactly the same). But in this case, there were fewer elements, because I did not install turrets. For the purity of the experiment, I built 2 stations, but with 1600 defensive platforms. In the first case, there were a total of 19,200 shields. I also flew 100 km from the station and began to approach. The FPS decreased from 40 to 30 (30-35 close). In the second case, there was the same number, but turrets. The FPS decreased from 30-10 (15-20 close). Without turrets and shields 60-40 (35-40 close). I find it difficult to answer whether this is related to the physics of the turrets. It is more likely that this is a visualization problem and the comparisons provided confirm this. The FPS is different with different turrets, because the turrets themselves are different in appearance. At the same time, turrets have less FPS than shields, because they are larger and more detailed.
In general, everything is logical, but here's the question: Why draw turrets and shields at a distance of 100 km, if at a distance of 5 km I can no longer see them on a 32-inch monitor? The same thing happens when docking with a station. When docked, I see about 0 elements out of tens of thousands, but the game still draws them. Because of this, even small stations in large numbers can reduce FPS by up to 2 times, at least for me. Limit the radius of drawing small elements to 5-10 km, disable drawing small elements when docking with a station and see what happens. It's a pity there is no such button in the game.

[Sprengmeister]

Simple: it draws them because under 100km you are in Sector, so ships, turrets and stations are drawn like you are there.
If a fight happens, everything will be calculated like you are there. If you are, let´s say, 300km away, everything will be calculated like you are out of sector, so you are not there.
If you are far away (300km and OOS) and then select a ship or station close to your Big Station and use F3 (Target camera), it will load In Sector, as mentiond and your FPS will drop.

[Alan Phipps]

Yes Sprengmeister is correct. It's not the turret group engagement range that matters, it's the range at which the turret groups start to exist and start detections to see if any target is in range and coverage etc. The switchover threshold/range is the one when going from low attention to high attention simulation methods, and that doesn't just affect turrets groups and assigned ships and drones etc. It also affects for example whether individual asteroids exist and are drawn, and hence a more complex methodology to be used by mining ships. It is a very important threshold in computational terms.

[Shark22]

But I don't understand why such a large radius was made. To watch from afar how the station shoots at enemies? The most I will see is traces of shots, but not turrets and certainly not shields. In my opinion, such a large loading radius is not necessary. 99% of the time the station is not attacked, but it still requires a huge amount of calculations for rendering, because it can be attacked. Maybe it is worth adding some kind of trigger so that these calculations begin after a certain event, and not all the time? For example, let's compare this with an attack on a friendly station by a player. If you give the order to attack, the station immediately becomes hostile and will attack the player. I assume that something similar happens with bots. It receives an order to attack my station and becomes a target for it. After that, you can start calculating. And it is advisable to do this when the player is at a distance of 10-20 km from the borders of the station, and not 100 km and does not even look in its direction. As far as I understand, these calculations are currently performed all the time and for each turret separately. But in essence, the station is one huge turret. It is stationary, the area around it in which it can attack the enemy does not change. Why not make it so that the space around is scanned by this one "giant turret", and not tens of thousands of small ones? And as soon as an enemy appears in the kill zone, calculations and drawing of ordinary turrets begin. Ideally, it would be possible to divide the station into sectors, let's call them mega-turrets. For example, divide the station into 8 identical cube-sectors. As in the 1st example, they also have an unchanging kill zone, but each of these turrets can shoot further in a certain direction. And as soon as an enemy appears in the kill zone, all the same physics calculations and rendering begin, but for turrets only in 1 sector, and the other 7 do not spend resources on calculations until the enemy flies even closer, getting into their kill zone.
For asteroids, the rendering range is needed. They can at least be seen at a distance of 100 km and you can crash into them. But reduce it for turrets and shields. Shields do not need it at all. 5 km is enough for them (without zooming in, you can no longer see them at such a distance).

[Alan Phipps]

I suspect that having one or more new 'attention' levels between low and high would be a massive game design undertaking and not something just dropped into an existing game engine loop.

It's also not really about what the player can see as that's more a gpu aspect, it's what those various interacting game assets and the game loops now have to start doing (or start doing in a more complex/detailed manner) as you become closer to those assets that will increase load on the cpu. Just my non-coder opinions though.

The potentially amusing thing given the thread topic is that low attention is the game using cpu resources more efficiently - but it is just not 'realistic or responsive' enough for player interactions and observation in person.

[CBJ]

I suspect that having one or more new 'attention' levels between low and high would be a massive game design undertaking and not something just dropped into an existing game engine loop.

We actually have about a dozen different attention levels, with about two thirds of them falling under the broad umbrella of "high" attention. As a simple example, we distinguish between "visible" and "nearby".

I don't have time to address all of the points in the OP's last post individually. To summarise, though, there are some false assumptions, some correct assumptions with conclusions that don't work in practice, and some other correct assumptions with conclusions that describe things we already do.

[alt3rn1ty]

I thought station turrets being an FPS killer was a well known thing.

Something I always do when building a station is change its loadout from default to custom, so that the station manager will not go building a ridiculous amount of turrets into all sorts of modules like storage, and bring my laptop to its knees (got the T-Shirt, rebuilding that station was a PITA).

There should be a government health warning against Station Loadout being set to Default