Curious as to why X4 is not configured for high end gaming computers.

[apricotslice]

This has been on my mind for a while now, while playing and modding X4.

Why is X4 not configured for high end gaming computers?

Video is, but it still actually isn't.

As far as I can see, the game is CPU bound, and RAM restricted.

My system CPU is I9-10900 @ 2.8Ghz. That's getting a bit old now. It gets up to 11% used.

64Gb RAM, but the game never uses more than 7Gb.

RTX 3090, but the GPU only rarely gets to 1% used. I've got the game on maximum settings for everything.

Monitor is 3840 x 2160 43 inch, using a Display Port cable.

As far as I can tell, the system is largely not properly utilized. And while I understand limiting the game to an average system, the average system has been getting higher and higher for years now, and the game hasn't adapted.

One of my pet peevs about the game is the fact that we're limited to 1 screen, and locked into the endless repetition of going from screen to screen and back again and back again and back again.

I use a constant rotation of either the ship or the player office, the map, station build, station overview, multiple ship and station information, and ship building.

The lag going to and coming out of station building is usually major, on the order of a minute each way. [I build stations that big. My shipyard has 3200+ modules] So you add some modules, wait a minute to get the map to find a builder ship, then wait another minute while it goes back to the build screen. Sometimes while adding modules, you need the overview to see what is already there, and that's 2 more 1 minute waits each way.

Where am I going with this?

I've got 3 wide screen monitors on my system. The main one as I said is 3840 x 2160. The other 2 are 3440 x 1440 and 1920 x 1080. All 3 can be configured to double the resolution. Added to that I've go an old 55 inch tv connected as well. And I've got 2 more unused ports.

In this day and age of multiple monitors on 3080 or 3090 GPUs, why am I limited to only 1 game screen?

I was actually considering updating my CPU and going to an RTX 4090. But then I saw just how little X4 was using my existing system. I may still do the CPU, but the RAM and GPU I have are basically not being used. [I do use Blender, so the ram and 3090 are utilized, and I may go 4090 just for that.]

The thing is though, when multiple large monitors is becoming the normal, why is the game stuck inside a single monitor, and single window on that monitor, when the constant back and forth of the main menu items just drives you insane?

This is something I would pay for as an add-on, to have a reworked exe file that spawned off each menu item to a separate window, which allowed you to position and size them wherever you wanted them across however many monitors on the computer. That way I could spread them across 3 to 5 monitors, and simply shift my chair to work on any of them.

So how about it Egosoft?

Anyone else here want something like this? And will you pay for it?

----------------

While I'm on annoying things, what's up with the Player Office?

It's only on the PHQ. And that's fine if your PHQ is bog standard, or you've only done some minor building on it. But if you're PHQ causes frame rate issues when you're in sector, you can't use it!

It makes no sense to me that the Player Office is only on the PHQ. It should be on the Administration Module as well. Or it should be a special module you can add to any station. Then you can put it on any station you build, and when you're not on a ship, you can use the office on a station where being there doesn't impact the game at all.

The other thing I don't understand with the Player Office is why it's not set up the same as being in control of a ship. There's no messages. No music. No warnings of any kind. And yet, you're running your in game entity from that office the same as running your ship from the control area.

So what happens? You lose ships because you never got any message they were in danger. Stuff happens you are totally unaware of. And it's quiet.

Don't get me wrong, the docks announcements are not wanted. But the Office should be feeding you all the information coming from your stations and fleets of ships. It should be an information multi-view, not just a place to stop and use the menu system from.

And what's with the placement of the Office on the PHQ? It's a really lousy view, and why it should be a module you can build on somewhere with a great view.

And why the obstacle course to get to the chair? Just open the travel car right there, and click the chair to sit without leaving the travel car!

Honestly, I'd love a suppress all the station movement option. The whole walk around the station thing was old back on day 2 after the original game launch. It's just annoying these days having to move around stuff.

Anyway, just wondered about it. Great addition, but like so many things Egosoft, only half done imho, or not fully thought out.

Comments welcome, but Egosoft doing something to bring the game fully into the 2020's would be better.

And as I said, I will pay for it.

[Rei Ayanami]

With regards to multi-monitoring : One big issue I see with it is controls. Once the map opens, part of your control scheme is overridden by the map controls, so from a controls point of your your ship would still be mostly uncontrollable, even if you put the map on another monitor.
Once you are in the station builder screen, controls again get overridden so you have camera controls to see your planned stations from different angles.

What if you have the main ship view, the map and your station building screen active, which, for example, should take priority for your WASD keys?
If you press the Esc key at any point, which screen/window should close? Or should the pause screen pop up?

And why the obstacle course to get to the chair? Just open the travel car right there, and click the chair to sit without leaving the travel car!

Pardon me, but what is a travel car in X4?

[Alan Phipps]

The ship teleporter compartment where you arrive in-ship, I expect

[Eyeklops]

With regards to multi-monitoring : One big issue I see with it is controls. Once the map opens, part of your control scheme is overridden by the map controls, so from a controls point of your your ship would still be mostly uncontrollable, even if you put the map on another monitor.
Once you are in the station builder screen, controls again get overridden so you have camera controls to see your planned stations from different angles.

What if you have the main ship view, the map and your station building screen active, which, for example, should take priority for your WASD keys?
If you press the Esc key at any point, which screen/window should close? Or should the pause screen pop up?

I'd love to see multi-monitor for multi-monitoring of my fleets status with the info panel for my current target. It doesn't have to be interactable until I press "M". I wish there was a program that hooked into X4 (via an api?) that did this without considering the game "modified".

[jlehtone]

Still on first page: viewtopic.php?f=146&t=449874 (Can I run X4?)
The focus can't be on the "big guns" and there simply aren't enough resources to do both.


I probably find the office as useless as you, since I fly a ship 99.9% of the time.

[Imperial Good]

Why is X4 not configured for high end gaming computers?

Short answer is that it has to be playable by people with more budget friendly or older gaming computers.

My system CPU is I9-10900 @ 2.8Ghz. That's getting a bit old now. It gets up to 11% used.

11% usage is actually close to 22% actual usage due to how hyperthreading works. Anything above 22% usage usually starts to lower maximum clock speed on higher end processors and runs into other bottlenecks so I would say that of the potential capabilities of the processor over 30% are being used easily.

The bottleneck comes down to single thread performance as ultimately to prevent race conditions there will be a single thread of execution that is holding back updating the game state.

Since the 10900 is a 10 core processor it also is not really representative of what the average player might be using (6-8 core). For this reason, practically no game optimised for 10 or more cores. Reaching roughly 30% utilisation of a 10 core processor is pretty good for most games.

64Gb RAM, but the game never uses more than 7Gb.

And the extra memory would be used for? Games only need as much memory as is useful for them to store state in. Although the X4 universe is complicated, it is not that complicated that it would be able to find a use for even 16 GB of memory let alone 64 GB.

Since most players will have 16 GB of memory games are designed to be highly playable with that amount. This practical limitation hinders using/requiring so much memory.

Outside of asset caching, of which X4 likely does not even have 64 GB of assets to cache, it is not really practical to use so much memory for the working set of a game. This is because processing so much memory at the low memory bandwidth of typical RAM, especially older DDR4, is not really practical. Even iterating 64 GB of memory will take a good part of a second, let alone doing it every frame. Most games try to build their most accessed working set of data to be small enough that most of it can fit in the processor cache, which is just a few MB in size, with the rest that is less frequenctly accessed sitting in memory of a few GB. Most games use the rest of the memory to cache assets, which is not even that useful anymore due to how fast NVMe SSD storage is.

RTX 3090, but the GPU only rarely gets to 1% used. I've got the game on maximum settings for everything.

Monitor is 3840 x 2160 43 inch, using a Display Port cable.

You can always get an 8k or 12k display. Pretty sure that will use your GPU. Especally since Nvidia marketed the RTX 3090 as an 8k gaming GPU.

X4 is quite easy for modern GPUs to render due to being built for the Vulkan API. Older GPUs like the GTX 700 series do struggle a lot with it but any sort of RTX or even GTX 1000 or newer GPU seems to have little issues maxing the game visually. Part of this is due to its setting being space, and like in real life space is often quite empty so simple to render. There is also an issue that making the render more complex can require more CPU time doing render preparation which in turn can lower performance where it matters.

The lag going to and coming out of station building is usually major, on the order of a minute each way. [I build stations that big. My shipyard has 3200+ modules] So you add some modules, wait a minute to get the map to find a builder ship, then wait another minute while it goes back to the build screen. Sometimes while adding modules, you need the overview to see what is already there, and that's 2 more 1 minute waits each way.

This does not sound right. Even for very big stations which caused my old GPU to run at 100% going into the station edit window was not more than 1-2 frames of render latency (so faster GPUs that are not taxed as much do this near instantly).

What has always caused me issues was going into the personal tab. Opening that tab can take a few seconds, because I think it has to trawl all your assets to tally what you own.

I've got 3 wide screen monitors on my system. The main one as I said is 3840 x 2160. The other 2 are 3440 x 1440 and 1920 x 1080. All 3 can be configured to double the resolution. Added to that I've go an old 55 inch tv connected as well. And I've got 2 more unused ports.

You might find that in practice this does not work as you are not using a Nvidia quadro or A series enterprise GPU. At least back with the RTX 2000 series there was a cap on 3 connected displays for consumer GPUs. Not sure if that is just for joint desktop though, or if applications can still force the displays to work via appropriate API calls.

Gaming on quadro or A series enterprise GPUs is not usually supported by Nivida.

In this day and age of multiple monitors on 3080 or 3090 GPUs, why am I limited to only 1 game screen?

Few people use multi monitor setups in a way that games can be designed to require them.

The other answer is that the render preparation for additional views would tax your CPU even more, likely lowering performance significantly even if the GPU can handle it.

I was actually considering updating my CPU and going to an RTX 4090. But then I saw just how little X4 was using my existing system. I may still do the CPU, but the RAM and GPU I have are basically not being used. [I do use Blender, so the ram and 3090 are utilized, and I may go 4090 just for that.]

Upgrading your CPU to a decent 13th generation Intel processor will see significant performance improvement. Ryzen 7000 should also have a good performance improvement but that would require moving to DDR5 memory and currently is unlikely to out perform Intel 13th generation.

The thing is though, when multiple large monitors is becoming the normal, why is the game stuck inside a single monitor, and single window on that monitor, when the constant back and forth of the main menu items just drives you insane?

Multiple large monitors are not the normal though. Even people who have them might prefer to run something other than X4 on their additional monitors while playing.

It's only on the PHQ. And that's fine if your PHQ is bog standard, or you've only done some minor building on it. But if you're PHQ causes frame rate issues when you're in sector, you can't use it!

This is my biggest gripe with the office. On one side they give you a space to sit around and manage your empire safely (HQ is literally indestructible), but on the other they want you to terraform planets so your HQ is some mega massive station with 6+ wharves, and hundreds of storage modules...

I really wish you could press a button to close the shutters in the office, causing the office to enter a void and the surrounding space to become low attention. This way you could sit at your HQ without having the HQ high attention and so perform like mud.

It makes no sense to me that the Player Office is only on the PHQ. It should be on the Administration Module as well. Or it should be a special module you can add to any station. Then you can put it on any station you build, and when you're not on a ship, you can use the office on a station where being there doesn't impact the game at all.

It kind of does make sense that it is only on the HQ. It has unique trophies that you earn during play that are on display in your office.

And what's with the placement of the Office on the PHQ? It's a really lousy view, and why it should be a module you can build on somewhere with a great view.

It is my understanding that it is semi randomly generated where the HQ office is placed. As such you can have anything from the most amazing view of your HQ to a buggy/clipped view of a module wall.

[apricotslice]

With regards to multi-monitoring : One big issue I see with it is controls. Once the map opens, part of your control scheme is overridden by the map controls, so from a controls point of your your ship would still be mostly uncontrollable, even if you put the map on another monitor.
Once you are in the station builder screen, controls again get overridden so you have camera controls to see your planned stations from different angles.

What if you have the main ship view, the map and your station building screen active, which, for example, should take priority for your WASD keys?
If you press the Esc key at any point, which screen/window should close? Or should the pause screen pop up?

Most of the time even if I'm on a ship, I stand back and let the captain do the work. So when I'm "working", I don't need ship control anyway.

I'm okay with taking control of a ship rendering the other screens inactive while you do, although still open.

Also okay with if you click into anything but the map, you automatically disconnect from the ship and the captain takes over.

@Imperial Good - Interesting info, thanks.

[adeine]

On the CPU front, it's not as simple as 'optimising' things for more cores. Unless you're dealing with a workload that can virtually be 100% in parallel (such as slicing up encoding or compression jobs) you very quickly run into a hard limit of what can be done. It's what is referred to as Amdahl's law, if you're curious to read more.

To put it simply, for games or other suboptimal workloads you can rarely expect much improvement from just throwing more cores at it. Per core performance is really what matters.

[Raevyan]

I guess you could always run a benchmark on endless loop in the background if you feel your system is not used enough.

[BitByte]

Stupid question but what the heck you have in your shipyard if it have 3200+ modules?

[apricotslice]

Stupid question but what the heck you have in your shipyard if it have 3200+ modules?

It's a closed loop.

So everything needed to make ships in large quantities, when you supply enough resources.

That includes something like 200 mil in container storage, and 120 mil each in solid and liquid.

It wasn't supposed to be that many, but it ended up that way after I kept tweaking it to get it as balanced as possible.

[Scoob]

If CPU tech had continued to go Ghz - i.e. a high-end gaming system today was an 8ghz Quad Core with HT - then X4 would doubtless run like a dream on such a system. However, CPU Tech went more cores with only a moderate bump to Ghz. Many games I have make good use of the multiple Cores on my three year old 3900x. CPU is only lightly taxed overall, but there's a good spread of that CPU load over multiple Cores / Threads, so they run beautifully. Note: these are demanding games, using significantly more CPU resources overall than X4, they just thread really well. X4 on the other hand, while it does use several Cores - and is certainly better at this now than at launch - seems to place the lions share of the load on ONE Thread. This means that one Core is pushed hard in many instances, limiting the game.

There is constantly talk about the economy being complex etc. and that's why performance is how it is. However, this is NOT what I see. It's NOT the wider economic simulation that limits performance for me. Ever. It's what's going on in my current sector 100% I can be near the biggest battle, the largest complex, the most trade ships etc. and get a marked slow-down while In-Sector. I teleport away and fps recovers fully. Sure, those things are still going on in the background Sim, but they have minimal fps impact, which is great. It's all about the In-Sector goings on, which appear often be able to push one Core to its limit. If there were a way multiple cores could be better utilised for this In-Sector stuff, I imagine it would help immensely - or if my hardware could boost one core to 8ghz lol.

Perhaps, with some many game-week long games where the player has totally saturated the Universe with Stations and Ships, the slow-down might be more consistent irrespective of what's going on in the player's current sector / locale. But even with my own games lasting weeks, I've never hit that point. Game fps drops below my (chosen) 60fps vSync almost exclusively only when something is occurring in my current sector, close enough to be high-attention. Battles are the worst culprit here, but also station traffic on larger stations, and the act of Wharf-style docks launching Fighters / Drones often has a high impact too. When standing ON a station - on foot - the presence of NPC's wandering around can have a signifiant impact on performance. You can test it easily. When you first teleport to a station, it can be several seconds before the wandering NPC's are spawned in. Until they do, I can regularly see solid FPS, with it dropping markedly when the various NPC's spawn in. You can also test this with a station denied Food / Meds, as the NPC's leave, performance on the station gets better.

RAM utilisation is a funny one. I've had games - modified - using 16GB consistently as the game matures. Problem is, Windows throws a bit of a wobbly here at times, and hits the page file too keenly, meaning stuff the current scene needs NOW has been paged out. I hit some dire - game-ending really - performance issues a while back, where I'd be getting seconds per frame rather than frames per second consistently after loading a save. I could see, via diagnostic tools, that the data X4 needed for the current scene had been paged out (incorrectly) by Windows, so the game was constantly accessing the Page File rather than RAM. I quickly proved this by turning off my page file. RAM utilisation was exactly the same - 16GB upon loading the save - but everything was back to being a perfectly smooth 60fps. Changing X4.exe priority to "High" has a similar effect, meaning more game data is kept in RAM (if you have ample RAM, I have 32GB) and the game isn't constantly being paged out.

Note: all of this was in a mature, modified (SWI) game where I had lots of assets in the current sector, as well as lots of other Faction ships and regular fights going on. So, while performance is usually due to one Core maxxing out, this issue was related to Windows badly managing RAM and was a new one to me at the time. I'm fairly certain that a Windows update changed something to do with RAM management, as there are posts about this issue elsewhere too that I found - not limited to X4 by any means.

There are certainly obvious areas for improvement here, so, why to wandering NPC's have such a high performance impact? Why is station traffic quite demanding of CPU? Why is the act of Ships / Drones launching from a Wharf-style dock so impactful? Plus of course, more understandably, can anything be done to improve performance in battles? LOTS of projectiles looks great, and there are impacts and ship manoeuvring etc. all very busy, is there any way more Cores / Threads can be utilised in such scenarios so a single Core doesn't max out, lowering the fps so significantly?

For me personally, I consider performance to be generally ok. But then, as the game matures, I'm very much the guy ordering ships around rather than flying them myself. It is however a real shame that many of the best battles I've experience I cannot join in piloting a ship myself as the FPS is too low for fun flying and accurate shooting. I just leave it to the AI and manage the battle. The nature of the game changes over time.

Note: I've hit all these issues in mature, vanilla games. However, likely due to the extra assets, more ships and bigger battles, modified games can demonstrate the issue sooner, purely because there's simply a lot more going on.

[apricotslice]

Thanks for all that.

Hopefully that is food for the Egosoft thought processes.

But one question. How do you get window 10 to use 16gb of RAM for X4?

Mine won't go above 7.

I do run X4 on the highest priority. What annoys me is I can't find out how to get the game to run with it automatically.

[Scoob]

Thanks for all that.

Hopefully that is food for the Egosoft thought processes.

But one question. How do you get window 10 to use 16gb of RAM for X4?

Mine won't go above 7.

I do run X4 on the highest priority. What annoys me is I can't find out how to get the game to run with it automatically.

No problem.

Well, I'm running Modified games, which load more assets (more ships, weapons etc.) as well as a generally more active universe. More ships per Faction, more Factions, more conflicts etc. I've not run a Vanilla game since the 5.10 Beta I think it was, I cannot recall what sort of RAM Utilisation I saw then.

Regarding Priority, I create a little .bat file:

d:
cd Games\X4 Foundations - 5.1 HF3 (SWI)

Start /High X4.exe -showfps

This is to launch my GoG version of the game. I assume the no-Steam exe version could be launched in the same way.

Note: the shortcut to the .bat need to be "Run as Administrator".

Edit: my game folder name is "X4 Foundations - 5.1 HF3 (SWI)" to be clear.

[apricotslice]

Even on high, I don't get the game to use more than 7gb, and I've got 64gb to use.

How do you get it to use 16gb?

Anyone know if you can do the same thing as that bat file does, inside the Steam start, like you can add the debug statement?

[Scoob]

Even on high, I don't get the game to use more than 7gb, and I've got 64gb to use.

How do you get it to use 16gb?

Anyone know if you can do the same thing as that bat file does, inside the Steam start, like you can add the debug statement?

I only really started monitoring RAM usage closely when I had issues, and those were with a modified game. I really cannot recall - if I even checked - what my mature vanilla games used. I'll see if I have a backup somewhere of a vanilla save and I'll check...

I don't know about debug, never used it. Is it just another -blah option that could be added to the .bat? Or is that not what you mean?

[linolafett]

The entire game is like 21gb installed, it would be unreasonable to expect to fill up your 64gb of ram just because you got it.
I personally see no advantage in doing that.

[apricotslice]

Even on high, I don't get the game to use more than 7gb, and I've got 64gb to use.

How do you get it to use 16gb?

Anyone know if you can do the same thing as that bat file does, inside the Steam start, like you can add the debug statement?

I only really started monitoring RAM usage closely when I had issues, and those were with a modified game. I really cannot recall - if I even checked - what my mature vanilla games used. I'll see if I have a backup somewhere of a vanilla save and I'll check...

I don't know about debug, never used it. Is it just another -blah option that could be added to the .bat? Or is that not what you mean?

I've never run a vanilla X4 game. I think I lasted about 10 minutes after buying the game before I cancelled the window and went looking for mods.

-debug, yes. There's a place you put it in the steam loader, so the game generates a debug file every time it starts. Essential for mod making when things don't work.

The entire game is like 21gb installed, it would be unreasonable to expect to fill up your 64gb of ram just because you got it.
I personally see no advantage in doing that.

If it sped up the transition between menu items, it would be worth it.

At the moment, I get significant pauses when going from a build to the map to find a builder, and back again after, as well as between build and station operations. It's a significant game stops period, and a lot of that is storing all the modules in the build, or the number of modules in the finished station you're in sector with.

I'm pretty sure Egosoft has never tested the game with a station build with over 3000 modules before, limited to only 7gb of RAM.

Yes, I'm pushing the limits, but that's where the game is for me currently. Building large closed loop stations.

And the game lag between menu items is killing the enjoyment.

Not to mention what was said about FPS in sector with large build stations. That's hopeless at times.

I had to abandon putting weapons on my stations, because being in sector while the guns were firing was a major slideshow. It was totally unworkable.

[Rei Ayanami]

I'm not sure if it's even possible to force Windows (or any other modern OS) to keep the entire game and all of its assets in memory, as it's up to the OS, not the apps, to decide how to handle memory allocations and what to keep in RAM and what to cache out on disk.
Also, why force X4 to use more memory than it thinks it needs? If X4 really needs more than 7 GB at once, it will request more and will most likely get more from the OS. At times I've seen it go up to 12 GB on my PC, which has 16 GB RAM. That doesn't mean that it necessarily ran faster, though.

If it sped up the transition between menu items, it would be worth it.

At the moment, I get significant pauses when going from a build to the map to find a builder, and back again after, as well as between build and station operations. It's a significant game stops period, and a lot of that is storing all the modules in the build, or the number of modules in the finished station you're in sector with.

[...]

And the game lag between menu items is killing the enjoyment.

If I had to guess then the most likely reason why there is a detectable pause between different screens is that map data, station build data, etc are probably not updated while that particular menu/screen is not active, so when switching to a menu/screen it has to poll game data and build the menu/map from scratch. Even if everything was in RAM, it'd still take some CPU time to poll everything and build the menus. Of course, the more complex an entity, such as your 3000 module station, the longer it takes to fully poll all data for it.
Unless you want to have all screens always updated, even when they are not active, (which would be a CPU time waste), the only three alternatives would be a) having a small pause for menu entries to load (as it is now) or b) opening a blank screen and streaming in all information over time (which would look rather bad), or c) only displaying the menu after everything has fully loaded, which means considerable input lag because after pressing the button you have to wait for the menu to be fully ready (also bad). Pick your poison.

There are some screens which COULD be preloaded in a speculative manner in the background (station selected -> preload station build menu and station logistics just in case player wants to see them), but that too has downsides, like selecting one station on the map -> begin preloading that station for all of its possible menus/screens, in case player wants to select it, but then player select another one -> abort preloading previous station, begin preloading new station's menus/screens would also be kind of a CPU waste and would also most likely have some form of lag (have to wait for assets to get streamed in).

Not to mention what was said about FPS in sector with large build stations. That's hopeless at times.
I had to abandon putting weapons on my stations, because being in sector while the guns were firing was a major slideshow. It was totally unworkable.

Well, that's the limit of software. It's the same with real time strategy games without unit limits : It runs fine as long as you only build in expected amounts, but if you build 1000+ units it''s reasonable to expect that there will be some kind of slowdown. If you have thousands of players on an MMO server at the same location, chances are there will be lag. And your 3000 modules are not only an extreme version when it comes to calculating production, but also rendering time (I assume one or more draw calls per module) and possibly collision time (ships near station doing pathfinding, turrets checking for line-of-sight, etc).

And no, I'm sure Egosoft didn't test with a station with 3000 modules, because that's not what's expected to happen in 99.9% of all played games, just like Microsoft probably doesn't test for Windows having 10000 apps open and active at the same time. Of course, you can try it, but I don't think criticizing Microsoft for not having an OS able to have 10000 apps active at the same time while keeping acceptable performance would be fair.

I'm not saying that there is nothing the X4 engine could improved in, but If you test out the limits of software and perform unexpected actions in extreme ways, you have to expect to bump into limits in return (low performance, high memory usage, etc).

[Scoob]

If you have 64GB, pop the game into a RAMDrive. I used to do that, but with "only" 32GB the game is too big now.

Edit: Btw: try creating a new instance and applying the Star Wars Interworlds mod to it. This is likely on the more-demanding end when it comes to mods, and I certainly see 16GB+ used by the game process during play, my game is a few days old now though.