An optimization pass on the game would be an extremely welcome sight

[Eyeklops]

On a modern PC, when not GPU limited, X4 is predominantly RAM bandwidth constrained during large battles and when simulating highly populated universes.

The data below shows X3D's dominate in performance of the Young Gun scenario. Young Gun is a fresh start situation with relatively low population. As the universe becomes older and more complex (Dense empty) we see the Intel processors closing the performance gap. Finally, the most intensive benchmark shows the Intel processors pulling away from AMD's.

X3D's dominate in small universes because the massive L2 cache has a much higher "hit ratio" than Intel's smaller L2 cache. As the universe becomes larger, the frequency of L2 cache misses increases and CPU's with better performing RAM subsystems (usually Intel) take over.




To demonstrate how sensitive X4 is to RAM bandwidth I'll quote myself from another thread. The important thing to notice is the delta change of performance vs the delta of ram bandwidth in relation to the benchmarks universe complexity.

13900KS & DDR5-8200Mhz with tuned timings.
NOTE: Parentheses is change from untuned DDR5-8000.

Game: 6.10 with all DLC
CPU: 13900KS (EVGA Z790 Dark Kingpin stock settings with 300w limiter)
GPU: RTX 4090 (stock settings)
Ram: 2x24GB DDR5-8200 CL38-49-49-36 CR2 (tuned XMP profile)

Resolution: 2560x1440
Graphics Settings: Low
Young Gun Empty: 163~165 (+3%)
Dense Empire: 40~42 (+7%)
Dense Empire Empty: 143~146 (+7%)

8200 with tuned primary and secondary timings. Not fully stability tested.
AIDA64 Cache & Memory Benchmark
Read 128.73 GB/s (+7%)
Write 125.62 GB/s (+15%)
Copy 122.25 GB/s (+12%)
Latency 54.7 ns (-10%)

I would say the data suggests "Dense Empire" scales fairly linearly with memory bandwidth/latency for my CPU. I don't think I can get much more than 8200Mhz for a "daily" tune. I guess I'll have to settle for 40 FPS in Dense Empire.

[NightIntruder]

I believe in developers. They had years to find a way of optimizing the late game experience. Huh, they've already done it to some extent. They know this particular issue hampers the possibility of significant expansion of the current X4 universe or enriching the game with additional complex mechanics. It's a big shame as it also leaves us players with somehow already defined game, in most part. I don't believe we can have late game's great performance without sort of LODs of calculations as OP pointed out. I don't believe Egosoft is willing to rewrite the game engine to prolong it's life cycle with long line of DLCs, either. But, it's just my (uneducated) observations as a player, in which I honestly would love to be entirely wrong

[Captain_Canard]

Being like this, I honestly think the game would have become much more scalable, much less reliant on raw computing power and would be so much closer to both what we see in promo videos and what we 'see' inside our mind's eye.
Thanks for reading a wall of text. Hope it helps someone, someday.

I used to defend this game's performance, because it performed well for me. But I've been testing the beta, and performance is rubbish compared to 7.0 / 7.10 (I skipped over 7.50). Early game with some small fleet skirmishes brings down fps to 25 or less, no matter which direction I look in. I ran performance monitor in the background, and it's recording under 40% CPU utilization and only 1% (!!!) GPU utilization. So explain to me these terrible framerates? I'd be happy with a locked 30 fps, but anything less starts looking like a slideshow.

I didn't have this problem in 7.0, so now I'm wondering if it's the beta or other changes made in 7.50. I'm abandoning the beta for now and testing 7.50 to see if performance is better or worse. It's hard to believe that 2.60 is a "performance update", seeing how terrible it is compared to earlier versions of the game.

UPDATE - I just repeated the test in 7.50, actually provoking an even bigger battle, and performance remained rock solid for me (a locked 30 fps which is what I've always played this game at). So it appears 7.60 is indeed the culprit, well at least for me.

[Daemonjax]

It's hard to test performance in this game, but especially so when using different game starts using different save points -- it's not really an apples to apples comparison.

[Shadow_rainbow]

You keep demanding that other people listen to you, but you don't seem to be prepared to extend the same courtesy yourself.

Your thread title is also grossly misleading. You're not talking about an "optimisation pass"; you're talking about a completely different game that doesn't share the core simulation features that, for better or for worse, make X4 what it is. Hand-waving about statistics and fractals does not change that.

You're welcome to discuss your ideas for a game, but the way things are currently going this will end up being retitled and moved to the off-topic forum. And don't be too surprised if people stop engaging with you if you keep insisting that they don't understand their own gaming preferences.

Not only do I assess other's opinions and try to answer most notable points in their replies, but actively ask them to present user stories (aka their player experience) where mechanic they keep calling core matters in exactly the manner that is implemented right now. I do believe listening to others, even when not agreeing (especially when not agreeing!) with them entails exactly that.

As for the thread title, it correlates to my observations described in the first paragraph of the opening post. The game slideshows at the aforementioned segment. Loading takes a lot of time. I'm not the only one having issues like this. As such, the game must overcalculate -- something, somewhere. There is no misleading here to my knowledge. Some of my suggestions further are just that -- suggestions. As far as I see it, we're just discussing some of them without going overboard.
Overall, this thread demonstrates a lot of takes on X4 performance in general, including some thorough descriptions of currently implemented ingame optimization and some of its problems. That alone makes it a valuable resource in my eyes.

The only thing that I firmly insist on is real-time calculation is not the only way to have every aspect of the galaxy state calculated in an exact way. And X4 does try and limit the calculus in some ways (like IS/OOS discussed in this thread). Like I said, I would not mind Egosoft extending this policy to greater lengths -- but I make no demands of such nature. Discussing it doesn't seem actively harmful -- after all, people can dream, can't they?

[Shadow_rainbow]

like this one? It works

No, I mean when ship does not disappear around the corner, it works very different compared to when you're dealing with GTA NPC which appears from nothing and disappears into nothing. As repeatedly stated. It is not about "user story" it is about being able to do it. The ship is real, has goal, orders, route and so on. It matters and makes a difference.

Personally, I don't see this difference. E.g. random traders snooping to and from for me are just that -- either booty to plunder or parts of some supply chains. Random pirates harassing my traders are just that -- random pirate activity I can do little about in general.
And that is one of reasons I'm asking you to paint a player-interacted scenario where this specific implementation is important to the player while not enforcing this approach will cause a loss of something. Just please have in mind I'm not talking 'disappearing around the corner' or 'into nothing'.

This line of reasoning is about expanding the possible universe, first and foremost.

Simulation quality is more important than size of universe. We already have Elite Dangerous. It is bigger. It is also unbelievably dull and that's why I'm not playing it.

And when we don't sacrifice simulation quality?
Also, please stop invoking ED. If you are to invoke an example, please do so with comparable example. Like I do with Falcon 4/BMS or CFS3 which operate within the same space of purpose, interaction, complexity and locality.

For FPS drop in large battles, there's another line of reasoning, similar to the first one, but different. It's just over there, along with the first one.
Both do track every bullet and can be implemented in a way that tracks every ship

For FPS drop in large battles that's the point where you ignore any ideas you have and sit down with a profiler to identify the actual bottleneck which most likely has nothing to do with the assumption. This is one hundred percent "you need source code" situation. This is also a very classic situation where you do not assume that you know why it is slow.

The game becomes slow because it starts overcomputing. You don't need a profiler or a source code to see that. All of the suggestions I have voiced scale computational demands down, if implemented the right way. I have no objections to calculations themselves be optimized, but that alone presumably can't push the bar down a whole lot -- all it'd take to lose that gain would be just a bigger battle.
Of course, chunk optimization will hit a similar limit at some point. But there comes another level of optimization after that to deal with it -- if required.

Of course I understand your concerns. But simply voicing opposition to the change can't help us in working out a common vision of any kind, even if would be agreeing to disagree.

There can't be a common vision, when you attempt to kill THE reason for playing the game. The proposal is fundamentally unacceptable.

The simulation has to stay. A pass of looking for possibility of algorithmic optimization could be done (the usual. Faster collision queries via spatial optimization, parallelization, switch to advanced instructions, data-driven design, seeing if you can offload computation onto GPU, etc), but the simulation cannot be dumbed down, cannot be removed and cannot be turned into smoke and mirrors.

It is THE killer feature. And in situation where it is not important for you a reasonable idea for you is to play something else. Another game where this feature does not exist. That's because there is only one X4 and a lot of other games which do not do what x4 does.

But what you're talking about is already visibly hurting simulation quality. I've seen graviton turret lob a red ball at me through K's hull just a few hours back. And if calculations will continue to simplify, it will take more and more toll. Of course, higher parallelism and GPU-offload can, once again, push the problem back a bit (the latter, probably quite a lot, actually). But I'm not sure at all it can be pushed back as far as game stop or maximum saturation.
And vice versa, what I'm talking about keeps the simulation without losing its quality -- just changes approach to calculating it.

[Shadow_rainbow]

Nope. To improve it. That would also enable scaling the whole thing up, as a bonus.
And if you carefully read what I've written, I have described a way to calculate battles of virtually any size without much in-time computing effort. And having large battles is a good thing.

Spoiler

Show

We, the players, do the "scale up" very predictably. Whenever the devs have made a pass of optimization that keeps the game from falling to its knees, we build more and bigger to bring it back down. Whenever we buy faster computer, we again build more and bigger to bring the game back down to its knees. We don't have to, yet we do.

You say that you did describe implementation for "large battle". Did you mean a battle where the player is in the midst of it, or a battle fought in some far away sector? The bits I've read were all about "far away". (One needs incentive before carefully reading repeating walls of text.) Why should we care about far away battles?

Statistically, all that matters is the rate of credits accumulating into our account. Battles mean death, death means trade, and trade means profitsss. The remote battles are indeed not important -- as long as we get the credits. Nothing on top of "add credits to player account" needs to be done, no visuals, no tracking, no computations. If X4 would go to that, then it would become very boring (at least for me).

(The devs could re-introduce the old school Xenon Q: anything and everything you send against Q will be oneshot dead. Same as pressing "Self Destruct" button on your ships -- if they had one.)

Yes, I have described implementation of optimizing a large battle with player in the midst of it. Thing is, it doesn't have to differ from a battle in other far away sectors. If segmented properly, it would provide only a minimal load while maintaining pinpoint accuracy of the events. The difference would be just visualization of what's happening around player's segment (and stuff happening directly inside player's segment of a battle).

What you have described is a management game. I am not as accustomed to the genre as to sims, but I have an impression there is some assortment of good and captivating games there, like Evil Genius, DEFCON or Plague Inc. X4 tries to be that, as well -- but in a very simplistic way, with very little tools and very little usability. But that is another matter altogether.

[vvvvvvvv]

You're revisiting months old thread.

E.g. random traders snooping to and from for me are just that -- either booty to plunder or

Well I do see the difference, and scenarios were provided before. Tracking ships via IDS. And I'm gonna invoke ED as many times as needed, because it is a perfect illustration of a wrong way to do it. All ships in this game are fake. As in you arrive at distant carrier in the middle of nothing, and t here are freight ships buzzing arounds none of them delivering a thing to the ship.

Why should I prove you what I want and paint the picture? If I say that it matters for me then it is true. If you say it does not matter for you, that changes nothing, as I'm not you.
If you try to convince me that this feature for me is not necessary, it is quite bold of you to assume that you know better than I am what I like and what I do not like. My preferences exist for a reason, you not seeing the reasons does not make them disappear. At the end of the day if I say this feature matters to me then it is true. Nothing to discuss. If you say the thing that matters should be removed, OF COURSE I'm gonna object.

The game becomes slow because it starts overcomputing. You don't need a profiler or a source code to see that.

No, you absolutely need both.

Saying "Oh it is so OBVIOUS" is a common programmer error. People ASSUME they know what the cause is and waste time. That's the reason why "premature optimization is root of all evil".

Operate with expectation of your assumptions of performance being wrong by default. Only trust tools.

But what you're talking about is already visibly hurting simulation quality. I've seen graviton turret lob a red ball at me through K's hull just a few hours back. And if

That's not simulation quality that's someone forgetting to toggle collision flag when ship moves from OOS to IS. A bug and not performance optimization. Only affects Ks sometimes, appeared around 7.50.

A ten years old computer should be able to handle roughly ten thousand graviton balls, by the way. On a single core. You need the right collision algorithm. Sweep and prune works.

[Shadow_rainbow]

You're revisiting months old thread.

For me it dates just a single argument back. Nothing else has changed, so everything in question is still valid.

Well I do see the difference, and scenarios were provided before. Tracking ships via IDS.

OK, Tracking ships via IDS it is. Now, how do you receive the specific ship's ID and what do you use this tracking for?
E.g. in user story I have linked above, a pirate has made itself notable to the player when it attacked and destroyed his resources. Its ship's ID has become more than some random number. Before this event has occured, it held no real difference from dozens upon dozens of the same random pirates. So if it has appeared out of thin air at the moment just before the engagement, the player would not know the difference. What's more, after some time the player wasn't able to find this pirate. As such, if this notorious pirate has dissolved into a pirate threat statistics after a while, he would also be oblivious to that.
So, between these two events this ship was notable and its ID held some meaning. Before and after it was just some unusable information -- was it not?

I'm gonna invoke ED as many times as needed, because it is a perfect illustration of a wrong way to do it. All ships in this game are fake. As in you arrive at distant carrier in the middle of nothing, and t here are freight ships buzzing arounds none of them delivering a thing to the ship.

It can be used as a story of how not to implement stuff relevant here, but its world is semi-static in general. You push some numbers around with your actions, but that only reflects on event accessibility in systems, and only at the extremes of those numbers. In between, there's little (if any) difference. As such, I fail to see how invoking this example can help us in anything but illustrate difference between fully living world and a (semi)static one. Since we're discussing different implementations of fully living world, this example does not seem to hold value. Aforementioned Falcon 4/BMS is much better illustration since it is a relevant example with living world, not a static cone.

what I like and what I do not like

You have explicitly explained what is it you like. I'm not questioning that, I'm asking about why do you like it this way and why would you dislike it another way.

The game becomes slow because it starts overcomputing. You don't need a profiler or a source code to see that.

No, you absolutely need both.

Saying "Oh it is so OBVIOUS" is a common programmer error. People ASSUME they know what the cause is and waste time. That's the reason why "premature optimization is root of all evil".

Operate with expectation of your assumptions of performance being wrong by default. Only trust tools.

Premature optimization is a must for any project. Otherwise, you end up with a project designed from bottom to top in almost evolutionary way (which is exactly what you want to avoid). You calculate computational complexity of all branches of your code, you set them up in threads to sync in a fashion that minimizes wait time (being fully aware of input device/OS-induced or other delays), you have certain plans and expectations on how the project will operate way before you write the first line of code. Some projects (real time ones, mostly) are impossible without this approach. Non-real time ones - like OS, general software, games, etc - can be done with down-to-top approach, but this is a straight up mistake in engineering it as a system.

And no, if available computing power does not allow for the computing program to compute in its allowed time space, it means it overcomputes -- something, somewhere. This does not specify where the does the overload occur -- in one of many system queues, in memory bandwidth or wherever else it can get stuck -- but simply states the calculation required is not on scale with computational power available. What you know is when it happens to an ingame world, this is a problem.
What you need system tools to analyze the code for, exactly, is understanding just that -- where and why does this problem occur. But using those is a job for a person trained in a system analysis and certainly not a simple user. Besides, I do not believe X4 code base is open-source, so only Egosoft employees must have access to the key part of the equation.
What's even worse, knowing the point where it all starts to clog up, does not mean you have a solution. Unless you're doing parallel RnD and building your project from the ground up. Which is bound to create much more problems down the road -- to the point of complete redesign large parts of the whole project (top-down, preferably).

Besides, even if you have your choke point -- what do you do? Overclock your memory? Replace your whole rig? But this won't solve the problem in principle -- it will still slow down, just in a bit more time/a bit larger engagement. See Eyeklops' post above to understand the problem is not in computing hardware.

But what you're talking about is already visibly hurting simulation quality. I've seen graviton turret lob a red ball at me through K's hull just a few hours back. And if

That's not simulation quality that's someone forgetting to toggle collision flag when ship moves from OOS to IS. A bug and not performance optimization. Only affects Ks sometimes, appeared around 7.50.

I remember seeing stuff successfully shooting through corners not only with K's turrets some times before. So it does not appear to be this specific bug.

A ten years old computer should be able to handle roughly ten thousand graviton balls, by the way. On a single core. You need the right collision algorithm. Sweep and prune works.

As for tens of thousands bullets, at least in aircraft sims they generally aren't computed directly. Instead, they are checked for object intersections via trajectory analysis -- so you can have about as many as you like. It's what happens when these hit something that can stall the game engine to a halt.
And yes, computing power today is day and night compared to what what existed in Descent Free Space times. BTW you can stall its engine too (which some enthusiast-made modern campaigns for it can do even on modern hardware), but never with Volition-produced missions. Knowing engine limitations and working around those is just as essential as making the engine work in the first place.

[vvvvvvvv]

For me it dates just a single argument back. Nothing else has changed, so everything in question is still valid.

Well, months ago you've already received the response. For people this feature matters. And here we are again.

However, you've asked a very good question later, so I'll address it

One detail:

but its world is semi-static in general.

You can now spawn new settlements within it. But that's beside the argument.

So, the very good question:

I'm asking about why do you like it this way and why would you dislike it another way.

It matters because it affects the way I perceive the world and amount of enjoyment I derive from it as a result. Here's the breakdown:

Elite ships are fakes. They are flying decorations with some guns and armor, if you blow them all up, nothing changes (other than your reputation). That makes them boring.

X4 ships have routes and schedules, every single one of them. For example, one way to discover a station that definitely has a black market is to follow a smuggler and see where it docks.

That makes the world feel a lot more "real" and meaningful. Effectively this is a dwarf fortress approach which also tries to simulate entire universe and has "legends"(I think) mode which allows you to go through world history to the very beginning, and for example, see whom exactly a local deity killed, when, and where the deity came from. Critter can elevate to sort of "celebrity" status become named, and then NPCs speak of them. X4 doesn't go this far, but it is the same type of approach. Full simulation.

Your proposal is to remove this full simulation aspect and approximate it with statistics. However, this will kill the part where every ship has a chance to interact in one way or another with every other ship. So this will result in less varied outcome, and me knowing that approximation was used will reduce enjoyment of the game by significant amount. Basically, imagine that you're writing planetary generator. One is perlin noise the other is tectonic simulator. Tectonic simulator is more likely to provide an interesting outcome, meanwhile perlin noise will be one of the permutations of the base algorithms. That's the reason why procedurals are often boring. When they do not simulate history, they produce highly similar variations of a base template. This can be seen in NMS and Elite.

Therefore for me approaching X4 simulation as optimization is not an acceptable solution, because this is a major feature, one of the main reason to play it, and that's the only game that does it. That's literally trying to kill the killer feature. The optimization will remove the core reason to play. And instead, while pursuing optimization, I'd be pushing for greater parallelization, gpu compute, data oriented design and so on. While keeping the feature intact.

However, that requires profiling and source code access. Because the reason for poor performance can be quite crazy. For example check this article out:
https://web.archive.org/web/20140113221 ... -by-value/

This is from 2009. In this situation programmers described in great bewilderment, that their code performed faster when they used copy constructors that copied by value vs copy constructors that copied by reference, even though common sense at the time said it should be the opposite. The reason was compiler copy elision. Copy by value was costing nothing instead of copying field, while copy by reference was doing extra work. This was addressed later in next standard (C++11?) with move semantics.

This stuff is common. That's why do not trust your assumption and only trust tools.

[vvvvvvvv]

As for tens of thousands bullets, at least in aircraft sims they generally aren't computed directly. Instead, they are checked for object intersections via trajectory analysis -- so you can have about as many as you like.

I believe that I should've mentioned that it was "10 thousand bullets that can collide with each other". And no, you can't have as many as you like, because they'll still chip away at CPU time, at memory, etc.

Or I suppose there's this video:
https://www.youtube.com/watch?v=s-fTSUsuyls

Which is one more reason I propose to profile without touching simulation part. Or to be more specific, without gutting or simplifying it.

[jlehtone]

And no, if available computing power does not allow for the computing program to compute in its allowed time space, it means it overcomputes -- something, somewhere.

If you want your slaves to work 25 hours per day (as in https://www.youtube.com/watch?v=VKHFZBUTA4k ), then obviously that is not possible. You cannot make them overwork like that. You have to accept that a slave can do at most 24 hours per day. Accept that you get less per day, or get more slaves on the job (if possible). The third option is to find a way to do the work more efficiently, to get same output with less work.

Your statistics is essentially a get less product solution. A change in what we get.
The get same thing with less work is more traditional "optimization".

The low attention (combat resolution) does already "get less". That is either annoying, because it differs from high attention results, or cheese that we (un)happily exploit.

What you need system tools to analyze the code for, exactly, is understanding just that -- where and why does this problem occur. But using those is a job for a person trained in a system analysis and certainly not a simple user. Besides, I do not believe X4 code base is open-source, so only Egosoft employees must have access to the key part of the equation.

Isn't it obvious that when vvvvvvvv wrote "use profiler", it did mean "the devs should use profiler"?
Didn't CBJ earlier imply that they do?

---

Aren't there broadly two cases:
A. The devs know better. No matter how clever my idea looks, the devs (already) know whether it is (not) feasible.
B. I'm a genius. The devs are not skilled enough to implement what I have in my mind; I should become a dev to get the idea into the game.

I'm not B.

[chew-ie]

work 25 hours per day (as in https://www.youtube.com/watch?v=VKHFZBUTA4k )

[Shadow_rainbow]

X4 ships have routes and schedules, every single one of them. For example, one way to discover a station that definitely has a black market is to follow a smuggler and see where it docks.

Spoiler

Show

That makes the world feel a lot more "real" and meaningful. Effectively this is a dwarf fortress approach which also tries to simulate entire universe and has "legends"(I think) mode which allows you to go through world history to the very beginning, and for example, see whom exactly a local deity killed, when, and where the deity came from. Critter can elevate to sort of "celebrity" status become named, and then NPCs speak of them. X4 doesn't go this far, but it is the same type of approach.

Full simulation.

Yup, this is correct and is effectively a definition of a living game world.

Your proposal is to remove this full simulation aspect and approximate it with statistics. However, this will kill the part where every ship has a chance to interact in one way or another with every other ship. So this will result in less varied outcome, and me knowing that approximation was used will reduce enjoyment of the game by significant amount. Basically, imagine that you're writing planetary generator. One is perlin noise the other is tectonic simulator. Tectonic simulator is more likely to provide an interesting outcome, meanwhile perlin noise will be one of the permutations of the base algorithms. That's the reason why procedurals are often boring. When they do not simulate history, they produce highly similar variations of a base template. This can be seen in NMS and Elite.

Well, technically you can rip out the world's beting heart and replace it with something like glorified Perlin noise, but that would not be the thing I am suggesting. The key feature of a living world, as you have stated correctly here, is every part of it being able to interact with every other interactable part, with lasting consequences. Did I propose cutting that out? Never. What I have proposed is clusterisation of those interactions into events, events to slices, then and pre-computing ways these slices (or better yet, sub-slices) evolve in a number of ways large enough to produce all or total most available results.
I didn't address event placement -- but that does not seem to pose any sort of a critical problem. X4 space is very simple it topology, so a lane inside a sector is generally a straight line (or a set of those if we take highways into account), a fleet is a blob, a station is a small volume, etc. Intersecting it with another lane or a hazard cloud is a trivial thing to describe.

Does that preclude interactions between each and every part possible? Not at all. On the contrary, it describes it to the last detail -- computed in a preliminary way, of course.
What's best it is can be applied to the combat itself. The game is already set in a way that interacts in such slices. Remember that each weapon has its range, so if you're eyeballing it, you're not ruining somebody's day sometime somewhere. The projectile will just dissipate on the edge of its envelope.


Let's take an example.
In the idealized current implementation Trader A ship going from station B to station C, meets Pirate D ship, then they execute an interdiction scenario, the engine calculating it at every point.
In proposed one, trading lane between station B and station C comes through a pirate hazard area, manifesting an interdict scenario with an exactly the same frequency, in some place along the line (corresponding to probability simulated earlier). Instead of calculating it at every point, the engine replays one of the pre-calculated ways this scenario happened during simulation. At every point in time, if a player arrives to the scene, he will see how did it evolve to the current moment -- and has a way to interfere, transfering the slice directly accessible to him into the only thing really required to be calculated in 'real time'.
I do not see the loss of interactivity here.

Or let's take a smuggler example you offered. How finding a smuggler ship then tracing it to its destination differ in the two models? It wouldn't. A fraction of trading lane would be materialized in a trader ship image, moving to its destination under observation, then merging back into timed capacity packet after some time unobserved.
Again, all the interactivity is still there -- this time with tracking an important target. All the other unobserved smugglers would just silently do their job as part of background statistics.

Also, you've suggested Ship ID tracking possibly causing an issue with this model. Please answer the questions asked about it, as well.

while pursuing optimization, I'd be pushing for greater parallelization, gpu compute, data oriented design and so on. While keeping the feature intact.

I'm totally for anything done on this front. Playing an extremely simple little game like X4 in 2025 with 10-20 FPS is appaling.

This stuff is common. That's why do not trust your assumption and only trust tools.

That is exactly the kind of system analytics stuff that can be addressed at the lowest level. But we aren't Egosoft system engineers and we're definitely not telling those guys how to do their job. What we are able to notice in a reliable way is general performance which can be observed with a naked eye even without FPS meter to back it up. Just like I said -- something is overcomputing. My proposition of less 'real-time' computing takes into account that game world with all its economy and flight/combat mechanics is one thing, but it's just a basic layer upon which all the others are supposed to be based upon. If even the world can't compute at stable 60 FPS, how do you, for example, have such basic thing as wingmen respecting static station threat radii?

you can't have as many as you like, because they'll still chip away at CPU time, at memory, etc.

In general, you can. For each bullet in trajectory, only starting time would need to be stored even without train optimizations and the like. That's so little data that you'll run out of space before running out of memory. And considering CPU, only intersections would require computation.

[Shadow_rainbow]

And no, if available computing power does not allow for the computing program to compute in its allowed time space, it means it overcomputes -- something, somewhere.

If you want your slaves to work 25 hours per day (as in https://www.youtube.com/watch?v=VKHFZBUTA4k ), then obviously that is not possible. You cannot make them overwork like that. You have to accept that a slave can do at most 24 hours per day. Accept that you get less per day, or get more slaves on the job (if possible). The third option is to find a way to do the work more efficiently, to get same output with less work.

Your statistics is essentially a get less product solution. A change in what we get.

The thing I'm talking about is more akin to outsourcing production to the place with more workforce. Pre-compute, then use the results of that pre-computation. And without

The low attention (combat resolution) does already "get less". That is either annoying, because it differs from high attention results, or cheese that we (un)happily exploit.

-- every situation would evolve in the exact same way no matter in or out of sector it happens, unless a player involves himself somewhere. In that case, a small area around him would have to be re-computed on his machine -- but that's so much less than everything else that it shouldn't matter.

Aren't there broadly two cases:
A. The devs know better. No matter how clever my idea looks, the devs (already) know whether it is (not) feasible.
B. I'm a genius. The devs are not skilled enough to implement what I have in my mind; I should become a dev to get the idea into the game.

I'm not B.

I'm not a genius. And I don't have to be. I do notice an apparent problem since it's right before my eyes. Having everything slow in non-epic battles is not 'better' despite the options provided. But I know having everything slow in epic battles would also not be a good thing.

[Gavrushka]

Will nothing make you accept that the game will simply challenge any computer hardware in certain instances because it's a full simulation?

As someone who has played since the first day, I'm staggered at the monstrous improvements in performance over the years. - To simply say, as an armchair wielding layman, 'it needs optimising' must feel very insulting to the team of developers, many of whom have been working for Egosoft for decades.

I don't doubt that when you're an far older and wiser version of yourself, hardware will have caught up sufficiently and you'll have all the performance you can shake a stick at.

I'd sat this one out until I saw you now are suggesting Egosoft now change their business model from a staffing perspective. Two armchairs now... Impressive.

Wouldn't it be easier to accept you're wrong and move on? Maybe contribute to the beta testing team rather than let your frustration at your hardware's inability to cope with the simulation at its most demanding manifest what is another utterly pointless thread demanding *waves hands* Egosoft 'do something'...

Maybe they already have? (Spoiler, they have, and they continue to do so right now.)

[vvvvvvvv]

Did I propose cutting that out? Never. What I have proposed is clusterisation of those interactions into events, events to slices, then and pre-computing ways these slices (or better yet, sub-slices) evolve in a number of ways large enough to produce all or total most available results.

Yes, you propose cutting this out because that what your precomputation is gonna do. You replace simulation with array of templates and you'll end up with equivalent of boring NMS/Elite planet instead of tectonic simulation. Templates are noticeable, and people quickly pick them up. Perlin noise is INCREDIBLY noticeable, by the way.

The key issue here is that you begin with correct observation:
"Performance is not good", but then you make an ASSUMPTION. "It is because of simulation!".

Nope!

The actual correct conclusion here "I do not know why it is slow, and I have to measure it"

That's where you have to MEASURE it with the profiler.

Instead of measuring it, you make a proposal to deal with simulation, without having checked if it is actually a bottleneck. That's a major programming error. Because you have to identify bottleneck first, and the bottleneck can be ANYTHING.

Meanwhile your proposals cripples the game's killer feature. As they say, "the road to hell is paved with good intentions"

You have to measure it. You really, really have to measure it. There's a very strong reason why "profile before optimizing" principle exists it is because people keep making this error. "Oh I already know how to make it faster!" No, you don't until you measure it. And for that you need profiler and source code.

For each bullet in trajectory, only starting time would need to be stored even without train optimizations and the like. That's so little data that

No, that's not how collision works. The performance killer here is not checking bullet vs plane or bullet vs bullet collision, but number of candidate pairs. In a scenario with 10k bullets that can collide with each other, you'll be looking at 100 000 000 queries worst case scenario. This WILL stall modern CPU. O2 complexity. You prune this number down, by implementing some sort of space partitioning. And then number of operations can go down to 10k from 100 mil O(n) or O(log N). Your "stored trajectory" will likely perform worse, because it'll have bigger AABB during broadphase, triggering larger number of false positives.

Either way.

Instead of trying to cripple the game's most important feature a better choice would be to upload a save with poor performance to tech support section. People who made this game aren't stupid, there's a ton of optimizations and interesting features that are not immediately obvious. That's why you need to profile and that's why you need source.

[jlehtone]

Instead of trying to cripple the game's most important feature a better choice would be to upload a save with poor performance to tech support section. People who made this game aren't stupid, there's a ton of optimizations and interesting features that are not immediately obvious. That's why you need to profile and that's why you need source.

With the "save with poor performance" being the source that the devs need to profile with.



Dynamic molecular simulations model interactions of atoms. The molecules of interest are surrounded by water. Water is implicit as the "pre-calculated effect" is usually sufficiently accurate, but some water molecules have to be explicit as they participate in crucial interactions. Those waters do not shift between statistics and explicit; they are explicit the whole simulation.


Unit A (certain fleet, load, and behaviour composition) attacks unit B. Sure, you can "pre-compute" the result. How many different compositions can there be, how many scenarios to pre-compute?

On some other games the result of attack would be 30% casualties on B, 10% on A. In X4 one side practically always suffers 100% casualties. The only questions are: which side and how many casualties does the other side take? If the outcomes are pre-computed, then the game could/should show prediction and we would took only battles that we want?

One could even suggest removing player from the game, because presence of player wrecks all precomputations. That, however, would be a totally different game.

[Y-llian]

Having a night of insomnia and opted to read through this thread. Some interesting points overall but I do agree with @vvvvvvv and @alt3mgty….

The fact that the x-verse is fully simulated really matters to me. It’s the core reason I play the game. I like knowing that out there, some Boron-friend (or whoever) is just going about their business same as me. Trading, fighting, surviving, whatever….And if I want to, I can go help them, hinder them or just spy on them until I lose interest.

But more importantly, that I know their ship / cargo etc. is made from, used by, manufactured and help drives the simulation. They’re not candy, they have a functional purpose in the dance that is the simulation. And that dance, because everything is simulated, can have unpredictable consequences that I, as the player, can exploit or just ignore to see how the dominoes fall.

The fact everything (well nearly everything) is simulated with such granularity means the universe is unpredictable, in a way other games are not and even to the extent, that the developers sometimes can’t predict the consequences. For us long timers, remember the great Xenon horde in early betas? When Ego tweaked the Xenon slightly only to find they’d created an untameable, rampaging monster? By their own admission the cascading consequences even took them by surprise.

It’s precisely that things aren’t always predictable, in a very real way, that I love this game and why, the simulation is important. Simply put everything truly matters: the raw materials, the wares, the production, the ships, the logistics, the wars, everything. Knowing it has purpose ensures everything means something and to me as a player, that really matters.

[alt3rn1ty]

1 - Worst case scenario is when landed on a large station, with the map open, and a large battle is going on nearby, even members of the community with the best desktop specifications get fps hits in that circumstance.

2 - VIG sectors with large swarms of fighters are another bad hit to fps

Just want to correct a couple of points from my first post - With the advent of v8.0 Beta 2:

Point 1 in the quote is now history if you have an NVidia 50xx GPU, Egosoft have introduced Frame Generation which is frankly quite amazing, I can now be landed on my self sufficient Shipyard, with the map open, and a battle occuring nearby and still have 120 fps constantly (my current fps cap is 120).

Point 2 in the quote is also now history, Egosoft found that VIG fighters were building excessive large swarms and have culled the source problem viewtopic.php?p=5283248#p5283248

The Vigor Syndicate apparently sneakily accumulates more and more ships as the raiding parties they send out get destroyed and the survivors of those raiding parties limp home. Fix should be in a future update. Thanks for the report and the save.

I now have 120 fps literally everywhere with v8.0 betas' (this is on a laptop, see specs in my signature below), so no need to go cutting out important parts of the game anymore. Desktop guys can easily upgrade a GPU card in the next couple of years, and usually need to in order to keep up with current gaming needs, thats just an inevitable part of life with a PC.

TLDR; Egosoft continue to find ways to make the game better in all aspects. Historically thats what they do with all of the X series games, it takes years to get to the polishing stage as they are a small indie, so people with low attention spans will unfortunately pass by and not get to experience the final product ..
.. but Egosoft are experts in what they do. Have faith