Now getting >90 FPS at 1080p - info on issue affecting high-end rigs [spoilers]

[Alexium]

Setting xrebirth.exe to run in High Priority helps a bit.

That can make your controls laggy because you're not leaving Windows much time to prcess KB / mouse.

[Astyrrean]

Not normally, no. GPU utilization should be at 100% once FPS starts to drop as the GPU can't keep up.

A GPU utilization below 100% is indication that something el.

That much is obvious. What I was asking is does your GPU utilization go down when your FPS drops. If it does - it makes perfect sense.
What I don't understand, though, is why FPS depends so heavily on what direction to face. In light of CPU bottleneck theory, I'd assume that CPU load shouldn't change very much depending on what you're currently looking at. Even if you don't see a highway behind you, that highway still should be processed so that it's in the right state if/when you turn around. I don't know if it's CPU bottleneck, or if the GPU is starving for some specific resource it doesn't have too many of, but it must be something rendering-related.

Got it now. Yes GPU utilization does indeed go down when framerates go down. Also note that it's not a CPU bottleneck (see the screenshots). Hence the thread sync hypothesis.

[TKz]

Got it now. Yes GPU utilization does indeed go down when framerates go down. Also note that it's not a CPU bottleneck (see the screenshots). Hence the thread sync hypothesis.

Your theory is interesting.
But if it's true, it's affecting everyone, not only specific high end systems. And if it's true, it affected the game of 100 people who tested XR before us, but none of them seen that. How is that possible ?

[TKz]

Setting xrebirth.exe to run in High Priority helps a bit.

That can make your controls laggy because you're not leaving Windows much time to prcess KB / mouse.

High priority is not Realtime. Windows can still process KB / mouse in high priority.
Only in Realtime a process cannot be interrupted.

[Alexium]

Also note that it's not a CPU bottleneck (see the screenshots)

The screenshots are inconclusive. There can be 1 thread that fails to do its work in real time pace. 1 thread would utilize single-core CPU at 100%, dual-core at 50% and quad-core at 25%. All we can see is 50% quad-core CPU utilization, but we can't get per thread utilization breakdown. What I'm saying is there might well be one thread out of many that's bottlenecking, and we wouldn't see it.
Anyway, synchronization issue is also CPU-related issue, and I don't see why would CPU load profile differ so drastically depending on what you're looking at. All the sector around you still has to be processed, I think. Hence my conclusion - it's not related to general sector processing logic, but is related to rendering specifically.

[Astyrrean]

Also note that it's not a CPU bottleneck (see the screenshots)

The screenshots are inconclusive. There can be 1 thread that fails to do its work in real time pace. 1 thread would utilize single-core CPU at 100%, dual-core at 50% and quad-core at 25%. All we can see is 50% quad-core CPU utilization, but we can't get per thread utilization breakdown. What I'm saying is there might well be one thread out of many that's bottlenecking, and we wouldn't see it.
Anyway, synchronization issue is also CPU-related issue, and I don't see why would CPU load profile differ so drastically depending on what you're looking at. All the sector around you still has to be processed, I think. Hence my conclusion - it's not related to general sector processing logic, but is related to rendering specifically.

Look at the screenshots again... I am monitoring individual cores. That's why there are 4 values. Not a single one of them is maxed.

[Alexium]

Also note that it's not a CPU bottleneck (see the screenshots)

The screenshots are inconclusive. There can be 1 thread that fails to do its work in real time pace. 1 thread would utilize single-core CPU at 100%, dual-core at 50% and quad-core at 25%. All we can see is 50% quad-core CPU utilization, but we can't get per thread utilization breakdown. What I'm saying is there might well be one thread out of many that's bottlenecking, and we wouldn't see it.
Anyway, synchronization issue is also CPU-related issue, and I don't see why would CPU load profile differ so drastically depending on what you're looking at. All the sector around you still has to be processed, I think. Hence my conclusion - it's not related to general sector processing logic, but is related to rendering specifically.

[Astyrrean]

Also note that it's not a CPU bottleneck (see the screenshots)

The screenshots are inconclusive. There can be 1 thread that fails to do its work in real time pace. 1 thread would utilize single-core CPU at 100%, dual-core at 50% and quad-core at 25%. All we can see is 50% quad-core CPU utilization, but we can't get per thread utilization breakdown. What I'm saying is there might well be one thread out of many that's bottlenecking, and we wouldn't see it.
Anyway, synchronization issue is also CPU-related issue, and I don't see why would CPU load profile differ so drastically depending on what you're looking at. All the sector around you still has to be processed, I think. Hence my conclusion - it's not related to general sector processing logic, but is related to rendering specifically.

Look at the screenshots again... I am monitoring individual cores. That's why there are 4 values. Not a single one of them is maxed.

And since I'm replying to a double post, I'll add something. CPU and GPU utilization does not go UP based on high busy the scene is. It actually goes DOWN (at least GPU utilization).

PS
Re: "synchronization issue is also CPU-related issue". It is not actually. It is purely a code-related issue. See a description from IBM:
http://publib.boulder.ibm.com/infocente ... hreads.htm

[Brachra]

I know a lot of you are pointing towards bottleneck as an issue here.
But its gotta be a bottleneck caused by the software.

otherwise?
like 80% of all PC owners who have this game .. Even if its top-end beast machine? Are bottlenecking..

theres just no way THAT many people's hardware isn't sufficient.

[Alexium]

Have you heard of thread scheduling? In short: OS is not obliged to run every thread on its own specific core. Once a thread's time quantum expires, it's put into suspended state and unloaded. The next time it will be rescheduled to execute on any core that's available at the moment. Which means that if you have a single-thread application that's working 100% of the time (for example, it has an infinite loop), you won't normally see 1 core loaded to 100% and other 3 with 0% load. You will see even 25% load across all cores. Now, an operating system does proved a way for a developer to "pin" a specific thread to a specific core (that's called thread affinity), but even that is more of a hint to scheduler than a strict rule it must follow no matter what. And again, in most cases there's no reason to set thread affinity, that's rarely done.

But its gotta be a bottleneck caused by the software.

Of course it's the software! The term "bottlenceck" doesn't neccessarily refer to hardware. Or, it can be a hardware bottleneck caused by poor software design (the game starving on hardware resource that it shouldn't be using much in the first place).

[spool1]

This is because that new sector basically has no ships flying around, where the tutorial sector has thousands of ships flying around...

[devastat_]

I am experiencing similar results with my machine, but when I drop the resolution down I will start to get a more consistent and better performance overall, no matter where I am. I guess it means that in my case my GPU is the bottleneck? (altho it is never fully utilized either) Radeon HD 6970.

[Astyrrean]

I know a lot of you are pointing towards bottleneck as an issue here.
But its gotta be a bottleneck caused by the software.

otherwise?
like 80% of all PC owners who have this game .. Even if its top-end beast machine? Are bottlenecking..

theres just no way THAT many people's hardware isn't sufficient.

In any given piece of software, by definition, there is always a bottleneck. The question is where the bottleneck lies (i.e. if it's beyond 200 FPS, it likely doesn't matter as the target for most systems is a sustained 60 FPS).

The question being investigated in this thread is: Given that both GPU and (individual cores') CPU utilization is well below 100%, and that GPU utilization actually goes down, could it possibly be a thread synchronization bottleneck?

[Pimpace]

...

My hypothesis is that it is a thread synchronization issue between the (very numerous) ai-controlled ships in Albion and the other threads. That would explain low utilization and low FPS seen...

...

Best,

Astyrrean

Dude, you can easily find this: open X Rebirth with windowed mode, download Process Explorer then put windows on your monitor to see well, right click xrebirth.exe in Process Explorer, then properties, now click on "Threads" tab on Process Explorer. Now you can measure thread loads of all cores of your CPU, so you can see if any of the thread is overloaded. (maybe stuck at 100%) The name of the thread and the hexadecimal number which is the call point could help devs to reveal this issue, if yout theory is right.

Something like this: screenshot
(I'm on a dual monitor, but you can do this easily with one single monitor too)

However, most functions calls point to an Windows external dll, so in this case pointless its name, but if devs get your thread in this forum, maybe we could ask them that programatically name their threads, (I don't know their programming habit, most of the time programmers don't declare a name for working threads in their programs, except debugging purposes) so all users can measure and check your theory and we'll see what thread causes the problem, so they can investigate it.

Hope this helps.

ps.: for taking screenshot in Windows, no need any external program. Just hit ctrl-PrintScrn button. (not just PrintScrn) Then you can paste this SS to any paint program to crop/cut/anything, then save.

[Alexium]

That will help find an overloaded thread, but not synch issues.

Anyway, this is something devs should do. They have their source code, code knowledge and, hopefully, runtime performance analysis tools to find bottlenecks or thread synchronization stalls. The best thing we can do is to help devs reproduce the issue.

[onedreamer]

I run it with a GTX670 and I have had no problem anywhere.

[Astyrrean]

That will help find an overloaded thread, but not synch issues.

Anyway, this is something devs should do. They have their source code, code knowledge and, hopefully, runtime performance analysis tools to find bottlenecks or thread synchronization stalls. The best thing we can do is to help devs reproduce the issue.

Alexium is correct

[johncage]

i would say it's got to do with the ship pathfinding. maybe they need to devote more threads to it?

[Astyrrean]

Looking at Bernd Lehahn's post on Steam (here: http://steamcommunity.com/app/2870/disc ... 371458323/)

"[...] two cores can crunch on our two main threads, one core can do the graphic driver code and the fourth one is free for the lower priority threads of our game such as pathing calculations, loading and sound. More cores will unfortunately not help much at this point in time."

Thus it looks there are at least 6 threads requiring synchronization:

• * "Main Thread 1"
  * "Main Thread 2"
  * "Graphic Driver Code Thread"
  * "Path Finding Thread"
  * "Sound Thread"
  * "Loading Thread"
  * (likely more "service-like" threads unlisted)

@Devs: Often times it is possible to replace mutex objects with semaphores, or even remove synchronization completely (which by experience is typically overly done out of caution in initial coding - which is a best practice, but should be optimized in later code review).

PS
Have you set relative thread priority in code?

Cheers,

Astyrrean

[Scoob]

Does anyone else think these issues are similar to those Skyrim suffered at launch? There were numerous areas that'd see a huge FPS drop, yet CPU and GPU utilisation would also drop. They fixed the initial compiler issue - well, programmer issue as the settings were wrong - but even now you still see the issue, though much reduced, in several areas.

For me, for the most part, I see 40-60% CPU utilisation on each of my four cores, and my GPU is regularly at 95%+ However, when I get the HUGE slowdowns it's almost always showing much LOWER resource usage. Incidentally, for me, it's always the GPU load that drops the most. Often a save then reload will recover some FPS, so possibly a leak somewhere...

FYI My System Specs:

2500k @ 4.8
2x GTX 680 @ 1200mhz - though only one is use of course.
Water cooled, so no temp issues
W7 Ult 64 all up to date
Latest WHQL NV drivers
Latest Realtech Sound Drivers

System is on a fast SSD, though my Steam folder resides on an old 1tb spinner.

Scoob.