Now getting >90 FPS at 1080p - info on issue affecting high-end rigs [spoilers]

[mcdjfp]

I have run into problems before with a game it had 4 GB of memory allocated to it. It was on my previous system which had 8 GB total. I decided to let the game claim 4 GB to allow it to keep more stuff loaded. When I did this the performance plummeted (60+ fps to <15). Windows took a look at that allocation, the memory it was using, and the memory used by other processes and decided that there was no where near enough free space in memory for new programs to load into, and it immediately took steps to solve the problem. Having Windows trying to mess with page files at the same time as the game was trying to pull stuff from the hard drive did not work well. I dropped the allocation to 3 GB (this was a 64-bit program) and the problem vanished.

As a result of this past experience, I consider 8 GB a bare minimum for any game which wishes to claim (just claim not necessarily use) 4 GB, with 12 - 16 needed if I want to turn up all of the settings.

I don't know what the problem is with Rebirth. To be honest, if I was not looking for information about the trading and story bugs I would not even know that there were frame rate issues. I have had no difficulty so far with the frame rate (it is not always in the 40 - 60+ range I would like, but it is high enough to be smooth and not affect gameplay) even when running at 1920x1080 in 3D. My current system has 32 GB of memory, one GTX 680, and a slightly overclocked (4Ghz) Extreme Edition i7-3960X.

By the way the subtitles are at the wrong depth in 3D. Everything else is fine but they seem to be set behind my eyes.

[curt428]

I have run into problems before with a game it had 4 GB of memory allocated to it. It was on my previous system which had 8 GB total. I decided to let the game claim 4 GB to allow it to keep more stuff loaded. When I did this the performance plummeted (60+ fps to <15). Windows took a look at that allocation, the memory it was using, and the memory used by other processes and decided that there was no where near enough free space in memory for new programs to load into, and it immediately took steps to solve the problem. Having Windows trying to mess with page files at the same time as the game was trying to pull stuff from the hard drive did not work well. I dropped the allocation to 3 GB (this was a 64-bit program) and the problem vanished.

As a result of this past experience, I consider 8 GB a bare minimum for any game which wishes to claim (just claim not necessarily use) 4 GB, with 12 - 16 needed if I want to turn up all of the settings.

I don't know what the problem is with Rebirth. To be honest, if I was not looking for information about the trading and story bugs I would not even know that there were frame rate issues. I have had no difficulty so far with the frame rate (it is not always in the 40 - 60+ range I would like, but it is high enough to be smooth and not affect gameplay) even when running at 1920x1080 in 3D. My current system has 32 GB of memory, one GTX 680, and a slightly overclocked (4Ghz) Extreme Edition i7-3960X.

By the way the subtitles are at the wrong depth in 3D. Everything else is fine but they seem to be set behind my eyes.

You sure seem to be experiencing the fps issue, believe it or not. Not always in the 40-60 fps range? You get <40 fps with THAT rig?? With those specs, you should be 60fps+ constant. Not a hiccup.

I just wish I had those specs so I could ignore it too

[Astyrrean]

Ok. So I've collected a bunch of stack traces runtime of X-Rebirth's code, to gather additional data for analysis.

Seems like 95% of the stacks I pulled (for the two main threads that is) are busy with the following function:
XRebirth.exe!xmlSAX2SetDocumentLocator+0x19d04b [impossible to tell what it really is without access to your Symbols' path, but you would know on your end looking at similar stack traces - note I think these have nothing to do with the actual SAX2 library, though I may be wrong]

Now each and every one of those calls tries to acquire a lock via the subsequent default system call:
kernel32.dll!WaitForSingleObject+0x12

That COULD result in poor threading performance. I hope this helps.

Threads at runtime on my machine:
http://s10.postimg.org/tt68ggo61/X_Rebirth_Threads.png

Stack trace of two main threads:
http://s10.postimg.org/jgfcgn595/X_Rebi ... Trace1.png
http://s10.postimg.org/ceheug1nd/X_Rebi ... Trace2.png

Cheers,

Astyrrean

***
PS
Threads for:
Input (DINPUT8.dll+0x92a6)
Audio (XAudio2_7.dll!DllUnregisterServer+0x91db)
... have ~1% CPU utilization so they should be fine

The joint threads which I'm guessing account for the graphics driver code:
XRebirth.exe!??4_Init_locks@std@@QAEAAV01@ABV01@@Z+0x142c6
nvd3dum.dll!QueryOglResource+0x1d3870
... collectively account for <20% CPU utilization; assuming bottlenecked by the threads above?

[lithast]

Ok. So I've collected a bunch of stack traces runtime of X-Rebirth's code, to gather additional data for analysis.

Seems like 95% of the stacks I pulled (for the two main threads that is) are busy with the following function:
XRebirth.exe!xmlSAX2SetDocumentLocator+0x19d04b [impossible to tell what it really is without access to your Symbols' path, but you would know on your end looking at similar stack traces - note I think these have nothing to do with the actual SAX2 library, though I may be wrong]

Now each and every one of those calls tries to acquire a lock via the subsequent default system call:
kernel32.dll!WaitForSingleObject+0x12

That COULD result in poor threading performance. I hope this helps.

Threads at runtime on my machine:
http://s10.postimg.org/tt68ggo61/X_Rebirth_Threads.png

Stack trace of two main threads:
http://s10.postimg.org/jgfcgn595/X_Rebi ... Trace1.png
http://s10.postimg.org/ceheug1nd/X_Rebi ... Trace2.png

Cheers,

Astyrrean

***
PS
Threads for:
Input (DINPUT8.dll+0x92a6)
Audio (XAudio2_7.dll!DllUnregisterServer+0x91db)
... have ~1% CPU utilization so they should be fine

The joint threads which I'm guessing account for the graphics driver code:
XRebirth.exe!??4_Init_locks@std@@QAEAAV01@ABV01@@Z+0x142c6
nvd3dum.dll!QueryOglResource+0x1d3870
... collectively account for <20% CPU utilization; assuming bottlenecked by the threads above?

Ok, now that does look suspicious.

[lithast]

Disclaimer: Unmanaged code is not really my main area of expertise, so take the following with a grain of salt.

I got curious and decided to dig a little deeper. I think the mutex indicated by the wait call is probably a red herring most of the time since it seems like Rebirth has a number of worker threads that are probably just blocked and waiting for work to do. Most other applications running have a bunch of similar calls.

http://s24.postimg.org/9844yvhs5/Rebirth_1.png

The first time I captured data from every thread in the application, and it indicates they are spending a lot of time in the mutex. However as per above I have seen this before with worker threads which are waiting for work (kinda like a blocking queue kind of thing if you know what I mean).

http://s24.postimg.org/7uci3kiit/Rebirth_2.png

This second dump is just from the main thread (which was spinning at 100% on one of my cores I might add). The call for xmlCleanupThreads looks very suspicious to me, obviously I have no idea what it actually does, but...

I guess I'm left wondering what on earth it is doing with xml that means the most expensive 3 things on all the threads in their application are xml related calls?

[lithast]

Ok, more stuff here. The xmlCleanupThreads call is not their code, it looks like an open source library for xml parsing called libxml2.

http://www.xmlsoft.org/

The API documentation for the call in question is available here:
http://stuff.mit.edu/afs/sipb/project/p ... reads.html

It does look like it uses mutex internally...

Now I'm really curious what they are doing with XML that is taking 12 times as long as the D3D calls and requires multiple threads..

The second most expensive call is related to loading XML into a DOM and manipulating it. What on earth would be continuously creating and cleaning up XML parsers and DOM's?

http://www.xmlsoft.org/html/libxml-SAX2 ... etPublicId

[bumpinthenight]

Disclaimer: Unmanaged code is not really my main area of expertise, so take the following with a grain of salt.

I got curious and decided to dig a little deeper. I think the mutex indicated by the wait call is probably a red herring most of the time since it seems like Rebirth has a number of worker threads that are probably just blocked and waiting for work to do. Most other applications running have a bunch of similar calls.

http://s24.postimg.org/9844yvhs5/Rebirth_1.png

The first time I captured data from every thread in the application, and it indicates they are spending a lot of time in the mutex. However as per above I have seen this before with worker threads which are waiting for work (kinda like a blocking queue kind of thing if you know what I mean).

http://s24.postimg.org/7uci3kiit/Rebirth_2.png

This second dump is just from the main thread (which was spinning at 100% on one of my cores I might add). The call for xmlCleanupThreads looks very suspicious to me, obviously I have no idea what it actually does, but...

I guess I'm left wondering what on earth it is doing with xml that means the most expensive 3 things on all the threads in their application are xml related calls?

Looks like you gave the game about 60 seconds of runtime to gather these stats, how much of that was spend loading towards a save game? I imagine a lot of it, maybe even 25 seconds worth? Just my initial impression any ways.

[mattek1979]

Yes it does. Could this be related to some people having performance improvement after deleting massive UIData.xml files. If there's a wait somewhere for xml parsing...

[lithast]

No, I attached the profiler after the game had loaded. It has been loaded for 30 minutes now and I get the exact same profile.

[bumpinthenight]

So far it looks like all of the game's initialization stems from reading and reacting to the contents of XML files, that function that's eating a lot of time could be the one that's triggering all of the loading so its not just the XML reading but the sub functions responding to that XML to create the runtime environment.

No, I attached the profiler after the game had loaded. It has been loaded for 30 minutes now and I get the exact same profile.

Ahh, disregard my above then.

[lithast]

I ran it again for a minute, jumping into and out of highways, the game has been loaded from the save file for 30 minutes at this point.

Same xml calls at the top..

Edit: Left it running for about 5 minutes with the ship parked near a station with no input from me at all. Still get the same profile. I was wondering if it was happening because it was loading game data from xml as I moved around, but even staying still it is still showing the same profile:

http://s15.postimg.org/r1asgt17v/Rebirth_3.jpg

[mattek1979]

I'm not at the game at the moment. But something like filemon could monitor file access during game time

[bumpinthenight]

I ran it again for a minute, jumping into and out of highways, the game has been loaded from the save file for 30 minutes at this point.

Same xml calls at the top..

Edit: Left it running for about 5 minutes with the ship parked near a station with no input from me at all. Still get the same profile. I was wondering if it was happening because it was loading game data from xml as I moved around, but even staying still it is still showing the same profile:

http://s15.postimg.org/r1asgt17v/Rebirth_3.jpg

I was just reading up on what sleepy is and how it works: It works by pausing the target thread and polling what its current instruction is. This would probably explain why its constantly spotting blocking and/or idling processes and tallying their time up. Meanwhile that third shot you just posted looks like further evidence this game spends more time waiting for unblocks then it does processing. IE badly synced threading.

[lithast]

I'm not at the game at the moment. But something like filemon could monitor file access during game time

Not seeing much disk IO or any files open that catch my eye. Looks like whatever it is doing is happening in it's memory address space.

[lithast]

I ran it again for a minute, jumping into and out of highways, the game has been loaded from the save file for 30 minutes at this point.

Same xml calls at the top..

Edit: Left it running for about 5 minutes with the ship parked near a station with no input from me at all. Still get the same profile. I was wondering if it was happening because it was loading game data from xml as I moved around, but even staying still it is still showing the same profile:

http://s15.postimg.org/r1asgt17v/Rebirth_3.jpg

I was just reading up on what sleepy is and how it works: It works by pausing the target thread and polling what its current instruction is. This would probably explain why its constantly spotting blocking and/or idling processes and tallying their time up. Meanwhile that third shot you just posted looks like further evidence this game spends more time waiting for unblocks then it does processing. IE badly synced threading.

Yep, this is monitoring elapsed time, not CPU load or cycles consumed etc so you would expect a fair amount of time in the mutex for a threaded app.

I.e quite often you might have some worker threads that block on a mutex waiting for an event to fire to release them when some work becomes available and that shows up here as a wait call.

The bit that makes no sense without further info is why around half of these waits seem to be related in some way to XML manipulation.. I'm half temped to dump its address space and have a poke around, but I'm not sure if I'm really that motivated.

Anyway this info might be nothing or it might be something, hope it is of use to somebody.

[mattek1979]

could be internal data handling through xml, maybe messages passed between methods. scary thought.

[bumpinthenight]

could be internal data handling through xml, maybe messages passed between methods. scary thought.

Hehehe indeed, I'm hoping its less sinister like threads meant for loading periods that are left idling rather then killed off and respawned when required.

[Astyrrean]

I think we're getting to something...

[mcdjfp]

You sure seem to be experiencing the fps issue, believe it or not. Not always in the 40-60 fps range? You get <40 fps with THAT rig?? With those specs, you should be 60fps+ constant. Not a hiccup.

I just wish I had those specs so I could ignore it too

Three things here.
1. I did not want to claim a higher frame rate than I was sure of. What I know for certain is that it is high enough to look smooth.
2. I am including the times I have had 3D on, which would make things worse from a GPU standpoint.
3. I have only had one hiccup the entire time. Just after one of the patches there was a slowdown just after load. Once it finished whatever it was doing in the background, it was back up to full speed.

My main point, in addition to trying to provide more information about what systems are having which problems, was that this game appears to make extensive use of the hard drive as it pulls whatever data it needs without mask of a loading screen. Past experience playing other games has shown that OS access to hard drives in these conditions causes frame rates to crash.

[Il Khan]

Every reference in procmon that I see to a xml file has the result "PATH NOT FOUND", sometimes i see 50-100 such entries in a row until I get a "BUFFER OVERFLOW", which tere are a lot of IMO. I think it doesn't look quite right but I'm not really qualified to say. Process I/0 seem to be at 100 when experiencing lag.

*Edit* I just overclocked my i7 920 CPU from 2,67Ghz to 3,33Ghz and the game feels a lot smoother and better playable. I no longer seem to drop down to 1 FPS but instead to 5 FPS in those situations. In crowded places the framerate is about 5 FPS better now aswell, 15 FPS in Watergate.