A debate on VR issues/characteristics in games such as X4 (Split from an X4 poll)

Anything not relating to the X-Universe games (general tech talk, other games...) belongs here. Please read the rules before posting.

Moderator: Moderators for English X Forum

vvvvvvvv
Posts: 1235
Joined: Tue, 28. Nov 23, 15:38
x4

A debate on VR issues/characteristics in games such as X4 (Split from an X4 poll)

Post by vvvvvvvv »

Imperial Good wrote: Sat, 28. Sep 24, 17:09 My main concern with making X4 VR compatible is that it is very prone to low and inconsistent frame rates due to the nature of the complex sandbox simulation. Generally these properties are not well suited for a VR experience where high, stable FPS is required to avoid motion sickness. I know there are fame interpolation techniques that are used in VR to help reduce motion sickness, especially with respect to user movement, but these may not be fully compatible or useable with the way X4 does its graphics.
Virtual Desktop and systems which allow you to turn off reprojection work well with low framerates. That's how people play X-Plane (whose normal framerate is 40) and No Man's Sky (which can dip below 20) in VR. So you don't need max framerate and for some reason this is not a common knowledge.

Basically you need a system which allow you to turn reprojection and frame interpoluation OFF, while still displaying last frame in correct direction. The only effect of low framerate would be runaway black borders at the edge of your view when you turn head quickly.

Again, this is one of the things that changed. Right now headsets are semi-autonomous, meaning if PC fails to feed a frame, headset still displays the last one and still tracks head rotation. Things were different in Oculus DK2/Rift time when headset was a display, frames were displayed by PC, which is why it needed to run at max framerate at all times.
Imperial Good
Moderator (English)
Moderator (English)
Posts: 4933
Joined: Fri, 21. Dec 18, 18:23
x4

Re: POLL: Should Egosoft make X4 VR compatible?

Post by Imperial Good »

vvvvvvvv wrote: Sat, 28. Sep 24, 18:10 Basically you need a system which allow you to turn reprojection and frame interpoluation OFF, while still displaying last frame in correct direction. The only effect of low framerate would be runaway black borders at the edge of your view when you turn head quickly.
It is my understanding that this is exactly what causes motion sickness since without reprojection or other interpolation techniques it messes with a persons head to not see projection change quickly when they move their head around, even with small motions.

Also runaway black borders implies asynchronous reprojection of some sorts, so maybe you meant to turn it off at an application level? Generally an application could do reprojection better since it can have more knowledge of the scene depths and other effects. However a driver implementation might be able to perform reprojection with lower latency and latency is a big part of what causes motion sickness.

It might also be the case that the motion sickness people describe are with immersive VR experiences, such as standing up and playing a physically exerting game like beat saber. Possibly motion sickness is less of an issue when using the VR headset to view a game while sitting.
vvvvvvvv
Posts: 1235
Joined: Tue, 28. Nov 23, 15:38
x4

Re: POLL: Should Egosoft make X4 VR compatible?

Post by vvvvvvvv »

Imperial Good wrote: Sun, 29. Sep 24, 05:24
vvvvvvvv wrote: Sat, 28. Sep 24, 18:10 Basically you need a system which allow you to turn reprojection and frame interpoluation OFF, while still displaying last frame in correct direction. The only effect of low framerate would be runaway black borders at the edge of your view when you turn head quickly.
It is my understanding that this is exactly what causes motion sickness since without reprojection or other interpolation techniques it messes with a persons head to not see projection change quickly when they move their head around, even with small motions.
That does not match my experience of using VR headset.

Also, I think I need to explain it.

While the game produces sub-maximum framerates, the headset paints the last frame it received and it remains in direction it was received for. Headset continues to track your head rotation. Meaning if you look forward, and game freezes, and you turn left/right, the image won't stuck to your head, but will float where you were looking. It is like a VR movie theater, 180 video with black borders. People do not puke their guts out in VR movies, although they run at suboptimal framerates and do not respond to head position. Same deal. Works this way on virtual desktop systems and on headsets with Android which are Oculus and Pico.

As I said, original headsets likely could not do it, that's why you needed max framerate.
Alan Phipps
Moderator (English)
Moderator (English)
Posts: 31735
Joined: Fri, 16. Apr 04, 19:21
x4

Re: POLL: Should Egosoft make X4 VR compatible?

Post by Alan Phipps »

I'm tempted to move this to Off Topic as it is becoming a very technical or hardware discussion and has very little to do with X4 since the devs said that it won't be happening at their end. Also the poll results so far strongly support their decision.
A dog has a master; a cat has domestic staff.
Imperial Good
Moderator (English)
Moderator (English)
Posts: 4933
Joined: Fri, 21. Dec 18, 18:23
x4

Re: POLL: Should Egosoft make X4 VR compatible?

Post by Imperial Good »

vvvvvvvv wrote: Sun, 29. Sep 24, 06:34 Meaning if you look forward, and game freezes, and you turn left/right, the image won't stuck to your head, but will float where you were looking.
It is my understanding that this is what asynchronous reprojection does. To quote google...
Asynchronous reprojection is a graphics technology that ensures high performance VR rendering, even during times of high system load. It provides consistent and judder-free motion when the user rotates their head.

The 'reprojection' part of asynchronous reprojection works by adjusting the position of the rendered frame just before it is seen by the user to account for rotational user head movement while the frame is being rendered. The 'asynchronous' part means that the technology will always use the most recent rendered frame, ensuring that even if the app drops a frame, the user-visible frame rate does not drop.
Also from Google's guidelines...
In 6DoF, apps should target 75 FPS. Drops below 60 FPS will have noticeable artifacts and cause user discomfort.
In 3DoF, apps should target 60 FPS. Drops below 45 FPS will have noticeable artifacts and cause user discomfort.
X4 would mostly class as a 3DoF application, as the player is not expected to be moving their head significant distances. As such the recommendation would be 45 FPS which is something X4 cannot guarantee in some demanding situations and so would be prone to causing user discomfort.
vvvvvvvv
Posts: 1235
Joined: Tue, 28. Nov 23, 15:38
x4

Re: POLL: Should Egosoft make X4 VR compatible?

Post by vvvvvvvv »

Imperial Good wrote: Sun, 29. Sep 24, 16:09 It is my understanding that this is what asynchronous reprojection does. To quote google...
That is not what asynchronyous reprojection is doing. Reprojection attempts to reconstruct depths, that causes flickering and artefacts. That is not necessary. Also, I'd recommend to get personal experience with headset instead of quoting google.
Imperial Good wrote: Sun, 29. Sep 24, 16:09
In 6DoF, apps should target 75 FPS. Drops below 60 FPS will have noticeable artifacts and cause user discomfort.
In 3DoF, apps should target 60 FPS. Drops below 45 FPS will have noticeable artifacts and cause user discomfort.
Once again, I suggest to get personal experience instead of quoting external sources. Reprojection will produce artefacts when framerate drops, which is why people turn it off. Virtual desktop does not reproject the scene, but treats it as a movie frame, with separate picture for each eye. That is sufficient, does not produce artifact, and it is possible to comfortably play it at very low framerate. There is lag, of course, and runaway black borders, but it does not trigger VR sickness.
Imperial Good
Moderator (English)
Moderator (English)
Posts: 4933
Joined: Fri, 21. Dec 18, 18:23
x4

Re: POLL: Should Egosoft make X4 VR compatible?

Post by Imperial Good »

vvvvvvvv wrote: Sun, 29. Sep 24, 20:14 runaway black borders
This implies reprojection since the driver or VR headset is rendering new frames by transforming the frame buffer image?

To quote Wikipedia...
Asynchronous reprojection is a class of computer graphics technologies aimed ensuring a virtual reality headset's responsiveness to user motion even when the GPU isn't able to keep up with the headset's target frame rate.[1] Reprojection involves the headset's driver taking one or multiple previously rendered frames and using newer motion information from the headset's sensors to extrapolate (often referred to as "reprojecting" or "warping") the previous frame into a prediction of what a normally rendered frame would look like.[2]
Oculus has 2 differing implementations of Asynchronous reprojection...
Asynchronous timewarp uses the headset's rotational data to extrapolate a new rendered frame based on the last frame it received. Asynchronous spacewarp additionally uses depth information to help compensate for perspective and other geometric changes.[5][6][7]
It sounds like you are talking about having asynchronous timewarp enabled and asynchronous spacewarp disabled.

You may also be referring to changing a setting within asynchronous reprojection. Such as disabling edge extrapolation/interpolation and instead using a plain black boarder.

If it is not asynchronous reprojection, then what is the name of the technology being used to generate new output images for each eye asynchronously from the game framebuffer that factor in head rotation but instead perform the transformation with black boarders?
vvvvvvvv
Posts: 1235
Joined: Tue, 28. Nov 23, 15:38
x4

Re: POLL: Should Egosoft make X4 VR compatible?

Post by vvvvvvvv »

Imperial Good wrote: Mon, 30. Sep 24, 00:33 This implies reprojection since the driver or VR headset is rendering new frames by transforming the frame buffer image?
No.

I managed to summarize it to comprehensible state:

* The VR headset itself maintains a high refresh rate for head tracking.
* The game's frames are displayed as a "frozen" image in 3D space, updating at whatever rate the game can manage. An "image sphere", like a movie theater.
* When you turn your head faster than the game can update, you see black borders instead of distorted imagery.
* This approach avoids the artifacts and potential discomfort caused by reprojection techniques.

----

Basically, imagine a 180 video. You can turn head to the side, you'll see black borders. You can look back, there will be nothing. If the video is paused, that does not cause discomfort, as you can still turn head freely. Virtual Desktop is like this. An image sphere with separate image for each eye, which is projected at the location you last looked. When the video stops, image remains at last projected spot, and you can still turn head around.

So, there's no extra frames created, but rotational tracking happens at max framerate. Meaning you do not get VR sick, but there's no reprojection.

Reprojection usually attempt sto create "intermediate" images for missing frames, which causes artefacts, which is why people turn it off. Artefacts look like this.
Image

It works because headset is repainting the "image sphere" and not the PC. Early models of headsets had no onboard chips and couldn't do this. That's why max framerate was necessary.

----

Got split off into a separate topic, eh? Well, either way I can answer questions, as I've been using the headset for some time.
Imperial Good
Moderator (English)
Moderator (English)
Posts: 4933
Joined: Fri, 21. Dec 18, 18:23
x4

Re: A debate on VR issues/characteristics in games such as X4 (Split from an X4 poll)

Post by Imperial Good »

vvvvvvvv wrote: Mon, 30. Sep 24, 01:10 * The VR headset itself maintains a high refresh rate for head tracking.
* The game's frames are displayed as a "frozen" image in 3D space, updating at whatever rate the game can manage. An "image sphere", like a movie theater.
* When you turn your head faster than the game can update, you see black borders instead of distorted imagery.
* This approach avoids the artifacts and potential discomfort caused by reprojection techniques.
This technique classes as asynchronous reprojection though? Since it is asynchronously creating new frames by reprojecting the static image onto the inside of a spheroid and rotating that sphere to compensate for head rotation.

If it is not asynchronous reprojection, what software technique is it?
vvvvvvvv wrote: Mon, 30. Sep 24, 01:10 Reprojection usually attempt sto create "intermediate" images for missing frames, which causes artefacts, which is why people turn it off. Artefacts look like this.
It can cause those, but does not have to.

Return to “Off Topic English”