Headtracking / Tobii Eyetracker 5 Improvements for X4 4.0

[bahamada]

Star Citizen recently released support for Tobii Eyetracking: https://gaming.tobii.com/games/star-citizen/

The default settings there work so well that I didn't have to tweak anything to just have an awesome experience. Rotation and Postional movements feel really smooth and awesome with the Tobii Eyetracker 5.

In X4 however its a different story: It works well that the postional and rotational inputs are accurate but there is a big problem. There is no autocentering support so you are constantly hammering the recenter button and the directional pointers of your ship are like 30 cm in front of you which leads to a feeling that you have to constantly looking through a VR-Weapon-Scope which is very disorienting and make one dizzy after a while. Also there are too many dots flying around at your hud which don't align. What really could help is to move the dots like 10 meter in front of your virtual hud so you don't have that much movement on head movement and you could still tell where your ship is heading even if you move your head 50cm to the sides IRL.

So maybe the X4 devs could look at the Star Citizen integration and include some options and tweaks for 4.0 as the integration in SC shows really well that it can work like a charm if configured right. Would be a dream if X4 could have and integration at the same quality level.

[/root]

Have you found any solution? I just got the Tobii ET5 and the experience in X4 is... well... without any exaggeration whatsoever... it's utterly unplayable.

My "in game head" is positioned 30cm to the left so I can see only parts of the hud at the right side of the screen while the "VR reset" (ger: VR-Ansicht zurücksetzen) and "VR on/off" (ger: VR-Tracking an/aus) mappings don't work at all (nothing happens), so I'm stuck with that wrong position. Is there another "recenter button"?


Sadly even if I work around that by sitting ~30cm to the right and holding my head in a really awkward angle, the virtual joystick (cursor) doesn't align with the center target reticule and it's next to impossible to just fly straight ahead, since there's no indication of the "center of screen" where I have to put my cursor in order to not pitch/yaw.

I can switch to head tracking only and it seems to "recenter" my position when I change these modes (buttons still not working tho), but even when it's "somewhat fixed" turning my head to the right makes the tracker register me leaning forward and I get so close to the HUD that yet again I can't see it properly. Plus even when holding my head still and not moving my eyes, my in-game view is still bobbing and moving around. Maybe my unit is broken or something?

This is by far the worst eye or head tracking experience I ever had. Well, actually not, it's not a tracking experience at all - it's just plain broken - fubar - and playing with the "native tobii controls" in game doesn't seem to help either


(edit 2 of 2: I've added a post scriptum while a reply has been written - I'll remove it and post as a reply instead)

[bahamada]

nope, but I think I just disabled translation axis so it only accounts for rotational inputs.

[/root]

I see.. that really is kinda disappointing

Btw: The "deadzone" seems to act strange too. When the deadzone is set to (just making up numbers) 5% I'd expect it to register 0-5% movement as 0 and then start at 6% being like 1,052631579% (that's 1% divided by 0.95 (to factor in the reduced total travel distance of 1 minus 0.05 deadzone)). Instead it appears to do nothing from 0 to 5% and then directly jumping to 6% as if there was no reduced travel due to the set deadzone and the stuff inside the deadzone just ain't accessible anymore at all. However, this seems to be more prominent looking up than looking down..? I'm really not sure if my unit is broken or what's up with this. I already reduced other light sources in my room and even covered the window behind me etc. I never had issues like this using my TrackIR. So is my new Tobii tracker that bad or just broken or is X4's implementation massively bugged/flawed? Guess I'll give SC a shot...

Given that Tobii fails to track my eyes properly on the up/down axis (probably due to me wearing aspherical glasses) I somewhat tend to call BS on Tobii's calibration and calculations *sigh*

[rboerdijk]

You can manually recenter with scroll-lock. This should work normally.

Only case I'm aware of where it incorrectly reacted to the head position was with someone having a second tracker on another machine in the same room, which was causing weird interference.

The deadzone indeed will ignore the first x%, and when you leave the deadzone, jump (or actually interpolate) to the position where your head actually is.
This is actually the first time I hear someone comment on this. You could try disabling the deadzone entirely (or set it to a smaller value) if it doesn't feel good for you.

[kmunoz]

I've commented on this elsewhere. The way the game is designed, the HUD targeting reticle is not an overlay on the screen, it's actually a physical entity within the 3D space of the cockpit. (The same is true of the map: it's a physical object, a flat plane right in front of your face.) Therefore, when you move your position while head tracking, which happens *constantly* unless you either turn off translational tracking or set the dead zone really high, your relationship to the location of the HUD changes.

The problem isn't so much that the HUD is a physical object, it's that it's not at the same distance between you and the cockpit window. The window frame elements are farther away than the HUD, which means that the parallax causes problems when trying to interpret the relationship between your position, the HUD and the target. Ideally, the HUD would be on the cockpit window, so that the window frame moves at exactly the same rate as the HUD. (It would *not* be ideal to make the HUD a non-physical entity, because you'd still have issues with the perception of parallax, they'd just be even weirder.)

On the other hand, I've never had an issue where the HUD is way off to the side. Sometimes it does drift, but that drift comes from the fact that my body has moved. I suspect the radical off-side HUD is happening because of incorrect calibration.

And I'm not using Tobii - I'm using OpenTrack (free) with a ~$5 app on my iPhone that uses the camera to track my head movement. If I can get the calibration to work right on something that costs less than a fancy cup of coffee, it's got to be possible to get the calibration right on a Tobii.

That said, turn off translation tracking. Keep rotation tracking. There is no serviceable purpose to having translation tracking in the cockpit.

[/root]

You can manually recenter with scroll-lock. This should work normally.

Only case I'm aware of where it incorrectly reacted to the head position was with someone having a second tracker on another machine in the same room, which was causing weird interference.

First: Thanks for the answer!

No I don't have a second tracker active and I'm the only one in this room.

For me the recenter key doesn't work with Tobii Eye Tracker 5. I tried scroll lock as well as mapping some other keys - but when I press them nothing happens. I used TrackIR in the past and with that the recenter key worked just fine, but with Tobii Eye Tracker 5 it just doesn't do anything. Just to clarify what I expect when pressing the recenter button: From TrackIR I'm used to the experience of pressing the button and my in-game pilot head "jumps back" to the default "zero" position that it has when not using a head tracking device. When I do the same when using the Tobii tracker nothing happens - head position just stays where it is.

Is there any lead I could investigate?

The deadzone indeed will ignore the first x%, and when you leave the deadzone, jump (or actually interpolate) to the position where your head actually is.
This is actually the first time I hear someone comment on this. You could try disabling the deadzone entirely (or set it to a smaller value) if it doesn't feel good for you.

Yeah, I disabled the deadzone.. but just think about the philosophy: a dead zone should give the option to "remove" a certain area from the input range. That area should not have an effect on the response curve anymore. In the past it's been used to counteract the unstable center position of a joystick that couldn't be zeroed by calibration. If you code it like X4 where it just jumps from 0.00% to deadzone.01% then you lose a lot of previously accessible area around zero. A deadzone should _move_ the 0 on the X axis of the response curve to the right, not just cut off the line so it starts somewhere in the middle of the diagram.

Seen from the other side: a deadzone can also be introduced to cut a bit off the max end, like deadzone from 95% to 100% which helps with devices that can only register like 98% max force. In that case you also don't want to "cap" the max possible input - you just want to limit the travel distance needed to reach max input.

In my humble opinion an axis with a min deadzone of ~10% and a max deadzone of ~5% should look like figure B, not like figure A. If in X4 it's coded like figure A I'd appreciate if you would kindly forward my suggestion to the devs and ask them to change it to figure B instead




By the way, if you actually are going to forward something, could you please also let them know that the key mappings display is bugged when mapping the keys F13 to F24? They're being mapped and registered, but the bound key shows up empty.

[rboerdijk]

For me the recenter key doesn't work with Tobii Eye Tracker 5. I tried scroll lock as well as mapping some other keys - but when I press them nothing happens. I used TrackIR in the past and with that the recenter key worked just fine, but with Tobii Eye Tracker 5 it just doesn't do anything. Just to clarify what I expect when pressing the recenter button: From TrackIR I'm used to the experience of pressing the button and my in-game pilot head "jumps back" to the default "zero" position that it has when not using a head tracking device. When I do the same when using the Tobii tracker nothing happens - head position just stays where it is.

Is there any lead I could investigate?

When you press scroll lock, to 're-center', then that position of your head is the new "zero" position. So you should position your head "comfortable in front of the monitor" and then reset pose. All future Tobii-head movement is now relative to this new center. As a (crazy) example, if you'd position your head far to the left and press scroll-lock, then you'd have a very small movement range to the left and a way larger than usual range for detecting head movement to the right. So you are saying it doesn't behave like this for you (any other Tobii users running into this)?

Generall Tobii specifies where the head is, and we have a few parameters to tweak that, but we are not doing our own spatial positioning solution. The ways we can tweak it is with the deadzone (in cm's if I recall correctly) and a few multiplies to exaggerate movement (so you get more extreme movement/rotation) - that's it. The deadzone specifies a limit of a few cm's around the "centered zero head position". The "outside limit" is limited by whatever the physical limit of the device is. That limit is assumed to be reasonable, Tobii is not a room-scale positioning solution after all.

Let's take a look at my paint-example, view is from the top, monitor in black, tobii sensor has some sensor limit (purple box), deadzone in yellow. Starting in the center of the deadzone, move your head long the blue line to the left. When you hit position 'A', the actual position is x=-5 cm... But we 'fix' it by saying this is zero (so no popping, exactly as you want). Now follow the movement according to the blue box, until you enter the deadzone again (while moving your head backwards into it)... Then the x position is still going to have a -5 offset (either that, or it pops the moment you enter the deadzone). Repeat that movement a few times, and eventually the game will have you at x offset -1 meter and beyond.

top-view.png

(11.24 KiB) Not downloaded yet

Maybe I'm misunderstanding what you want. Can any other Tobii user state if the requested behaviour is reasonably expected, or are there references of other games which behaves like requested?
Also, at best this would increase the precision over the trackign range slightly, but I don't see any advantage besides that (and at a first guess, the precision gain would likely be quite minimal, compared to the jitter which is smoothed away anyway).

[/root]

When you press scroll lock, to 're-center', then that position of your head is the new "zero" position. So you should position your head "comfortable in front of the monitor" and then reset pose. All future Tobii-head movement is now relative to this new center. As a (crazy) example, if you'd position your head far to the left and press scroll-lock, then you'd have a very small movement range to the left and a way larger than usual range for detecting head movement to the right. So you are saying it doesn't behave like this for you (any other Tobii users running into this)?

Yes, it didn't work when I posted this. And yes, your description of the feature matches my expectation (kudos for the paint job, lol ). But I'm happy to announce that I a) could make it work and b) probably found a bug.

Today I bound recenter to the U key, which works as expected. Binding tracking on/off to J works too. So all good in that department now


What didn't work before was binding to F6 and/or F7. Not sure if these are reserved or something. Binding to F6/F7 works in the settings menu and (in contrast to F13 to F24) also shows the binding - but in game these keys just don't do anything when pressed. Doing the same with U/J keys works without problem.

Keys F13 to F24 work too when assigned, but they don't show up as bound (no display of the text "F15" in the box) or anything else in the key settings menu - bindings to these keys are displayed as "unbound".


Conclusion:
Doesn't seem to be an issue with the Tobii Eye Tracker 5 (probably) but with X4 having issues with the function keys in general. Need to test other functions bound to F6/F7 to confirm if these keys are just dead

[Malakie]

I see.. that really is kinda disappointing

Btw: The "deadzone" seems to act strange too. When the deadzone is set to (just making up numbers) 5% I'd expect it to register 0-5% movement as 0 and then start at 6% being like 1,052631579% (that's 1% divided by 0.95 (to factor in the reduced total travel distance of 1 minus 0.05 deadzone)). Instead it appears to do nothing from 0 to 5% and then directly jumping to 6% as if there was no reduced travel due to the set deadzone and the stuff inside the deadzone just ain't accessible anymore at all. However, this seems to be more prominent looking up than looking down..? I'm really not sure if my unit is broken or what's up with this. I already reduced other light sources in my room and even covered the window behind me etc. I never had issues like this using my TrackIR. So is my new Tobii tracker that bad or just broken or is X4's implementation massively bugged/flawed? Guess I'll give SC a shot...

Given that Tobii fails to track my eyes properly on the up/down axis (probably due to me wearing aspherical glasses) I somewhat tend to call BS on Tobii's calibration and calculations *sigh*

There are two ways of implementing a deadzone but that is up to the X4 devs.. Many games use the deadzone where anything that moves inside that range, the screen is dead and does nothing, does not move at all. Many flight type games do the same thing with one caveat, once you hit the end of the deadzone range, the slowly start to move the view ramping up the speed of transition the more your look moves outside the core deadzone so you don't get that perceived "jolt" or pop of movement suddenly.

[/root]

@Malakie
Aside: you don't know any Greek perchance?

Yeah, it's up to them and I'm not even sure how exactly it's done atm - I just got the impression that it's a simple cut-off and not moving the curve because I got that awkward jump when going outside the deadzone. Then again I only experienced this when moving in a certain direction while having general issues with that recenter button not working. Generally I think the absolute non plus ultra is having the option to define a response curve for every axis as we can do with programs like Joystick Curves; but I realize that's quite a lot to ask from a game dev - could as well ask for the option to remap keys like control, shift or even make anything we like into a key combo (so we could use any key we'd like as modifier (not just changing the modifier key (actually being allowed to combine anything (Elite Dangerous does it (I think (parenthesis ftw!))))).

[rboerdijk]

I'd have to look in the code, but from memory it would just smooth over a few frames, so while it's not instant it would indeed be near-instant.
I'll think a bit more about the suggestions... that auto-center does sound nice also.

Note: Not promising anything

[Malakie]

@Malakie
Aside: you don't know any Greek perchance?

Yeah, it's up to them and I'm not even sure how exactly it's done atm - I just got the impression that it's a simple cut-off and not moving the curve because I got that awkward jump when going outside the deadzone. Then again I only experienced this when moving in a certain direction while having general issues with that recenter button not working. Generally I think the absolute non plus ultra is having the option to define a response curve for every axis as we can do with programs like Joystick Curves; but I realize that's quite a lot to ask from a game dev - could as well ask for the option to remap keys like control, shift or even make anything we like into a key combo (so we could use any key we'd like as modifier (not just changing the modifier key (actually being allowed to combine anything (Elite Dangerous does it (I think (parenthesis ftw!))))).

LOL no, my 'Name' is not my real name, it is from a name/callsign in the US Military actually. Pronouced, Mal uh k eye

I have talked to the main dev here at Egosoft quite a bit about the nuances used across many games and while they are swamped right now, they are definitely going to take a look at more function/capability etc as time allows. I will continue to offer feedback myself as I am well vested in the Tobii 5 thanks to Tobii making me an affiliate a few months ago and supporting my stream channel, albeit small and not really known. Personally I cannot play X4 (and Elite, Star Citizen, DCS and a few others) without now. I agree with you there is a real difference between most of those and the X4 implementation and I have been trying to find a way to describe exactly what the differences are to help Egosoft on this. I am now going to make a small video comparison for Egosoft to try and show them what I mean in terms of the difference etc. It could even be something more simple like speed and precision being extended more, especially as everyone has different hardware and something I have noticed is the game plays different and eye tracking 'feels' different on my three different machines from each other.. Honestly, it does not 'feel' as smooth on my full 3840x2160 UHD 4K system (RTX 3080 and i9-9900k) as it does my machines using i7 processors and older GTX 1080ti cards... Which is why I was thinking maybe it could be something more simple like more speed and smoothing and deadzone size options scaling.. I don't know though as I am not Egosoft and don't code for the Tobii myself, I am just guessing and making opinion based on my experience and observations of course.

[James Spartan]

I just discovered something tonight while tweeking my tobii settings. I was tweeking because I find the tracking is not as smooth as in other games. And I discovered it's the head tracking that is not smooth, the eye tracking is. If I only active the head tracking, movements are very stuttery !! What ever settings I'm using, it stutters, even with smoothing maxed out. While using only the eye tracking, it's always smooth !!
Have you noticed this as well ?

[Malakie]

I just discovered something tonight while tweeking my tobii settings. I was tweeking because I find the tracking is not as smooth as in other games. And I discovered it's the head tracking that is not smooth, the eye tracking is. If I only active the head tracking, movements are very stuttery !! What ever settings I'm using, it stutters, even with smoothing maxed out. While using only the eye tracking, it's always smooth !!
Have you noticed this as well ?

Sorry for delay.. real life with harvest season etc, got in the way.

And yes, I have also noticed this in the v5+ builds. Just have not had time to sit down and do more with troubleshooting. But you are right, eye tracking seems 'jittery' now in X4. Additionally, I really would like Egosoft to add a lot more options for control to the game for us. We need better precision and deadzone abilities and larger levels of all options instead of the super limits we currently have. ** If the dev is reading this, I can post a list of options some other games use to good effect that make QOL with the Tobii in game so nice!

I have also, like you, noticed that the Tobii in X4 is no longer as 'smooth' and precise as in every other game I use it in. I am not sure why but it is on both my machines I play on with X4. Like I mentioned, it seems more 'jittery' to me visually.