Serious Attempt to get into X4 as a Totally Blind Player

[CaplinCaplin]

Unfortunately having the game read out all the menu text isn't really an option. Those lines don't have, and indeed can't really have, voice recordings, and although text-to-speech is something we've experimented with, it isn't really a viable option for us at the moment, for a variety of reasons. We'd therefore need a clearer understanding of why your OCR wasn't able to provide what you need there, to try and work out whether there was anything we could do to enable it to work better for you.

Unfortunately the OCR setup I use is jury rigged; It wasn't really intended for this kind of thing. Consequently I don't know why it works the way it does. I have a couple different alternative systems which occasionally yield more understandable results, but it's all still a bit opaque.

A lot of the issue seems to relate to the presentation of text in columns. OCR tends to flatten things like price lists so it's sometime's quite difficult to tell what I'm paying for. It's less aggravating than it was in x3, where almost none of that info read at all.

If TTS is indeed not a viable option, mods of some kind to clean up the display would probably help. I don't know if the OCR gets distracted by the graphical ship displays and such, but anything that can be done to make the presentation boring and text-only would probably help. It might even make a fun challenge: how much can people do without access to the graphical map and the like?

Back when I was trying to play X3, I had help writing a mod which output trade prices, among other things, to debug logs where I could read them free of Ocr interference. In a worst case scenario something like that would also help, though obviously the specifics would be very different given that the x3 scripting language isn't being used any more.
I'm sure we can figure something out.

[oasis1701]

Hi there
I am also completely blind; however, I do not own the game yet.

There is a community of us out there playing the games that we are able to (there's very few of accessible games to be honest), but usually there's very few of us adventurous enough like @Caplin trying out games and figuring out if they could be playable to us or not.
I would be glad to get the game as well and help with the feedback and also play it myself and enjoy it if there's a possibility of us playing it and get more blind players to try it out as well.

Cheers

[CaplinCaplin]

Hi All,

I thought I'd post a bit about my efforts to use the game map, which is quite visual but behaves reasonably well, for the most part.

The map has a ton of information, some of which is probably presented in a format I can't easily detect. I appreciate that one of the things it offers is a sidebar listing (almost)all objects of interest depending on my current zoom level. This isn't perfect with OCR, but is a lot better than the actual graphics. I can generally route the mouse to ships and stations I want to interact with.

The problem comes when I want to move around at a larger scale, or get a sense of where things are relative to one another. This is particularly the case when moving between systems/sectors. The way I would do this in X3 would involve the universe map, which doesn't really exist in X4 as a separate entity, for understandable reasons.

A person on Reddit kindly pointed out that I can drop off nav beacons which I can then return to at will. They also make handy points of reference for centering the map on specific places, if I want to do that.

I appreciate the thematic element of dropping off a beacon I can then return to. I imagine I'll be using them more than most sighted players as I explore the universe.

The number one area of the map I would appreciate more info on is relationships between objects. A way to get distance and bearing info would be very useful, either from my current ship or between two arbitrary points. This info is obviously available to people visually but is a bit hard to deduce at the moment, though the ability to OCR the distance/speed gauges when flying around does help a bit.

kuertee had mentioned he hopes to work on a way to get distance/position info into a contact list of some kind. This will be very useful, it would be great if Egosoft could find a way to make the information available without need for a mod, but I appreciate any and all help.

I just thought this might be of interest to some people, as the map is still one of the components I struggle with, albeit it could be a lot worse.

[oasis1701]

Unfortunately having the game read out all the menu text isn't really an option. Those lines don't have, and indeed can't really have, voice recordings, and although text-to-speech is something we've experimented with, it isn't really a viable option for us at the moment, for a variety of reasons. We'd therefore need a clearer understanding of why your OCR wasn't able to provide what you need there, to try and work out whether there was anything we could do to enable it to work better for you.

Hello
I understand that adding speech tts might be not viable; however, do you think that it's possible that you could pipe the information from the game to our screen reading software? (for example, pipe the currently focused item/text to the screen reader)
This work has been done by a video game called skullgirls and i believe the library that they use is open for all to use. The library is called Tolk.
I thought that i ask, so you guys could take a look at it and see if it is viable for you to perhaps add such functionality. I know that a game like x4 can have many focusable items, compared to a fighting game. I have no knowledge of programming so i do not know if this would take a lot of time or not.

When we play the game, we have our screen reader running to perform OCR and also to alt+tab out of the game and browse the internet and such. Sending text from the game to the screen reader as the items in the game are focused with keyboard or controller, or focused with mouse could be very helpful to us, this way, there is no need to include a tts service in x4 as well.

edit: This appears to be tolk project on github: https://github.com/dkager/tolk
Cheers

[oasis1701]

I'm giving the game a try now!
One hard challenge i find is that when using the long range scanner, I think i am supposed to move towards the question marks that appear in the map, and detect the objects as my ship gets closer to them.
I do not know which direction to go though and i don't think i can ask the ship to auto pilot towards these question marks, can i?

[kuertee]

I appreciate the work of kuertee and anyone else willing to help out with this project.

Hey Caplin, here's the next version of the Accessibility Features mod: https://drive.google.com/file/d/1FERv9b ... sp=sharing
It's still in my Google Drive, but I'll upload it to Nexus once you get a good go at it.
You'll need to install my UI Extensions mod for its new custom menu to work: https://www.nexusmods.com/x4foundations/mods/552
Along with Sir Nuke's Mod Support API mod (https://www.nexusmods.com/x4foundations/mods/503) - which I know you've already installed.

The new feature is the Navigation Computer.
Here's a screenshot - in case your screen reader has OCR capabilities: https://i.imgur.com/aH5FeWB.jpg

It is accessible from the Interact Menu, so you need something targeted first.
In future versions, I'll attach it to the Dock/Launch menu from the top, so that it's accessible even when you don't have a target.

It lists objects in space in 2 columns:
(1) Points of Interests
(2) Known Locations

Tab between the two.
And use the arrow keys to move up and down the list.

(1) Points of Interests lists gates, and unknown stations and ships in the sector. This removes the need for the long-range scanner. But if you want that mini-game, I can amend this feature so that it only lists those that your long-range scanner has found.
(2) Known Locations lists known stations and ships in the sector.

Both lists order their contents from nearest to furthest.

When you click on a line (or press enter), its context-sensitive command menu opens.
Available commands are:
1. Auto-pilot.
2. Dock (if you're within 20km). This is the same as using the Dock Now command from the Interact Menu.
3. Target.

I tested it with the keyboard, but I'm unsure if your screen reader will be able to navigate and read it.
Hopefully, it can and you can use it.

[xant]

I'm giving the game a try now!
One hard challenge i find is that when using the long range scanner, I think i am supposed to move towards the question marks that appear in the map, and detect the objects as my ship gets closer to them.
I do not know which direction to go though and i don't think i can ask the ship to auto pilot towards these question marks, can i?

Autopilot (shift + a) will get you to whatever you have marked by the "Start Guidance to Object" command, be it a question mark or even empty space. Your target doesn't have to be actively selected, it is enough to have it as guidance target. It works from the cockpit as well as from the map.

I hope it helps!

[oasis1701]

Autopilot (shift + a) will get you to whatever you have marked by the "Start Guidance to Object" command, be it a question mark or even empty space. Your target doesn't have to be actively selected, it is enough to have it as guidance target. It works from the cockpit as well as from the map.

I hope it helps!

I think @kuertee's mod that he just posted helps a lot with the challenge that i was having, since i was randomly flying around and trying to find objects and jumpgates which i couldn't.
But i am curious to know, what do you mean by the guidance command? I was googling a bit but i am unsure; is marking objects done by page up and page down key and then pressing t? then pressing shift+g is the guidance command and then shift+a is toggle autopilot?

@kuertee thanks so much for this mod, I will also be trying it as I also use a screen reader as well! Very exciting.

[CaplinCaplin]

T should not be necessary, as far as I know shift+G will turn on guidance, and shipt+a for autopilot, however. The page up and down commands cycle through targets within radar range, which is around 40 km. T is for targeting whatever you are looking at visually, as far as I know.
You'll want to find the option in settings to have the computer speak out the name of targets with page up/down.

[xant]

Guidance command is a special option in the right-click-drop down menu, be it for an object on the map or any already selected object while flying a ship. Once such an object is marked, the shift-A autopilot will fly to it, even if said object isn't actively selected anymore. To make it clear, it works for objects that are in different sectors.

And as I already stated, you can also select any point in space and set a guidance point there, although this only works on the map and not from your cockpit. Once such a point is set, your autopilot will fly to that particular empty point of space that you marked as guidance point, as long as it is reachable in any way.

Now I don't know if it helps you in any way, but I admire your dedication and wish you all the best for your endevour.

[CaplinCaplin]

I appreciate the work of kuertee and anyone else willing to help out with this project.

Hey Caplin, here's the next version of the Accessibility Features mod: https://drive.google.com/file/d/1FERv9b ... sp=sharing
It's still in my Google Drive, but I'll upload it to Nexus once you get a good go at it.
You'll need to install my UI Extensions mod for its new custom menu to work: https://www.nexusmods.com/x4foundations/mods/552
Along with Sir Nuke's Mod Support API mod (https://www.nexusmods.com/x4foundations/mods/503) - which I know you've already installed.

The new feature is the Navigation Computer.
Here's a screenshot - in case your screen reader has OCR capabilities: https://i.imgur.com/aH5FeWB.jpg

It is accessible from the Interact Menu, so you need something targeted first.
In future versions, I'll attach it to the Dock/Launch menu from the top, so that it's accessible even when you don't have a target.

It lists objects in space in 2 columns:
(1) Points of Interests
(2) Known Locations

Tab between the two.
And use the arrow keys to move up and down the list.

(1) Points of Interests lists gates, and unknown stations and ships in the sector. This removes the need for the long-range scanner. But if you want that mini-game, I can amend this feature so that it only lists those that your long-range scanner has found.
(2) Known Locations lists known stations and ships in the sector.

Both lists order their contents from nearest to furthest.

When you click on a line (or press enter), its context-sensitive command menu opens.
Available commands are:
1. Auto-pilot.
2. Dock (if you're within 20km). This is the same as using the Dock Now command from the Interact Menu.
3. Target.

I tested it with the keyboard, but I'm unsure if your screen reader will be able to navigate and read it.
Hopefully, it can and you can use it.

I'm afraid this isn't working for me or Oasis1701. The UI just fails to appear. I've installed your UI extensions mod, and "Navigation computer," appears in the menu, but upon clicking it nothing happens.

I wonder if we're missing something? I note that I was trying this with an old save game, though would hope that wouldn't make a difference.

[oasis1701]

Hey Caplin, here's the next version of the Accessibility Features mod: https://drive.google.com/file/d/1FERv9b ... sp=sharing
It's still in my Google Drive, but I'll upload it to Nexus once you get a good go at it.
You'll need to install my UI Extensions mod for its new custom menu to work: https://www.nexusmods.com/x4foundations/mods/552
Along with Sir Nuke's Mod Support API mod (https://www.nexusmods.com/x4foundations/mods/503) - which I know you've already installed.

The new feature is the Navigation Computer.
Here's a screenshot - in case your screen reader has OCR capabilities: https://i.imgur.com/aH5FeWB.jpg

It is accessible from the Interact Menu, so you need something targeted first.
In future versions, I'll attach it to the Dock/Launch menu from the top, so that it's accessible even when you don't have a target.

It lists objects in space in 2 columns:
(1) Points of Interests
(2) Known Locations

Tab between the two.
And use the arrow keys to move up and down the list.

(1) Points of Interests lists gates, and unknown stations and ships in the sector. This removes the need for the long-range scanner. But if you want that mini-game, I can amend this feature so that it only lists those that your long-range scanner has found.
(2) Known Locations lists known stations and ships in the sector.

Both lists order their contents from nearest to furthest.

When you click on a line (or press enter), its context-sensitive command menu opens.
Available commands are:
1. Auto-pilot.
2. Dock (if you're within 20km). This is the same as using the Dock Now command from the Interact Menu.
3. Target.

I tested it with the keyboard, but I'm unsure if your screen reader will be able to navigate and read it.
Hopefully, it can and you can use it.

I am also unable to bring the UI up or it may be that our screen reader can't OCR it? There's a very short video of me trying it so you can see as well, if the UI is not coming up or if it does but my screen reader can't see it.
https://www.dropbox.com/s/ymv5piscr1565 ... 5.mp4?dl=0
Let me know if OBS does not show the game, I hope it does!

[kuertee]

I'm afraid this isn't working for me or Oasis1701. The UI just fails to appear. I've installed your UI extensions mod, and "Navigation computer," appears in the menu, but upon clicking it nothing happens.
I wonder if we're missing something? I note that I was trying this with an old save game, though would hope that wouldn't make a difference.

I am also unable to bring the UI up or it may be that our screen reader can't OCR it? There's a very short video of me trying it so you can see as well, if the UI is not coming up or if it does but my screen reader can't see it.
https://www.dropbox.com/s/ymv5piscr1565 ... 5.mp4?dl=0
Let me know if OBS does not show the game, I hope it does!

ohhh! I just checked versions of UIX.
Accessibility Features mod requires a new UIX which i've not released yet, but can be downloaded here: https://drive.google.com/file/d/1qA8M8F ... sp=sharing
(sorry, i should have checked that first.)

but ... what version of the game are you two running?
4.0 or 4.1beta1? UIX, is unfortunately limited to those two versions.
(I have too many mods that i'm delaying 4.1 support until its final version.)

also ... the new menu needs a "frame" or two to tick before it opens.
does the ocr pause the game? the video looks like its paused.

i can hear the mouse as it moves down to "Navigation Computer" but the video itself is static.
the mouse doesn't move. or is that just the video recorder?

if the ocr pauses the game, you may need to go back and unpause the game after clicking Navigation Computer for its menu to open.

[CaplinCaplin]

but ... what version of the game are you two running?
4.0 or 4.1beta1? UIX, is unfortunately limited to those two versions.
(I have too many mods that i'm delaying 4.1 support until its final version.)

Hey,

I can say that both of us are running 4.0, as I didn't think I wanted to be a beta tester while still learning how to play the game. This is very good indeed. There are a couple minor things I noticed, but I'm very happy with it.

Firstly, might it be possible to have the two modes on separate screens, that is, tabbed UI of some sort? The OCR flattens the list, so the two columns become one long one. I envision us primarily using this by routing the mouse to specific items, rather than arrow keys, just because if we use arrows it's harder to tell which item we're highlighting.

SEcond, maybe you could close the UI after selecting a gate or other object and choosing what to do with it? I noticed it stayed open, which was slightly confusing. I imagine it might be more so in the case of gates if you're still seeing stuff from a sector you're no longer in.

Thanks so mucch fro all the work on this.

[oasis1701]

ohhh! I just checked versions of UIX.
Accessibility Features mod requires a new UIX which i've not released yet, but can be downloaded here: https://drive.google.com/file/d/1qA8M8F ... sp=sharing
(sorry, i should have checked that first.)

but ... what version of the game are you two running?
4.0 or 4.1beta1? UIX, is unfortunately limited to those two versions.
(I have too many mods that i'm delaying 4.1 support until its final version.)

also ... the new menu needs a "frame" or two to tick before it opens.
does the ocr pause the game? the video looks like its paused.

i can hear the mouse as it moves down to "Navigation Computer" but the video itself is static.
the mouse doesn't move. or is that just the video recorder?

if the ocr pauses the game, you may need to go back and unpause the game after clicking Navigation Computer for its menu to open.

re: the video, I think the fact that i was not moving the camera made it look like it's paused, or owhen i OCR the screen, it puts me in a virtual hidden notepad like feature where i can freely move my cursor without effecting the camera, but it does not pause it thankfully!
re: game version, i think we're both using the current version on steam! haven't opted into any betas or anything.
By the way, the UI works now, thank you so much, is it possible to by some easy logic, these two columns become two different screens like my friend mentioned? so when one is shown the other is not. The reason is that our OCR throws a big chunk of text at us whenever we scan the screen, and i will show you an example output of our OCR result below.
points Of Interuts
Jump Gate
Stations
Unknown Station
Unknown Station
Unknown Station
Unknown Station
Unknown Station
Unknown Station
Sh*s
183 km
49.55 km
99.8 km
125.47 km
191.9 km
19225 km
197.34 km
52.23 km
63.56 km
107.31 km
108.08 km
113.71 km
117.96km
122.32 km
1228B km
12424 km
134_56 km
136.25 km
14281 km
143.07 km
Known
ARG Graphene Refinery I
ARG Argon Defence Platform
ARG Drone Factory I
ARG Graphene Refinery II
ARG Argon Equipment
ARG Medical Supply Factory I
Elite Vanguard
38.54 km
63.05 km
64.92 km
195.68 km
205.02 km
220.89 km
*
*
Unknown Ship
Unknown Ship
Unknown Ship
Unknown Ship
Unknown Ship
Unknown Ship
Unknown Ship
Unknown Ship
Unknown Ship
Unknown Ship
Unknown Ship
Unknown Ship
Unknown Ship

for example if we wanted to focus on the refinery, we had to press the arrow key 30 times to reach it!
edit: if that very long text looks very bad on the forum let me know so i can pastebin it

[kuertee]

...Firstly, might it be possible to have the two modes on separate screens, that is, tabbed UI of some sort? The OCR flattens the list, so the two columns become one long one. I envision us primarily using this by routing the mouse to specific items, rather than arrow keys, just because if we use arrows it's harder to tell which item we're highlighting. ... SEcond, maybe you could close the UI after selecting a gate or other object and choosing what to do with it? I noticed it stayed open, which was slightly confusing. I imagine it might be more so in the case of gates if you're still seeing stuff from a sector you're no longer in.

By the way, the UI works now, thank you so much, is it possible to by some easy logic, these two columns become two different screens like my friend mentioned?

nice, that the custom menu works!

i can continue adding to the list (e.g. when in radar range: mission offers attached to stations, when within 10ks: signal leaks list, etc.)
and, yes, i can put tabs at the top instead of presenting the 2 lists at the same time.
and i'll auto-close the menu after selecting a command. i kept it up open to allow continual browsing while auto-piloting. but that can also be achieved by simply re-opening the menu.

also, thanks for the ocr output. it'll help me re-layout the table so that it makes sense when the ocr reads it out.
it's weird that the list started with the "Points of Interest" title but then read the contents of the "Known Locations" first.
i'll probably just put the distance immediately after the name - instead of separating it into the second column.
also, the tabbed UI will make the ocr read only one list at a time.

but ... i might not get this "fixed" version out until tomorrow or the day after.
and i won't be able to release the version with more actions (mission offers, signal leak lists, etc.) until a few days after that.
sorry to delay your expeditions.

[oasis1701]

nice, that the custom menu works!

i can continue adding to the list (e.g. when in radar range: mission offers attached to stations, when within 10ks: signal leaks list, etc.)
and, yes, i can put tabs at the top instead of presenting the 2 lists at the same time.
and i'll auto-close the menu after selecting a command. i kept it up open to allow continual browsing while auto-piloting. but that can also be achieved by simply re-opening the menu.

also, thanks for the ocr output. it'll help me re-layout the table so that it makes sense when the ocr reads it out.
it's weird that the list started with the "Points of Interest" title but then read the contents of the "Known Locations" first.
i'll probably just put the distance immediately after the name - instead of separating it into the second column.
also, the tabbed UI will make the ocr read only one list at a time.

but ... i might not get this "fixed" version out until tomorrow or the day after.
and i won't be able to release the version with more actions (mission offers, signal leak lists, etc.) until a few days after that.
sorry to delay your expeditions.

Thank you so much for helping us with this mod, IT allows us to explore x4's world!
It is quite odd that the windows 10 OCR engine (that our screen reader uses for OCR features) behaves like this. , quite glad to know that there can be a way to make it read better.
Thanks once again.
edit: Is it possible to change the format of the game's text, or make them larger or have a contrast to the background for the OCR to be able to read it better? i think it reads almost to 90% accuracy, though.

[CaplinCaplin]

but ... i might not get this "fixed" version out until tomorrow or the day after.
and i won't be able to release the version with more actions (mission offers, signal leak lists, etc.) until a few days after that.
sorry to delay your expeditions.

HEy,

Thanks so much for the work on this. I totally understand the delay, take your time. I'm just ecxited to have access to a game like this at all. It wasn't possible until recently.

Re: OCR, it's super frustrating. There are other alternative systems which might give better output of data in columns, but they aren't as straight-forward to use. I haven't tried them much beyond quick sampling, though I might have to at some point for my sanity if nothing else.

I truly appreciate the mod and very much look forward to the future.

[oasis1701]

Had a question about crystals, can they be found by the targeting system? or they have to be found manually and be shot at.

[kuertee]

yes, x4's targeting system is quite granular.
i can target crystals specifically.

i'll be adding more tabs to the list.
one of them would be for mining (e.g. asteroid regions, mineable asteroids, etc.)

you may want to test my other mod, Loot Mining (https://www.nexusmods.com/x4foundations ... id=8762008)
that mod does 2 things after the asteroid breaks up (after you shoot it enough times) into mineable minerals and loot:
(1) dropped mineable items are added as mission targets which your auto-pilot can fly to one at a time
and
(2) the next mineable asteroid is targeted. once you finish collecting the mineable asteroids, you can set this with the Set Guidance To Object to make it your auto-pilot target.

just to clarify, there are several targeting systems in the game:
first system: mission targets: these can be multiple objects. and your auto-pilot will fly to these one at a time.
and second system: weapon target: you can only have one of these. and your weapons track these.
there's a passive third system which tags objects as detected quietly. this third system is used for scanning station modules - which require you to navigate around the station, so don't worry about this third system.