[MOD/API] Named Pipes API

[SirNukes]

Download: https://github.com/bvbohnen/x4-projects/releases
Documentation: https://github.com/bvbohnen/x4-projects ... es_API.md


X4 Named Pipes API

Adds support for Windows named pipes for inter-process communication.
Named pipes are OS level psuedo-files which support inter-process communication.
Pipes avoid the overhead of disk access from normal files (eg. debug logs), and are bidirectional.
X4 will act as a client, with one or more an external applications serving the pipes.

There are three components to this API:

• A low level lua plugin and corresponding dll for pipe access.
• An MD wrapper that interfaces between the lua and MD pipe users.
• A default, optional Python external pipe host server.

Requirements

• Lua Loader API extension
  - https://github.com/bvbohnen/x4-lua-loader-api/releases
• Time API extension
  - https://github.com/bvbohnen/x4-time-api/releases
• Optionally, Simple Menu API
  - https://github.com/bvbohnen/x4-simple-menu-api/releases
• Optionally, Python 3.6+ with the pywin32 package.
  - Only needed if running the pipe server from python source code.
  - An executable is provided as an alternative.

Installation

• Place the named_pipes_api folder in extensions.
• Place the X4_Python_Pipe_Server (exe or py) anywhere convenient.
  - Run this pipe server alongside X4.

Components

• MD Pipe API
  This api is found in Named_Pipes.xml, and allows MD code to read or write pipes.
• MD Server API
  This api is found in Pipe_Server_Host, and allows MD code to register a python module with the host server.
  The host (if running) will dynamically import the custom module.
  Such modules may be distributed with extensions.
• Lua Pipe API
  The lower level support for X4 to use named pipes is located in Named_Pipes.lua.
  Basic pipe usage can ignore this level in favor of the MD api.
  Advanced usage may want to use this api directly.
• Winpipe dll API
  The pipes themselves are accessed through a dll which wraps key Windows functions.
  This dll is limited to only what is needed: opening a pipe client, reading/writing/closing it, and error checking.
  If desired, the dll may be compiled freshly from source on github.

[spin1/2]

Hey,

I tried out the Simple Menu API (includes Lua Loader API), because my target is to create mission sequences like a visual novel.
For a simple test I took your "simple_menu_api_test", which means I had to inlcude the Hotkey API and the Namend Pipes API.

All four APIs are Version 1.0 and in the extensions folder. The server is running and shows the one line with the path.
When X4 is running it shows the Hotkey API and the Namend Pipes API in red (in the extensions menue).

thx for your work and help

[SirNukes]

Thanks for trying it out. After checking a few things, my best guess is that the winpipe dll had an unnecessary dependency on visual studio. Version 1.0.1 is up with a fix.

Regarding the menu api, the extra test extension only uses the hotkey/pipes stuff for opening a standalone test menu (using shift+b). Without the pipes, that same test menu can also be opened using the options menu. (Also, the hotkey stuff uses 3.0 md commands, so won't work on 2.6, but the menu api should be fine.)

[SirNukes]

Updated to 1.1. This changes the access timeout behavior a little bit. By default timeout is now disabled (used to be 10 seconds). When a timeout is requested by the user of this api, a new option is available to also flush out all pending accesses to the pipe that timed out. The change is motivated by the Key Capture API losing inputs due to their matching read requests having been timed out.

[Misunderstood Wookie]

You're going to have give some solid reasons why this is better then every other Lua API. I mean you require us to run server side software just to use it so what exactly does this do that is beneficial?

Is this just a method for live debug output?
Unless you're using a HDD I can't see how disk overhead is much concern tho.

[SirNukes]

You're going to have give some solid reasons why this is better then every other Lua API. I mean you require us to run server side software just to use it so what exactly does this do that is beneficial?

This API enables run-time communication with X4 from external processes. That is, you can send data *to* X4 dynamically. There is nothing comparable to it across any of the X games. (The closest I know of is that one modder that was trying to add multiplayer to X3 by messing around at the exe level.)

I leave to your imagination what can be done with such a capability. Personally, I actively use it for creating new hotkeys.

[Misunderstood Wookie]

I see so it's used to make things like we have for elite dangerous and push data to some their party tool or display live, or something else like maybe a live galaxy map which runs on a second screen all the time and shows everything you know ingame without slowing the game down having the map open hahaha

[Inchínn]

Any mods that utilise this so far?

Would love a html map screen etc,

[magitsu]

Any mods that utilise this so far?

Would love a html map screen etc,

Mycu's External App.
https://github.com/mycumycu/X4-External-App

[Inchínn]

Any mods that utilise this so far?

Would love a html map screen etc,

Mycu's External App.
https://github.com/mycumycu/X4-External-App

i found it straight after my comment - its really slick, so cool to see the job offers on another screen without having to click into the menus.

I really hope they expand on its features.

[X2-Illuminatus]

Added to Tools, Tutorials and Resources overview.

[beko]

It's not yet ready for public usage and has some issues that need to be ironed out but… I present you: Named Pipes on Linux PC

https://beko.famkos.net/wp-content/uplo ... pc_too.jpg

Hooked into this great mod by using Lua-Socket with only a few extra lines of code. Used the "X4 External App" to get some noise on the socket and here we go

[Strohmann]

It's not yet ready for public usage and has some issues that need to be ironed out but… I present you: Named Pipes on Linux PC

https://beko.famkos.net/wp-content/uplo ... pc_too.jpg

Hooked into this great mod by using Lua-Socket with only a few extra lines of code. Used the "X4 External App" to get some noise on the socket and here we go

Great news! Where can I follow your progress?
P.S.: Great blog! Enthusiastic people are always admired. And I perfectly understand the state when setting up or learning something is one of the most exciting elements of the game.

[beko]

Great news! Where can I follow your progress?
P.S.: Great blog! Enthusiastic people are always admired. And I perfectly understand the state when setting up or learning something is one of the most exciting elements of the game.

Thanks mate. My blog is my primary source and you can follow it thanks to bridges from various social media. They are linked in the upper left (not on mobile). I also have a tiny Discord channel where we're mostly bouncing ideas about SimPits around. Let me know if that is your jam

Currently looking for some testers to not run into "Works On My Machine"

[Strohmann]

Currently looking for some testers to not run into "Works On My Machine"

I'm ready to take part in testing.
But now there is not too much free time - all the time is occupied by a little baby and work.

I use X4 External App when playing on windows. I am also a Python backend developer, it would be interesting to understand how the Pipe-server works.

Is there a fork of the bvbohnen/mycumycu projects or instructions for the necessary changes?

[beko]

I did fork it but I didn't push to it yet. Really hacking away every free second and writing a well versed tutorial is kinda obsolete the moment I start typing Wouldn't mind to walk you through and brainstorm some ideas.

[beko]

Been a while but https://github.com/bekopharm/x4-project ... ick-manual for Linux PC is up and in sync [today] for 7.0. I've at least one successful install on the PC of another user running the "X4 External App" with it. So if you are a Linux user and want to have the

Code: Select all

sn_mod_support_apis

extension for Linux too: Now you can

Important: This does _not_ use the "X4 Python Server" or any of it's functions (like permission files) and this is why it will probably never be merged upstream. I mean I kinda assume that Sir Nukes will not accept it because it works way different. This also requires some DLLs though (".so" files on Linux) that your distribution may provide so you may trust into your distribution of choice for this. The manual describes where the libs have to be placed/linked. After configuring and starting it's X4 itself that will raise socket files for each extension requesting one (hence no need for the X4 Python Server) to which external apps can connect to.

I wrote this to scratch my own itch of dragging data out of X4 for my https://github.com/bekopharm/x4-simpit mod and while sending data back to it is supported I did never test it with the hotkeys extension. YMMV.

And yes it's a horrible and hacky mess I'm still refining. Also I'm standing on the shoulders of giants here. This would not have been possible without Sir Nukes amazing work and foundation and Mycu's inspirational use of it You rock!