LAN multiplayer mod concept

[nightshade221]

alrighty... i know im pushing into ground that has already been tread on and bringing up a concept that has been brought up and subsequently smashed into the ground. however. i do believe with a hardworking team, the right tools, and the technical know-how, this thing is possible. my concept is as follows:
lan multiplayer/online multiplayer for x3 and eventually all the other single player x titles. think of it like multiplayer in freelancer. everyone has a profile and has to work their way from the ground up. the admin of the server of course has the ability to allow script editing and also the ability to edit scripts while the other users can not (unless allowed, as previously stated) i am aware that there is no multiplayer code built into any games from the x universe (yet). in response to those who say this is not possible at all, take a look at the mod for elder scrolls 3. it obviously is. it just takes the right ...stuff. for lack of a better term.

just think about it. being able to get together with all your friends and folks from the forum and wage all out war against eachother, the ka'ahk, and the xenon. the universe in x3 is already expansive enough to allow this and for the servers not to have to worry about overcrowding. everyone here knows the moral code we all follow in the forum, so bringing that over to the game wouldn't be a challenge.

as far as the persistence of the universe goes, while the client is offline, there are several options:
1. force dock all of the client's ships at their home station using the safest route possible but not force the client to wait for that to happen. make a trigger that only activates when the client leaves the server.
2. same as above except when the client logs off, have all ships gain an invulnerability to damage of any sort and continue their objectives.
3. allow only one ship per user and eliminate user owned stations alltogether.
4. stations come completely automated with everything they need to operate including ts and tp ships.
5.etc... but you see where im going with this..

it is possible.
but keep in mind im only good for a concept. not too good with heavy modding and only some light scripting.

the only thing we need to make this happen is a dedicated team that has at least one member familiar with tcp/ip and the game's coding. and hell, this is the egosoft forum and im sure at least a few of their dev's will read this. assuming im not bludgeoned to death for bringing this up.. lol

comments and questions very much appreciated.

max

[sc_hornetAMF]

I have thought about mp deathmatch before and the only way I can see if working is by exporting the position of your ship into a file and then uploading that file to a server which then distrubutes it to everyone who then update it in the game.

But then im not sure if the X3 script has the ability to do file IO, and even then, it would probably be too slow to be workable.

Additionally, theres no way to track bullets and who would do the hit detection? there is a certain amount of randomness in the bullets and it would soon all get out of sync.

Basically its not possible. At all. Theres not a shread of support for it in the X engine. As a computer programmer myself, I can say that.

[amirite]

The fact of the matter is, X3 has the ability to interface with net protocols. That alone allows multi to be possible. If people didn't constantly trample all over the idea some bright young coder could come along and surprise us all.

It is possible. But the eventual solution will definitely not be ANYTHING like what anyone here is picturing. It won't be classic deathmatch, it won't be a free-roaming MMO-like experience. With enough creative coding though, not only can (some) of the technical obstacles be overcome, but new gameplay decisions can be made to bypass the problems in the first place.

It would definitely be a lightweight multiplayer experience. But the possibilities are there. All it would take is a dedicated team who knows their stuff.

Problem is, the X3 community is severely lacking in people that know their stuff, not to mention dedicated teams...


Now, here's where I start talking out my ass about things I don't know anything about: The first thing people always bring up when talking about this idea are the problems with syncing data. Yes, X has a large number of random variables, trajectories, etc. What's preventing all of this from just being calculated on a dedicated server as normal? I don't see why the connected clients have to be the ones doing that heavy lifting. If you combined this method with some sort of "online sector" it doesn't seem like there would be much data involved. Especially if you make that sector OOS only. Granted, it's not very exciting, but it would allow for multiplayer interaction with the economy, and be a large building block for future features.

[nightshade221]

and who's to say the online version needs multiple sectors? i know of a mod that is being developed right now that basically turns X into one enormous contiguous sector. with that mod in place on an online server, you eliminate alot of the problems. the only thing that would need changes are ship speeds and possibly a different jump drive. im not to great with scripting but changing a ships speed is probably one of the easiest tasks to do in any mod. all this takes is a few knowledgeable people.

[edit]
(didnt see the first comment...)

in response to the first comment. who's to say we cant just take the random factor out of the bullet trajectory?

[Shadow dream]

putting all the sectors into one? hm, basicly that's the ebst idea to solve this problem. Better, you can seperate this sector into many:
You can have more thann only 4 gates per sector (I don't know any limits) -> you can keep the sector sizes, you just have to place them far away enough from each other! That means lots of planets, many suns (or just one or two that are placed correct),... At least that part is possible!

I know that once has been a discussion like that, but can't remember where; further the thread is closed, but there are some good ideas in it!

Good luck!

Shadow

[nightshade221]

? closed? i cant tell if you're a moderator or not... and do you mean this thread or the old one?

[Shadow dream]

I'm no moderator If I was I had that sign over my avatar

I mean the old one I was talking about. Many people started to complain that it wouldn't work (and many said it would work) and so a (probably annoyed) moderator closed the thread until the author want it to be open again.
If you find this thread you can maybe contact the author or at least read the thread to get ideas

Shadow

[Carlo the Curious]

Moderators are easily identifiable by a tag under their name. You can also get a list from the index page under each forum.

The only recent thread I remember which discussed practical ways to mod in multiplayer is this one.

[nightshade221]

haha. i thought not. and i did manage to get to that thread a while back. i did notice that alot of the arguements could probably piss off a moderator... lol.

however im hoping that wont happen here. its very frustrating knowing that all this takes is the right person reading this thread. im kinda hoping that someone is a dev... haha. i just keep thinking how awesome it would be if you and a bunch of friends all logged in to a private server and could do battle against the xenon. it would remove the need for ship commands and be so much more fast paced and intense.

just picture a massive human controlled fleet consisting of a carrier and 2 destroyers approaching another fleet of xenon M1's and fighters. all of a sudden around 50 player controlled fighters come zooming out of the hangar bay and start blasting the crap out of the xenon. and of course everyone is connected through x-fire and the designated admiral is barking commands over the voice chat channel to each squadron. of course with the community we have here on these forums everyone would love it and totally get sucked in. god... i sound like a five year old playing pretend.. lol. but hey! its funnn! haha


{edit}

i just pm'd s9lent and asked him to take a look at this thread but as his last post on his concept was a month ago, im not sure how soon he'll be able to read this. could a moderator, or anyone really, point me towards some knowledgeable people who might be able to get this idea at least started or considered?

[s9ilent]

At the request of night shade I'm going to dole out some information as to how you can make such an idea possible.



Note: You will need to learn how to program/script in both X3, and an external program.

The basic way you get data flowing from one to the other is:
1. X3, outputs to a log file
2. Your externally made program reads the log file, clears it, then sends the information to a server
3. The server reads it, processes it or what ever, and sends a signal back
4. The signal is sent back and recived by the external app
5. The external app then sends the info back to X3 via a tfile
6. A script in X3 decodes the input from the tfile, and does the resultant action

Now this is the basic setup you would use. (Using a Synchronised TCP connection, which is the simplest and safest)




If you want to get a bit more into it, then you probably want to switch from a synchronised connection to an unsynchronised one, such as by using UDP (which is more common in games, as dropping packets is ok, you just program your apps to account for it.)
The advantage of UDP is you save a little on overheads

TCP has a whopping 128 bits of overheads
UDP can be sent with as little as 32
For a LAN game which updates multiples times a second, you would expect many smaller packets, also you would prefer clients to experience a lag drop out, rather then the connection trying to re-establish it self and retrieve lagging packets. (as that would lag the whole server, or result in unnecessary data transfer, as we don't care if I missed the last 10 seconds, I want to know what is happening right *now*)





In either case, this is a rather large project and you should not consider undertaking it lightly.
It requires serious technical skills (or the ability to learn said skills, probably the later) and lots of time and most importantly planning.

[nightshade221]

thank you very very much for your input!!! as i said, im mainly bringing up the concept. however i start college this fall and im currently learning some simple programing. is there any way that x3 could be modified itself to allow connections to a server without the use of an external program? basically cut out the middle man so instead of outputing to a file which is then loaded to the server, x3 sends the data directly. that way you cut out a decent amount of de-synching and get smoother gameplay.

[s9ilent]

Sadly there is no way to cut out the middle app
I really wish there was, but there isn't.

The only output (ever) is log files

And the only input (in game) is tfiles
(and physical inputs from peripheries, eg mouse, keyboard, joystick)

(And you can not establish connections with the game)



A possible alternative suggested by Jeff (the Jeff posting on my thread) is to make the middle application use a keyboard hook, such that it can send its data directly to the game via the keyboard. There are complications with that however (most notably that you would have to be in the flying in space screen with no windows open and not lagging for the commands to work properly, and it wouldn't accept double characters)

Of course this only removes one side of the lag (i.e. App-> X3, but the X3-> app lag will still be there)

Whilst in the realm of the work around, another (which is COMPLETELY beyond me...) would be to make "virtual" game controller (i.e. a joystick), and send commands to X3 via that directly. But thats completely beyond me, just an idea to not tie up the keyboard. (But all the same, it would still only reduce half the lag and it would be hard to learn how to do, and hard to set up, as you would have to bind the controls for everything)
-And ease of use is a VERY important factor which making any script/mod

[nightshade221]

what if we took the idea of the keyboard hook and turned it around? instead of a text file that the game creates, the middle program simply records the game's start and all user input which it then uploads to the server which then distributes the info to the rest of the players? and we're talking a small file size so speed shouldnt be too much of a problem. basically like recording a macro.

[s9ilent]

Hmm... I don't think that idea would work, as essentially, it will require you at some point, to put those inputs back into the game.
But one persons mouse/keyboard/joystick is not the same as someone else's.


That approach is also questionable, as it is not possible to tell what each action is doing. Moving the mouse in the menu, would not move the ship, where as moving in a flight move would move the ship. So its not really practical to try and send control data. It also is not possible to rectify errors with this method.

If you lost a packet, then each of the packets would build on the previous assumptions. (eg I pressed, c, 1, 5 for idle. But then if the 1 droped, it would just c 5, command none)


Where as by sending raw data, you can tell it things like the ships absolute position right now. So all previous data is not important.

[knifekillu]

First off to combat those saying its not possible because the game dosn't support it.

If a game supports file IO its nearly half way to being multiplayer already.
The true deciding factors are:
Game Requirements
1. Can the file input be parsed
2. Is the file output parseable or informative enough
3. Can the game read strings from files
4. Can the game compare strings and perform functions based on comparison (if statements while loops etc)
5. Does the game allow npc control by its mods
6. Does the game allow object and npc spawning by its mods

--
Thats all that is really needed and the level of support of those requirements will decide the level of enjoyment that multiplayer will bring.

The main subject then is implementing some form of system. In most of the posts from this thread and a few others I have seen when I came here to check mods etc; People have described the control from their client and updating the server of what happens in their client...
Any system built in a client centric point of view will fail.

Implementing Server Centric Multiplayer Environments...
The clients start with a completly empty sector and the server tells them who, what, and where. The server has to be in complete control of the sector. It has to be able to give the client a list of things to spawn and where and if they move it has to calculate the changes. If a player moves it has to calculate a safe direction and speed along with the transmission delays so it can update each client.

In X3 from my understanding of the scripting language the server is going to have to tell the client that it needs to have shipXXXX move to xyz at this speed and at this rotation and if the tracked object changes speed as its accelerating it has to update its speed. To a point the server has to try and predict the waypoint that a human ship is going to. For true NPC ships the easiest method is to use the game to build the sector and control the npcs in the multi-player world sending its updates to the server. This is a much laggier and less thought out idea the true solution is that the server does the AI and everthing else.. You are basically building a console based game engine to control the X3 game engine.

Thats movement.. and thats assuming the server dosn't lag or anything else. Im not sure if you can set a ships xyz rotation etc explictedly if you can that would give a good way to keep the worlds in sync at realtime, rather then having to have out of sync periods of downtime to postion a ship properly. Else you will have to have a large error margin set up so as the server or the client is not trying to micro-manage the postion so much so that it falls further and further behind.

The script engine in X3 is fairly weak from what I have seen at program control and timing so alot of effort will have to go into the command processing portion to get it to a level to handle the task. Starting small is the key though. Spawning a ship and letting a external program control it and add in one function at a time to the command processor.

The actual layout that is required for this task is
X3 Game scripts:
1. output what you do in game such as movement, firing a weapon, and other basic game functionalities to a file.
2. parse input from tfile and translate it to a function to run and the parameters that function needs to do whats required
3. the actual functions that will take data and turn it into actions performed in the game world.. (spawn ship, move ship, despawn ship, etc)
4. an error or out of sync response script to try and keep your game world synced at the scipt level with out causing a disconnection due to interfereing too much with the server

Client
1. Needs to parse the output of the game translating what it says to useable function calls in the client
2. Calculate as much as possible to take away from the server calculating out trajectories and movement
3. Needs to write the proper inputs to a file preferably doing the queuing of commands and calculating timing and figuring in new server updates before sending it to the game client
4. It needs to have a layer of sync correction to compliment the script correction aswell as error resolution when what it gets dosnt make sense
5. it will handle all communications with the server

Server
1. Build the game world and manage it storing the game world
2. Do some base calculations to send clients time sensitive data that is time corrected from the inputs of other clients
3. Send only relevant data to each client sifting through the junk that dosnt effect other clients. This also entails figuring on view distance and culling updates when needed
4. Like everything else it must have a layer of sync correction and error handling the end result of not being able to handle it simply disconnect a client or a more friendly approach force a client to do a full reload or a partial reload.. (if a player ship is too far out of sync simply despawn it on the OOS client and respawn it properly).
5. It must handle multiple communications from multiple clients even if its only 2 players if you are building a server you should just build in the idea of multiple ports from the start.
6. It has to be able to process client inputs and translate them to changes for each client this leads into it doing base calculations depending on client latency and other connection level things aswell as detecting events.
7. See into the future, well not really but it has to be able to calculate into the future rather then just dealing with the present as you wont send where something is now but where it will be by the time the client gets it.. the client then sends where something should be by the time the game recieves the command this all means that server and client has to be tuned to X3 physiques and beable to know how much thrust something has.

Hopefully this has been of any use to someone who is thinking of doing this. It is not as hard as it seems.. Just requires starting small and expanding and knowing what hurdles have to be cleared and how to view the problem to find the proper solution.

ps. I thought I would add a possibly frowned upon solution for the much slower and cumbersome file IO and relying on scripting engine.. Memory manipulation and native function execution. I have not read the AUP or anything else so I can not comment on legality of these methods, in all likelyhood they would require reverse engineering which I will take a guess at is not allowed, however being a single player game pure memory manipulation should be allowed although the means of figuring out what to manipulate in memory may require things that are prohibited.