[Official] Technical Engine Development faq / blog

General discussions about X Rebirth.

Moderator: Moderators for English X Forum

vadiolive
Posts: 337
Joined: Wed, 18. Dec 13, 04:36
x4

Post by vadiolive » Thu, 22. Jan 15, 19:06

i asked in S&M section but since guys tell me its hardcoded here go

Q: Diference between MK1 Mk2 and number drones i wish know somes example and calcs - i feel much less diference between

Plus:

Q: Savegame size make diference in game perfomace? usual i get dlowndown and freenzy after game save reach around 70-80mb at 90mb become uplayble (can be mods conflict dont test yet in vanila)

UniTrader
Moderator (Script&Mod)
Moderator (Script&Mod)
Posts: 14571
Joined: Sun, 20. Nov 05, 22:45
x4

Post by UniTrader » Thu, 22. Jan 15, 21:58

a small question at CBJ:

do the Savegames of XR represent how the Data is actually organized internally in the Game? or are there major differences between these?
if not stated otherwise everything i post is licensed under WTFPL

Ich mache keine S&M-Auftragsarbeiten, aber wenn es fragen gibt wie man etwas umsetzen kann helfe ich gerne weiter ;)

I wont do Script&Mod Request work, but if there are questions how to do something i will GLaDly help ;)

CBJ
EGOSOFT
EGOSOFT
Posts: 51740
Joined: Tue, 29. Apr 03, 00:56
x4

Post by CBJ » Thu, 22. Jan 15, 23:32

There is some correlation, of course, but how close it is varies depending on where you look, and there is also a lot going on in the game when it's running that doesn't need to be saved at all for one reason or another. In particular our XML serialisation and deserialisation process makes heavy use of defaults to avoid saving and loading values that have not changed, which has the dual benefit of reducing savegame size and also making patching easier.

Taking the universe structure, for example, the organisation of the overall universe structure in-game is pretty much as you see it in the savegame, but the details within each element of that structure often differ considerably from how they appear in the file. Other parts of the savegame, for example the script data, may well differ at the structural level too.

Night Nord
Posts: 1002
Joined: Fri, 13. Jan 12, 19:09
x4

Post by Night Nord » Fri, 23. Jan 15, 23:21

timon37 wrote: For general programming gnu/linux is much better, build tools and general methods for the development process are (most of the time) simply superior by design. VS project configuration is... beyond words. When I started with opengl on windows I simply said to myself "nope, not doin' that" and used cmake, which helped me a lot:)
I can almost feel the pain =) After being a low-level unix programmer and switching to a windows game engine development... It's like coming from XXI century back to the stone age. Not only tools are just lacking or require tons of 3rd party plugins to do even basic stuff, but also my colleagues showed lots of ignorance in some things I've considered very basic and "common truth". Like common and very strong belief that TCP is always buffered and that's why you should always use reliable UDP for any sane purpose (thank you MS for advertising your own broken implementation as a standard...). I'm so glad I've started my programming carrier from opensource development and I dream of a day I'll be able to port our engine to Linux...

But back to questions, suddenly to CBJ. A serious one, about the Lua code: do we still need the decompiler? I mean, it's kinda not working with post 3.0 scripts - you've broken it with your new fancy colors for trading offers ;P Soo... do we need to fix it or you are going to release Lua code in some foreseeable future?

Also, can you give us headers for Anark file formats? Then it will be, at least, possible to convert those files into some textual format (or into a new Anark/Nvidia studio format, or into a Blender format, or whatever) and then convert back to use in the game.

Or maybe I'm terribly misinformed and all this have been already released? Because people are still asking me to fix that decompiler =)

P.S. And yes. Why the new small Teladi ships look like flying forks and not like this. I mean. It seems to be obvious, isn't it? :P

MellowTigger
Posts: 2
Joined: Thu, 28. Sep 06, 01:43
x3

Post by MellowTigger » Fri, 30. Jan 15, 22:13

I'm so glad to hear that the game will be coming to Linux. I'll be sure to buy a copy when it's available. :)

(aside: I don't suppose you'd consider publishing via Good Old Games too? They also carry a variety of Windows/Mac/Linux games.)

timon37
EGOSOFT
EGOSOFT
Posts: 485
Joined: Fri, 14. Dec 12, 11:02
x4

Post by timon37 » Mon, 2. Feb 15, 10:15

MellowTigger wrote:(aside: I don't suppose you'd consider publishing via Good Old Games too? They also carry a variety of Windows/Mac/Linux games.)
Probably no. Honestly I already can't handle the maintenance work I have.
Just a dream, but what I'd really like is a the standard linux approach: we setup our own server with a few repositories for different systems and we tell everyone to just add it to their repo-list.

Authentication (verification of payment), can surely be somehow implemented if deemed necessary.
The main benefit is not having to deal with any 3rd-party which is annoying as hell.
Main disadvantage is bandwidth... we need to implement p2p package distribution;)

The bulk of the cost is in initial setup, later you mostly reap the benefits of a fully automatic process, assuming you've already got an epic build system with virtual machines for the supported distros;)

CBJ
EGOSOFT
EGOSOFT
Posts: 51740
Joined: Tue, 29. Apr 03, 00:56
x4

Post by CBJ » Mon, 2. Feb 15, 11:03

Sorry for the delay in replying, Night Nord; I didn't notice your questions.
Night Nord wrote:But back to questions, suddenly to CBJ. A serious one, about the Lua code: do we still need the decompiler? I mean, it's kinda not working with post 3.0 scripts - you've broken it with your new fancy colors for trading offers ;P Soo... do we need to fix it or you are going to release Lua code in some foreseeable future?
In the not too distant future we will be providing the raw Lua scripts for modders to work with, and eliminating the need for them to decompile. This should make most UI modifications a lot easier.
Night Nord wrote:Also, can you give us headers for Anark file formats? Then it will be, at least, possible to convert those files into some textual format (or into a new Anark/Nvidia studio format, or into a Blender format, or whatever) and then convert back to use in the game.
I'm afraid not. However we believe that the tools provided within the Lua UI framework will be sufficient for most modding requirements.

Night Nord
Posts: 1002
Joined: Fri, 13. Jan 12, 19:09
x4

Post by Night Nord » Mon, 2. Feb 15, 14:07

timon37 wrote: The bulk of the cost is in initial setup, later you mostly reap the benefits of a fully automatic process, assuming you've already got an epic build system with virtual machines for the supported distros;)
And for that you may just publish debs only - distros will make their own packages working around yours (and there are lots of automatic tools to transform packets from one system to an another). But authentication is going to be a problem - in my knowledge there is no authentication methods in common package managers aside of asking a user to go by the URL and download a package themselfs (providing any required information) - which kinda kills the whole point of having a standardized package system. Steam is still your best bet, IMO.

In the not too distant future we will be providing the raw Lua scripts for modders to work with, and eliminating the need for them to decompile. This should make most UI modifications a lot easier.
Oh, that's great news, thank you.
I'm afraid not. However we believe that the tools provided within the Lua UI framework will be sufficient for most modding requirements.
Well, that's a pity, but I guess you have some legal implications with Anark license preventing you from disclosing any of it's contents or formats. But it worth a try =).

If you mean "there will be enough tools to make new buttons etc" then yes - there are more then enough means to do new pages, tables and other kinds of UI. There is a bit of functionality hunger though, in terms of an exposed API - it's even smaller than a MD API. And won't allow us to add new graphical content - new icons, animations, etc... Unless you have something in mind to enable that.

User avatar
edqe
Posts: 73
Joined: Tue, 19. Nov 13, 15:30

Post by edqe » Tue, 3. Feb 15, 05:51

There will be a session in GDC'15 about glNext.

glNext: The Future of High Performance Graphics (Presented by Valve)

User avatar
ezra-r
Posts: 3420
Joined: Fri, 14. Oct 05, 21:04
x4

Post by ezra-r » Thu, 5. Feb 15, 22:31

@timon37 any news about the linux version?

Even tiny little anecdotic news will do (like rendering squares instead of triangles now) :P

timon37
EGOSOFT
EGOSOFT
Posts: 485
Joined: Fri, 14. Dec 12, 11:02
x4

Post by timon37 » Fri, 6. Feb 15, 10:33

Well I decided to change the gameplay a bit for the OpenGL version of the game.
It'll be like closure where if something isn't lit up by a light it ceases to exist and you can fly through that space!

[ external image ]

Nah just kidding;p This is an old screen shot of OpenGL on windows when I finally got my hands on an nvidia gpu and started testing on it.
The problem there was https://msdn.microsoft.com/en-us/librar ... 85%29.aspx, precisely the "If the length of the x parameter is 0, the result is indefinite". Notice this is the HLSL spec so this could have failed for us in DirectX as well, but it just happens that nvidia/amd/intel all return 0 in that case.
With opengl however nvidia returns a NaN, and a "NaN + whatever" is still a NaN. Thus everything that wasn't within a local light was black;p

Now please don't over-interpret this, there's still a very significant amount of stuff to do and the game's currently unplayable. Unfortunately porting involves mostly dealing with issues which pop-up as you go, so I really can't say how much longer it'll take.
But yeah it looks no worse than that screenshot on linux;)
Last edited by timon37 on Fri, 6. Feb 15, 12:13, edited 1 time in total.

User avatar
ezra-r
Posts: 3420
Joined: Fri, 14. Oct 05, 21:04
x4

Post by ezra-r » Fri, 6. Feb 15, 11:38

:lol:

no misinterpretation of any kind, I just like to see what you are on to relating the linux version, no need for ETA either, just share the fun/interesting bits or even if you just want to be vocal about whatever happens when you do linux port, frustations, etc, fine for me!

Keep up the good work and thank you very much for the update!

User avatar
edqe
Posts: 73
Joined: Tue, 19. Nov 13, 15:30

Post by edqe » Fri, 6. Feb 15, 14:55

timon37 wrote:Now please don't over-interpret this, there's still a very significant amount of stuff to do and the game's currently unplayable. Unfortunately porting involves mostly dealing with issues which pop-up as you go, so I really can't say how much longer it'll take.
But yeah it looks no worse than that screenshot on linux;)
Great to hear the project is going forward. It seems to be much further than I expected :)

Code: Select all

$ ./XRebirth 
Segmentation fault (core dumped)
OT, what features do you expect and would like to see in glNext?

timon37
EGOSOFT
EGOSOFT
Posts: 485
Joined: Fri, 14. Dec 12, 11:02
x4

Post by timon37 » Sat, 7. Feb 15, 13:08

Well for me personally the announcement of glNext (or whatever it's called) was a huge let down, to put it mildly.
Basically with everyone jumping on the "ooo new lightweight, close-to-the-metal, low-overhead, whatever" bandwagon opengl was the last API available which wasn't a dead-end, and didn't have glaring limitations (e.g. one platform, one vendor, etc). Furthermore with the examples from AZDO presentations it was clear-ish that it could deliver low-overhead already (how that really compares to mantle/dx12 is of course an open question).

In short, me and other developers could say "opengl works on practically everything, it has roughly all the features we need, it's extensible and can quickly evolve to the current needs, it can deliver low-overhead today, it's messy but what we invest in it doesn't get thrown out in 3 years".

Now the last part in that sentence is gone, and that's bad. Of course when I'm unhappy I'm assuming that opengl actually isn't a technical dead-end. I'm assuming that it can be further evolved to accommodate the changing hardware. Which may or may not be the case.

But the way I see it, it should be doable. The main issue with opengl is that there's a bunch of ways to reach the same goal (e.g. setting the vertex format). Right now new extensions are usually introduced such that the old functions still work, and you can mix between the older and newer api.
This is fine, but imho it's often unnecessary, I'd rather if enabling a certain extension disabled a bunch of old functions, since in most cases I have to put in additional effort to make sure I'm not calling the old stuff (because I'm not sure about all the details and corner cases).

So my opinion is that khronos is jumping on the bandwagon due to excessive hype generated by everyone else. They have to do something new because otherwise everyone will think they're obsolete.
They probably also see it as a chance to get rid of the "messy" badge, which isn't a bad thing in itself, but it does mean that my code will at some point get more messy because the plan of "we'll just focus on opengl and throw other renderers out" is no longer a long-term option.

As for what they actually do with the new api, well I mostly hope they don't screw it up and that things are well defined and drivers behave consistently.
Improving multi-threaded operation is definitely good (though shared contexts seem to work already). I also hope any unnecessary implicit synchronization is easier to avoid, or easier to detect & debug, moving to requiring explicit synchronization is also fine with me.
Having a good free-software debugger/profiler would be godlike, I hope something comes into existence sooner rather than later.
There's a bunch of small gripes I have with current opengl especially relating to glsl, optimizing it, and the interfaces around it, but honestly that's all details.
Truth be told since I'm still stuck in "porting from dx9" mode I haven't really moved to even some of the "current" stuff, and I'm not even close to AZDO, so saying what I'd like beyond a point I'm not yet at is a bit tricky;p

User avatar
ezra-r
Posts: 3420
Joined: Fri, 14. Oct 05, 21:04
x4

Post by ezra-r » Wed, 11. Feb 15, 00:53

there seems to be an interesting debate/comments (including some from an Aspyr Media person/developer) about glnext in here:

http://www.reddit.com/r/linux_gaming/co ... ed_at_gdc/

timon37
EGOSOFT
EGOSOFT
Posts: 485
Joined: Fri, 14. Dec 12, 11:02
x4

Post by timon37 » Wed, 11. Feb 15, 10:43

We'll see how it goes, I do hope that there are some really good reasons for redoing the whole thing.
I unfortunately never had a good look at the driver side of the API and when I think about it simplifying that is potentially more significant than the user-facing changes.
The main problem we're facing right now is that every driver on every platform behaves a differently, which is a major pain in the rear;p

User avatar
Tamina
Moderator (Deutsch)
Moderator (Deutsch)
Posts: 4543
Joined: Sun, 26. Jan 14, 09:56

Post by Tamina » Wed, 11. Feb 15, 11:55

Yeah there was a really big discussion between lots of developers a few months ago who talked about their concerns that they don't know if they should see glNext as the next big major step (or just forget about it) and if so if it is still worth investing time in current OGL then, as nobody knows when glNext is actually published and market-ready.

I mean is glNext going to be compatible with older cards?
My old video card was OGL2 HW compatible but the driver was OGL4 SW compatible, though he skipped many effects and other visual stuff it was very well playable.

Despite all of that, I am very thrilled about your OGL version.
I just want to see how it performs.
XR got better and better and you guys did a great job increasing the performance a lot but it still is partially a pain especially when compared to other games with better graphics, more meshes, more details and way more stable FPS together with higher DX-versions.
So I am very thrilled to see your work in action someday ;)

Code: Select all

Und wenn ein Forenbösewicht, was Ungezogenes spricht, dann hol' ich meinen Kaktus und der sticht sticht sticht.
  /l、 
゙(゚、 。 7 
 l、゙ ~ヽ   / 
 じしf_, )ノ 

User avatar
edqe
Posts: 73
Joined: Tue, 19. Nov 13, 15:30

Post by edqe » Sat, 21. Feb 15, 18:30

timon37 wrote:Well for me personally the announcement of glNext (or whatever it's called) was a huge let down, to put it mildly.
Basically with everyone jumping on the "ooo new lightweight, close-to-the-metal, low-overhead, whatever" bandwagon opengl was the last API available which wasn't a dead-end, and didn't have glaring limitations (e.g. one platform, one vendor, etc). Furthermore with the examples from AZDO presentations it was clear-ish that it could deliver low-overhead already (how that really compares to mantle/dx12 is of course an open question).
...
...
That was good read. It will be interesting to see how good/bad OpenGL 4.x support will have for new gaming GPUs once glNext has been release and ready for use.

For those who are interested, OpenGL AZDO presentation and demo:
http://www.slideshare.net/tlorach/openg ... eroverhead
https://www.youtube.com/watch?v=n6QP2ORmVOA

timon37
EGOSOFT
EGOSOFT
Posts: 485
Joined: Fri, 14. Dec 12, 11:02
x4

Post by timon37 » Wed, 25. Feb 15, 15:17

edqe wrote: For those who are interested, OpenGL AZDO presentation and demo:
http://www.slideshare.net/tlorach/openg ... eroverhead
https://www.youtube.com/watch?v=n6QP2ORmVOA
I didn't have time to really read it, I just glanced through the slides.
But woah cool stuff, though that's going "quite a bit" further then the previous azdo presentations;p

dmk
Posts: 682
Joined: Fri, 25. Nov 05, 00:59
x4

Post by dmk » Thu, 26. Feb 15, 04:35

edqe wrote: For those who are interested, OpenGL AZDO presentation and demo:
http://www.slideshare.net/tlorach/openg ... eroverhead
https://www.youtube.com/watch?v=n6QP2ORmVOA
this will be cool when it's get into standard, now it's can be used only for cad systems (i.e. write different (shaders) code for different hardware).

Post Reply

Return to “X Rebirth Universe”