Split from "Why have you guys traded metal for plastic?" thread

This forum is the ideal place for all discussion relating to X4. You will also find additional information from developers here.

Moderator: Moderators for English X Forum

BlackRain
Moderator (Script&Mod)
Moderator (Script&Mod)
Posts: 6053
Joined: Mon, 15. Dec 03, 19:53
x4

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by BlackRain » Fri, 5. Jul 19, 17:50

You were moderated probably because you made a personal attack on someone using offensive language, I don't know though since I didn't see it. My comment wasn't even harsh and please don't read meaning into it, just stating my observations as I see it. I didn't call anyone an idiot. I mean if you think the behavior I described is idiotic, that is your own personal interpretation. I consider it more to be along the lines of foolish, not necessarily idiotic. People do foolish things all the time, especially with the anonymity of the internet. I mean, on the internet, I can be anything I want to be when anonymous. I can claim myself to be an expert on anything and know everything. My issue is that you were basically insulting egosoft's credibility and the way they work, but you know nothing about why they made the decisions they did. You are not privy to their work environment and business decisions but felt the need to berate them on their choices. You then claimed to know better than the very people who made the game, especially CBJ. It is easy to be the one looking in from the outside and pointing fingers, especially when you aren't the one who had to put in the hard work and make the tough decisions.

And by the way, by no means am I saying there are no issues with the game. There are plenty of things to be worked on, so feel free to report the bugs and request the features you like. As for complaining about how they chose to do saving in the game, what is the point? It isn't going to change and I doubt it is something that could change at this point. This just isn't constructive.

Buzz2005
Posts: 732
Joined: Sat, 26. Feb 05, 02:47
x4

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by Buzz2005 » Fri, 5. Jul 19, 18:01

Thats the point, I could have wrote a wall. of text back then about having to wait loading for every sector you get into, it would get me nowhere

and to the saving xml thing, I would say it not a big problem for most of the players, hitting F5 here and there and waiting for half a minute is not end off all things problem, and for a guy that dont know anything about programming xml style save is very friendly to edit and see whats what

the game is very stable (at least for me) and you can configure the autosave interval, if it happens in some menu and its not clear still not a big deal, surely not something that would get me to stop playing

if you want something that's a potential big problem right now would be the NPCs building production modules on shipyards, thats screws up enough things to be mad about :evil:

CBJ
EGOSOFT
EGOSOFT
Posts: 37462
Joined: Tue, 29. Apr 03, 00:56
x4

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by CBJ » Fri, 5. Jul 19, 18:04

Midnightknight wrote:
Fri, 5. Jul 19, 17:32
First it's not multi threaded, your whole game goes unresponsive, and is completely frozen, so you were closing a menu when auto save kicked in, you are thinking the game crashed and are already ready to ctrl+alt+del ... And it takes forever. It's not a priority, fine, you put priority were you want, but at least you can't deny there is a real issue with this and it should be solved. (And in many other aspects in the game, and that are not "bugs")
My issue is more that sometime we feel like you say "Oh it's not important" like everything was fine and do not take it seriously.
Yes, the game is "frozen" while you save. If it weren't then the game state would be inconsistent, containing data representing states at different times across the duration of the save process. Using threading to make the game "responsive" during saving would be a terrible idea, and indeed most games avoid this. And yes, the save process takes a little time. Most games only have to save your progress through their "level" system, and the states of a handful objects on your current level; the X series games have to save the state of everything in the game universe (apart from a few things that can be re-created from other data, such as the physics environment). Again, the format of the resulting file is a factor in the time taken to save the game, but it's by no means the biggest factor. Would we like to improve save times? Of course. Would creating a whole new file format and re-engineering everything to use it be an effective use of developer time in trying to achieve this? No, it wouldn't.

Artean
Posts: 685
Joined: Tue, 14. Feb 06, 18:41
x4

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by Artean » Fri, 5. Jul 19, 18:12

Buzz2005 wrote:
Fri, 5. Jul 19, 18:01
and to the saving xml thing, I would say it not a big problem for most of the players
Indeed. To put your energy and complain about the saving system in X4 is just hilarious.
"In the beginning the Universe was created. This has made a lot of people very angry and has been widely regarded as a bad move." - D.N.A

IRONOX
Posts: 199
Joined: Mon, 23. Feb 09, 16:59
x4

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by IRONOX » Fri, 5. Jul 19, 18:14

Buzz2005 wrote:
Fri, 5. Jul 19, 18:01
Thats the point, I could have wrote a wall. of text back then about having to wait loading for every sector you get into, it would get me nowhere

and to the saving xml thing, I would say it not a big problem for most of the players, hitting F5 here and there and waiting for half a minute is not end off all things problem, and for a guy that dont know anything about programming xml style save is very friendly to edit and see whats what

the game is very stable (at least for me) and you can configure the autosave interval, if it happens in some menu and its not clear still not a big deal, surely not something that would get me to stop playing

if you want something that's a potential big problem right now would be the NPCs building production modules on shipyards, thats screws up enough things to be mad about :evil:
My game save for about 5-9 seconds. Which is faster than some of my citys in City:Skylines :P
And Ego using XML might be based on the fact that they used it since X2(?) for multiple things and so they get proficiend and used to it.

I cant see a damn reason why using XML is a "lazy" or "bad" thing....
Alle Angaben mit Gewähr auf eventuelle Fehlerhaftigkeit!

Buzz2005
Posts: 732
Joined: Sat, 26. Feb 05, 02:47
x4

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by Buzz2005 » Fri, 5. Jul 19, 18:20

IRONOX wrote:
Fri, 5. Jul 19, 18:14
My game save for about 5-9 seconds. Which is faster than some of my citys in City:Skylines :P
And Ego using XML might be based on the fact that they used it since X2(?) for multiple things and so they get proficiend and used to it.

I cant see a damn reason why using XML is a "lazy" or "bad" thing....
OMG yes my latest city aka the biggest and longest of saves does take some time, and the fps is like 15 8)

Falcrack
Posts: 2027
Joined: Wed, 29. Jul 09, 00:46
x4

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by Falcrack » Fri, 5. Jul 19, 18:52

Save times are only slightly annoying to me. Not something that detracts from my enjoyment of X4, and definitely not something that I would want to takes weeks or months of development time away from more pressing gameplay issues.

radcapricorn
Posts: 1909
Joined: Mon, 14. Jul 08, 13:07
x4

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by radcapricorn » Fri, 5. Jul 19, 18:52

BlackRain wrote:
Fri, 5. Jul 19, 17:13
This seems to be a big problem on the forum in general and it isn't just with any particular person but from many that I have noticed. There tends to be many so called self introduced "experts" who think they know better than anyone else how X, Y, and Z should be...
BlackRain wrote: ...I have been around way longer than you so I saw all of this and have first hand experience. You clearly came way later and your view/perspective is at a much different time...

...If you can't see the illogical aspects of your argument in regards to this, then there is no point in me discussing with you anymore...

...I don't think you get that...

...Since you are not a modder and don't know anything about the code at all, I guess I can understand why you can't see the work put into it...
Hypocrite much? Seriously, please try to measure up to your own standards before judging others.

EDIT: Misunderstood. Yikes :(
Last edited by radcapricorn on Sat, 6. Jul 19, 09:08, edited 1 time in total.

radcapricorn
Posts: 1909
Joined: Mon, 14. Jul 08, 13:07
x4

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by radcapricorn » Fri, 5. Jul 19, 19:02

CBJ wrote:
Fri, 5. Jul 19, 18:04
Yes, the game is "frozen" while you save. If it weren't then the game state would be inconsistent, containing data representing states at different times across the duration of the save process...
Can't you reduce the pause duration by taking just the snapshot of current state (which is what requires pause for consistency), and offload its processing and subsequent I/O to background? Assuming there are provisions in the engine for doing that, of course. E.g. if your state is structured in such a way that copying it wouldn't be any faster that straight-up conversion to XML, then it'd be moot.

Regardless, if anything else fails, please do try to find it in your heart to fix this. I.e. pause simulation before taking control away, and unpause it only after the game is again interactive. Please!

BlackRain
Moderator (Script&Mod)
Moderator (Script&Mod)
Posts: 6053
Joined: Mon, 15. Dec 03, 19:53
x4

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by BlackRain » Fri, 5. Jul 19, 19:03

radcapricorn wrote:
Fri, 5. Jul 19, 18:52
BlackRain wrote:
Fri, 5. Jul 19, 17:13
This seems to be a big problem on the forum in general and it isn't just with any particular person but from many that I have noticed. There tends to be many so called self introduced "experts" who think they know better than anyone else how X, Y, and Z should be...
BlackRain wrote: ...I have been around way longer than you so I saw all of this and have first hand experience. You clearly came way later and your view/perspective is at a much different time...

...If you can't see the illogical aspects of your argument in regards to this, then there is no point in me discussing with you anymore...

...I don't think you get that...

...Since you are not a modder and don't know anything about the code at all, I guess I can understand why you can't see the work put into it...
Hypocrite much? Seriously, please try to measure up to your own standards before judging others.
None of those posts are hypocritical. First, the initial comment I made had to do with perspective. I don't think you got at all what I was saying. What I was trying to say was that because I had been around longer and saw every release of every X game and went through the experience of having immensely buggy games at launch, missing features, the evolution of each of the X games, etc. this gave me a better perception of the X series overall. Which it does, and yes, experience/time does matter in this case. You came around at a time when X3 had already evolved and turned into a much different game than it was at release and then wanted to point out the flaws in X4 which hasn't been out for even a year yet. Not to mention, you couldn't even answer the question I asked and just ranted about bugs. Your comments, I have already seen thousands of times for pretty much every single X game although some of the concepts/features may have changed. You completely didn't understand this I guess.

The second thing, yeah your argument was illogical in my opinion and I got out of the discussion. This is called a personal choice and opinion, did I claim to be some kind of expert? Nope, but you certainly did, should I quote that? Not to mention, you are still being illogical now. I mean, you can't even argue a point without misconstruing my words and falsely accusing me of something that isn't even true. Nothing hypocritical in any of my comments, but it is a bit of irony that you are doing exactly what I said in my post here (cherry picking pieces of evidence which don't even fit the context).

Lastly, you have not a single modding credit to your name. If you do, please point it out for me. You claimed to be some expert on coding, but what have you contributed to the community to prove that? Nothing. Well guess what, I have contributed to the community quite a lot in fact. I have modded the X series since X3. Have you? Maybe you have and I just don't know it, please point me to the mods you have worked on. You just "Claimed" you were a programmer and looked at their code. Sorry if I don't take your word for it. I can prove I have done it though and yeah i know there are issues, but it is still impressive work in my opinion. So again, no hypocrisy there. I find this quite humorous though.

Well, whatever. This is another conversation I don't really want to post more on. I really have to stop wasting my time. So, go ahead, I'm done.

CBJ
EGOSOFT
EGOSOFT
Posts: 37462
Joined: Tue, 29. Apr 03, 00:56
x4

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by CBJ » Fri, 5. Jul 19, 19:12

radcapricorn wrote:
Fri, 5. Jul 19, 19:02
Can't you reduce the pause duration by taking just the snapshot of current state (which is what requires pause for consistency), and offload its processing and subsequent I/O to background? Assuming there are provisions in the engine for doing that, of course. E.g. if your state is structured in such a way that copying it wouldn't be any faster that straight-up conversion to XML, then it'd be moot.
State information is not stored in one place. The savegame is effectively the snapshot, with the structure of the XML corresponding fairly closely to the encapsulated code and datasets that generate it. Serialising it in a different format isn't going to change much.

On a moderation note: please stop the personal comments and the discussion of moderation.

User avatar
Tamina
Moderator (Deutsch)
Moderator (Deutsch)
Posts: 2022
Joined: Sun, 26. Jan 14, 10:56

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by Tamina » Fri, 5. Jul 19, 19:23

CBJ wrote:
Fri, 5. Jul 19, 19:12
radcapricorn wrote:
Fri, 5. Jul 19, 19:02
Can't you reduce the pause duration by taking just the snapshot of current state (which is what requires pause for consistency), and offload its processing and subsequent I/O to background? Assuming there are provisions in the engine for doing that, of course. E.g. if your state is structured in such a way that copying it wouldn't be any faster that straight-up conversion to XML, then it'd be moot.
State information is not stored in one place. The savegame is effectively the snapshot, with the structure of the XML corresponding fairly closely to the encapsulated code and datasets that generate it. Serialising it in a different format isn't going to change much.
If I understood him correctly, he was speaking about making a copy of the data in RAM and then save this copy, while the game keeps running.

I think it is weird that it takes so long to save. I deal with way bigger XML files (gigabytes) on a daily basis and it takes seconds to load them, parse them, process them and save them again. Then again, I have no clue what is happening in the background in the game but just reading the current state, doesn't sound that complicated at first. :D

Code: Select all

  /l、 
゙(゚、 。 7 
 l、゙ ~ヽ   / 
 じしf_, )ノ 
This is Tamina. Copy Tamina to your signature to help her achieve world domination.

strask412
Posts: 348
Joined: Thu, 29. Nov 07, 21:34
x4

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by strask412 » Fri, 5. Jul 19, 20:47

Tamina wrote:
Fri, 5. Jul 19, 19:23
CBJ wrote:
Fri, 5. Jul 19, 19:12
radcapricorn wrote:
Fri, 5. Jul 19, 19:02
Can't you reduce the pause duration by taking just the snapshot of current state (which is what requires pause for consistency), and offload its processing and subsequent I/O to background? Assuming there are provisions in the engine for doing that, of course. E.g. if your state is structured in such a way that copying it wouldn't be any faster that straight-up conversion to XML, then it'd be moot.
State information is not stored in one place. The savegame is effectively the snapshot, with the structure of the XML corresponding fairly closely to the encapsulated code and datasets that generate it. Serialising it in a different format isn't going to change much.
If I understood him correctly, he was speaking about making a copy of the data in RAM and then save this copy, while the game keeps running.
Heh.
And if I understood CBJ correctly when he said this:
CBJ wrote:
Fri, 5. Jul 19, 19:12
State information is not stored in one place. The savegame is effectively the snapshot, with the structure of the XML corresponding fairly closely to the encapsulated code and datasets that generate it. Serialising it in a different format isn't going to change much.
... he was speaking about making a copy of the data in RAM and then save this copy. He didn't directly say (that I can find anywhere) if the actual write-it-to-disk part is during, or after the pause, but what is apparently happening during this "just make a copy in RAM" is that it's making the copy and incidentally inserting some xml markup, while converting some binary stuff to UTF-8 on the side probably. He said the additon of this formatting during the "copy in RAM" part isn't significant from a performance standpoint.

As to the overall level of surprise expressed by people about duration of the save process, think of it this way: I just glanced at my paused copy of X4 and it's using almost exactly 4gb of RAM right now. If we "just make a copy" that's doubling our RAM consumption instantly. While of course the CPU is busy traversing the game's data in RAM and making the copies, I'm sure that there are also potential memory bandwidth concerns and possibly this sudden allocation of 4gb could trigger the OS to swap some other application out to disk or whatever, resulting in additional delays. I'm not trying to express "this is what happens", because I don't know all the specifics, but I'm trying to express "it is actually reasonable to expect that a lot of work is happening, and the duration of the save process doesn't particularly surprise me".

Edit: The comparison you make to your daily work:
Tamina wrote:
Fri, 5. Jul 19, 19:23
I deal with way bigger XML files (gigabytes) on a daily basis and it takes seconds to load them, parse them, process them and save them again. Then again, I have no clue what is happening in the background in the game but just reading the current state, doesn't sound that complicated at first. :D
may not be equivalent for a number of reasons; the one I can think of off the top of my head would be the memory requirement above. I don't know your code or anything, but from the words you used (load/parse/process/save) it seems unlikely to me that the entire file is stored in memory at once, it's probably got the input and output files both open for i/o and only the current chunk(s) of the input need to be in memory, once written to the output file they can be forgotten.

User avatar
Tamina
Moderator (Deutsch)
Moderator (Deutsch)
Posts: 2022
Joined: Sun, 26. Jan 14, 10:56

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by Tamina » Fri, 5. Jul 19, 22:57

strask412 wrote:
Fri, 5. Jul 19, 20:47
Edit: The comparison you make to your daily work:
Tamina wrote:
Fri, 5. Jul 19, 19:23
I deal with way bigger XML files (gigabytes) on a daily basis and it takes seconds to load them, parse them, process them and save them again. Then again, I have no clue what is happening in the background in the game but just reading the current state, doesn't sound that complicated at first. :D
may not be equivalent for a number of reasons; the one I can think of off the top of my head would be the memory requirement above. I don't know your code or anything, but from the words you used (load/parse/process/save) it seems unlikely to me that the entire file is stored in memory at once, it's probably got the input and output files both open for i/o and only the current chunk(s) of the input need to be in memory, once written to the output file they can be forgotten.
It's actually loaded as a whole and held in memory all the times but I don't think that makes a difference anyway because at the end it does the same. And for the next question: Yes, the RAM requirements during that time are tremendous, especially because it reads a lot of files in a few dozen threads simultaniously at any given time. And don't forget it holds the computed data in RAM as well :)

Anyway, every single file is bigger then a single savegame of X4 and the whole process takes roughly the same time. Which kind of counters the "XML is big and slow" (not wrong, but not to the extend many believe it to be). And this is exactly why I don't understand what X4 is doing during that time.

Is it building a DOM first? Please tell me it is not building a DOM!

Code: Select all

  /l、 
゙(゚、 。 7 
 l、゙ ~ヽ   / 
 じしf_, )ノ 
This is Tamina. Copy Tamina to your signature to help her achieve world domination.

Midnightknight
Posts: 79
Joined: Sat, 12. Jun 10, 11:49
x4

Re: Split from "Why have you guys traded metal for plastic?" thread

Post by Midnightknight » Fri, 5. Jul 19, 23:00

CBJ wrote:
Fri, 5. Jul 19, 17:05
Of course we could have thrown months of developer time into creating a format that could have achieved most of the above,
And be serious 3 seconds ... It do not cost months to have a save file system even one with all you mentioned. I don't even think it took one week to the man who wrote you parser from scratches ... And you can even go with XML if you wished and store a bit more data without repeating again and again the same tags.

Properties in XML do not required their own tags but are instead inserted in the parent tag between < and >. In your save format it's not, every damn property have it's own tag with the same words copied over again and again. Yes sur it has a cost, not only in memory but as a CPU too, you need to insert all those useless characters again and again, even on a XML perspective.
Last edited by Midnightknight on Fri, 5. Jul 19, 23:02, edited 1 time in total.

Post Reply

Return to “X4: Foundations”