Please DO HAVE EARLY ACCESS for X4 paying Customers

[CBJ]

Advantages of beta testing

The advantages of beta testing are not, and never have been, in any doubt. What is being discussed here is whether it is better to perform that beta testing in private, i.e. under NDA, and with some control over the number and quality of testers, or in public, with no NDA and little or no control over who tests or how they do it. There are advantages and disadvantages to both, many of which have been highlighted by people earlier in the thread.

[LittleBird]

The argument over if EA is responsible for the discovery of the performance issues is probably not one that can be proved one way or the other. But in any case, I would not mark this down as fans demanding a bad feature be removed.

That is not the point.
Ketraar asked about an example for changes to the core design. And that is one. No matter if the players demand removing it or the devs saw it does not work.

Kethaar stated:

I'm rather sceptic to how much "power" players have in EA games to warrant significant impact in structural design choices.

So I think the reasons for a features removal being stated as because of performance issues an not player input goes directly into the this point.

Let me pick one:

"Granted I have not gone the EA route for many games myself and only follow the development of a few more, but in no case did I find any indication that devs made changes to their core game design."

And they did. Not because the players asked for it but because the player tested it and the devs could see how it worked out.
Of course we will never know if the outcome would be the same without the EA phase.

And what I wrote earlier relying on examples is pointless. Do you really believe there is not a single EA game where people claimed and then the devs changed a core feature? What if somebody finds one that would exactly satisfy this case? It would mean nothing to the discussion.

[Ketraar]

Sorry but no, the argument was to have a way for player to impact the development of games. To which I still claim there not be any examples of where the direction of games design was influenced by having a game in EA over the "traditional" methods. As I said, games evolve and features are removed/added/balanced all the time, thats just how games are made, but I have not seen any indication that a core feature was changed or even added based on the fact that a game was in EA.

If you take X games for example many features were added based on player/modder feedback over the years, so thats nothing new, but none of those changes/additions lead to a fundamental redesign of core elements. When a game dev actually starts coding there has to be a core design in place as to what the game will be and what core features it will contain, I doubt you could start developing a RPG game and then change it into a platformer because you have it in EA and players rather like the later, it just does not make sense.

Which is imho the core argument here, that some players think that putting a game in EA is a magical solution and that it means they can have a say in the shaping of the game, which they wont beyond polish and balancing.

MFG

Ketraar

[BigBANGtheory]

If EA isn't going to work well for Egosoft then it will not work well for the player(s) end of story. Bottom line is we should respect Egosoft's decision on the topic because only they know what will work for them.

Personally I have a high tolerance for bugs, but a poor tolerance for weak design decisions regarding gameplay and user experience. Would EA help with those area's? Only if Egosoft is open to that which is why I'm happy to let it slide if that is their decision.

Anyone that is intolerant to both conditions needs to step back and assess the game maybe 3months post release is my advice but make ur own mind up just don't moan about your mistakes cos that gets dull real fast. You know the score from X3, you have your own experiences of XR if you buy on day one then expect it to be an EA experience with fixes and improvements coming down the line.

[sd_jasper]

Advantages of beta testing

The advantages of beta testing are not, and never have been, in any doubt. What is being discussed here is whether it is better to perform that beta testing in private, i.e. under NDA, and with some control over the number and quality of testers, or in public, with no NDA and little or no control over who tests or how they do it. There are advantages and disadvantages to both, many of which have been highlighted by people earlier in the thread.

"...quality of testers..." In my experience this is key. Number of testers means nothing if they are not good at testing. A bad tester at BEST gives no feedback, and worst contributes to the NOISE that makes it harder to act on useful feedback. Unless you utilize passive data collection which can provide some useful information... but that's a whole other can of worms.

[Slashman]

Sorry but no, the argument was to have a way for player to impact the development of games. To which I still claim there not be any examples of where the direction of games design was influenced by having a game in EA over the "traditional" methods. As I said, games evolve and features are removed/added/balanced all the time, thats just how games are made, but I have not seen any indication that a core feature was changed or even added based on the fact that a game was in EA.

I have been giving an example of just this for a long as hell time on these forums and people keep ignoring it. Amplitude Studios. Makers of the Endless games. All of their games use Early Access and are BUILT around community input. Their entire initiative is called Games2gether.

https://www.games2gether.com/

I hope we never have to have this discussion again as to if players can have a direct say into the core design of games and have those games be both successful and well-received. The answer is a resounding YES!

Players are currently voting as to the design of the next playable race to be added to Endless Space 2. The player community designed from scratch an entire playable race for Endless Legend (The Cult of the Eternal End). I know I have stated this before.

Yes, Ketraar, when properly structured, it works just fine. But it does take some effort and know-how.

[StoneLegionYT]

However this is only referring to applications. I am still not convinced that this applies to games like the X-series.

No no not just X-Series but all games. The term Beta sure 20 Years ago you could say this is what a Beta Means. But sadly there was a big abuse of the Term Beta during the MMO Era of gaming once it kicked off a lot around WoW Times. Then people used Beta's to pretty much Marker their products a early way to make money if it was to buy into the beta or other wise.

Then EA Games showed up on Steam and did pretty much the same thing.

That being said it really now depends on the game and company and how they handle it.

[BigBANGtheory]

I don't think it helps to talk about 'testing' as some generic term since there are many forms of testing particularly with respect to a video game.

imho functional tests should be performed by professionals and willing volunteers where as user experience tests require focus groups. Using your customer base as your focus group is a high risk strategy, cheap to implement but at the potential risk of reputation damage to your brand.

[Ketraar]

Yes, Ketraar, when properly structured, it works just fine. But it does take some effort and know-how.

Again I repeat myself, how is that exclusive to EA games? Also does this not fall under the "devs asked for input" category I mentioned above? There seams to be some sort of communication dissonance and we are talking past each other.

MFG

Ketraar

[Slashman]

Yes, Ketraar, when properly structured, it works just fine. But it does take some effort and know-how.

Again I repeat myself, how is that exclusive to EA games? Also does this not fall under the "devs asked for input" category I mentioned above? There seams to be some sort of communication dissonance and we are talking past each other.

MFG

Ketraar

Maybe we are talking past each other. I don't know.

When Endless Space 2 was announced, players were presented with Game Design Documents on the core systems. Those systems were then refined and in some cases completely changed during EA based on player feedback. As in, gutted and replaced based on player input.

The difference between the devs at Amplitude and the Egosoft devs is that all devs are actively encouraged to participate in forum discussions and OFTEN have back and forth idea sessions with the community. It is actually a requirement for their jobs that they interact with the community in this way.

We, the players, actively sought and got major changes to how government systems in the game work just recently and those changes went live in the last major update.

We, the players, actively sought changes to the combat system and got it completely reworked on several levels working with the devs. Heck I personally had a hand in getting swarm missiles and AoE warheads implemented when a dev thought my idea was a good one.

There are times when they will say that something is not possible due to resource constraints etc. but more than any other developer I have ever seen, the common members of the community impact the game in very tangible and noticeable ways. And pretty much all issues get some kind of dev response and discussion unless they were thoroughly covered elsewhere.

If you want some kind of example where some 12 year-olds just start screaming that everything sucks and a game got major changes because of that I don't have any. But I don't understand why you would think that what Amplitude does is not incredibly remarkable and very different in approach to what Egosoft does.

Or maybe I just missed when the Ego community helped design the Split from scratch...

[ZaphodBeeblebrox]

@Slashman

Some of that appears to be good for both the community and the developers.

However there is currently quite a deep schism in the community here.
One which I think needs to be healed before such actions as you describe can be successfully enacted.

I am hoping that X4 provides the means to bring all of the fans of the X games back together.

[Ketraar]

Or maybe I just missed when the Ego community helped design the Split from scratch...

There is no need to be sarcastic, especially wrongly applied one. X factions are based on existing fiction (Helge's Novels) and not something that can be designed by community.

I also didnt say that open systems are a bad thing, in fact I'm pretty sure I said both have merits. Still I'd put your example into the "devs asked for input" category, which again is nothing new and not exclusive to EA at all.

MFG

Ketraar

[Slashman]

There can be no changes made due to player requests unless devs are receptive to input. Whether asked for or not.

So categorizing my example as something that can only happen when the devs ask is kind of moot. I also gave 2 examples where players told the devs that two systems needed to be changed and they were extensively so. Both the combat system and the government systems. I guess they don't count either.

I am also not advocating that Egosoft adopt these policies or any policy in particular for that matter. I am just stating that they can work and they are not the nightmare or insurmountable hurdle that is being presented.

What I can say for sure is that games are art. And as such developers should be free to express themselves through their art as they see fit. But the reality is that this is balanced by the fact that they need people to purchase their games.

As such, unless they have unlimited time and budget to experiment as they see fit, they are beholden to their prospective audiences in a way that they ignore at their peril. The degree to which this is so is dependent on several factors including the type of game, the size of the existing audience and whatever is being requested by that audience.

I also agree with Zaphod about the community being divided at present. That is definitely a good reason not to pursue the route of increased player input right now.

Currently I think X4 looks to be on the right track in terms of delivering what most people want at the core of an X game. We don't have all the details though, and in a sense that does make me reluctant to go "all-in".

I hope that Egosoft has finally gotten to the point where they deliver a solid and relatively bug-light game at launch which leaves them more time and resources to devote to adding features and content both paid and otherwise which are requested by the community.

[LittleBird]

You know the score from X3, you have your own experiences of XR if you buy on day one then expect it to be an EA experience with fixes and improvements coming down the line.

Wait a sec.
Unfinished releases. Shall we make a... tradition out of it?
On top of that nothing anymore speaks against going EA with such a "release" version. But calling it "release" or "day one" is just a plain lie.

[Ketraar]

There can be no changes made due to player requests unless devs are receptive to input. Whether asked for or not.

That was the point. But if you read through forums, more so on steam, you will find that there is a sense of entitlement to be involved in shaping the games (not just X games). We come so far to have the same discussion in movies, see SW TLJ, where movie goers expect a movie to be shaped based on their vision.

THAT is the notion I was trying to highlight and claim that neither system will EVER cover that type of involvement unless the people making the art "allow" it and that if they dont, it not warranting criticism.

We are in agreement on everything else.

MFG

Ketraar

[Vandragorax]

The only successful "public" beta test I've seen for an early access game has been Subnautica. It had to be led by their QA lead (Obraxis) and it was controlled under NDA about who was invited and who wasn't. They always made sure they had a sufficient small number of testers in a regular weekly session, and as rightly pointed out by many above me here, they were controlled carefully with what aspects they were testing each week (areas that Obraxis knew had been specifically touched by dev's code changes that week).

I work as a business software test lead and know how difficult it is to get any useful details from bug reports from someone who has not been trained as a tester. There is a lot of information required in order to reproduce some issues and good feedback is essential for the devs to be able to collect and fix things without wasting a lot of time guessing what's going on or what the user's expectations were. It is absolutely no help to anyone to be flooded with bug reports saying "This thing doesn't work" or "I can't do so and so" with little other information.

IMO if any beta testing is to be done, it is essential to do it in a controlled way with a proper QA lead who can guide the testers and collate the information to present to the dev team. Having an open invite testing is not real testing, it is more as was mentioned above, user experience testing and has a completely different goal to functional testing.

I do honestly think that a lot of times when early access buyers think they are testing or shaping the game, they are only under that illusion but don't have any real influence in the game's direction, and I feel that would definitely be the case for an X game. Bernd and the team already know what they want to do with it, and they have a good technical understanding of how and what is/isn't possible. The usefulness they could find in a "public" test would be simply testing new functionality for bugs, or regression testing heavily changed functional areas, which should definitely be done under NDA in a controlled way.

[Graaf]

The only reason EA will be good, is you will see other data then just running your current build on 100hrs of SETA. You will have actual people playing the game using the THINK part to do things devs might not have anticipated. You will also get early reports of basic things missing, like radar (probably not this time).

[Fleabum]

Having an open invite testing is not real testing, it is more as was mentioned above, user experience testing and has a completely different goal to functional testing.

Being involved in development myself I have to disagree with you on the above point, open invite testing is essential to some phases of development. You can do simulated testing but it is nowhere near as good for data mining as using real life scenarios with the myriad of network, hardware and software platforms out there.

For example, stress testing multiplayer or patching environments, including patch, download, connection handoffs, latency, disconnections and ghosting. Open beta testing is a proven test strategy, backed up by the amount of mainstream developers who have open beta 'weekends' and sneak peeks to stress test these exact facets (amongst others) of development. Far Cry 5 recently used this strategy to test their arcade and sandbox game play elements.

I know multiplayer environment tests are not useful for single player games like the X series, but they do have their place in the beta test cycle. And that's why I disagree with you that open invites are not real testing, they are, but they have to be used in the correct manner.

I do have to agree with you in the fact that the majority of 'beta testers' have no idea about how to fill in a bug report form, and end up costing you more time and effort than the report is worth. I get the feeling that a lot of 'beta testers' are more bothered about playing the game earlier, more than than actually helping to quash bugs.

Regards
Flea

[StoneLegionYT]

I do have to agree with you in the fact that the majority of 'beta testers' have no idea about how to fill in a bug report form, and end up costing you more time and effort than the report is worth. I get the feeling that a lot of 'beta testers' are more bothered about playing the game earlier, more than than actually helping to quash bugs.

Very true but one thing I want to add to this is there is some out there who will say oh that bug when they cry about it and go in game and test it or try to attempt to get it then report it themselves. I know I have done that a lot with random open source projects I was part of over the years as a none coder just a user of the application and wanted to help and give back.

[Slashman]

There can be no changes made due to player requests unless devs are receptive to input. Whether asked for or not.

That was the point. But if you read through forums, more so on steam, you will find that there is a sense of entitlement to be involved in shaping the games (not just X games). We come so far to have the same discussion in movies, see SW TLJ, where movie goers expect a movie to be shaped based on their vision.

THAT is the notion I was trying to highlight and claim that neither system will EVER cover that type of involvement unless the people making the art "allow" it and that if they dont, it not warranting criticism.

We are in agreement on everything else.

MFG

Ketraar

Fair enough. You are right about the Star Wars thing. It is kind of crazy to have people claiming ruined childhoods over a movie but hey...if they want to spend all that angst...OK.