Open Development and Reality

I've mentioned before that for Landmark and EQ:Next (and H1Z1 which goes to early access next week) SOE is using an 'Open Development' style of development.

This means that the developers are very accessible to the community and they are getting their ideas into the communities hands at a much earlier stage of development than has traditionally been the case. They aren't just talking about their ideas with us, they are letting us test the versions at a much earlier stage, listening to our feedback (as well as their own observations) and making adjustments to the game in a way that we can see the process.

I love the idea. I'm very much enjoying being involved in the process, giving feedback and reading their responses to the issues that the players are bringing up. I love the insight that it gives into the process. We get to see their idea, but we also get to see it change and evolve and in the end we have an understanding of WHY they made at least some of the choices that they did.

That said I think they've made a few mistakes with it as well. First is that they made it an every day alpha/beta, meaning that the servers have been up and accessible continually (except for patches) since Alpha started last February. Don't get me wrong, I've played a lot and had a ton of fun on my schedule because of that, but it also meant that there were long periods of time where there was nothing new to test, and with as early as they started there was really nothing to do in game either. 

The result was that for long periods the servers were ghost towns as people stopped playing for very long periods of time. Keeping an eye on the forums and waiting for news of a feature that they were interested in seeing. I think part of this was the result of unforseen delays that meant that some of the systems that they had expected to have out in 2-3 months didn't get out until 8 months down the road. 

In this case I think they would have been better off either shuttering the test servers for a couple weeks a month to get some material together, patch it in, and then bring the servers up for a week or so to let us kick it around. I think that it would have kept people more directly engaged in the process. Conversely they could have waited and built up a larger feature base before opening the doors in the first place, giving us more to play with up front and then pushing things out regularly and fixing issues that came up.

(Additionally I think that they should have had the Alpha phase run longer than it did (Alpha was Feb 1 - March 26 ... so a little under 2 months.) This would have possibly allowed them to bridge the two options that I outlined above by keeping the testing community a little smaller.)

Their other mistake, and in my opinion probably the biggest mistake of the two, is that they haven't wiped data. The result is that people have built up more and more, and become more and more adverse to having that accumulated gear/builds and just the 'stuff' of having played wiped and lost. But the flip side is that as a result, unless people have purposefully done something the 'starting' aspect of the game isn't being tested by most of the testers (yes several, including myself, occasionally go back to using the starter tools, but we still have the accumulated resources and other stuff that we've already gathered, so it's not the same as a NEW player.)

It also means that some of the fixes may or may not be complete. There are areas in the world that are flagged as claimed even though there is no longer a claim there. Trees that can't be harvested. The bugs that caused these things are supposedly fixed, but they still exist in the world so are things being wiped properly now? I can't really tell because I still encounter them occasionally but without keeping a database of where all the ones I find are I can't tell if I'm finding new ones or just ones that I hadn't seen before.

While I understand the desire to let people keep what they earned in game, we aren't here to 'play' we're here to test and make sure things work right. I think that they could have found a balance here with a few regular wipes. Say in the 11 months we've been testing wipe 5 or 6 times that gives us 6 week to 8 weeks of testing before resetting. Again I think it would have kept people a little more engaged and it would have allowed us to reset the worlds and make sure that things that were supposed to be fixed were definitely cleaned up. (Keep in mind that there was a wipe between Alpha and Beta, and a wipe early in Alpha because of a database issue, but there hasn't been any since Closed Beta kicked off in March 2014).

I think that SOE has learned a bit from their mistakes with the Landmark Beta. I suspect (I don't have any direct knowledge of this just hunch and theory) that this is why the H1Z1 early access went from being in 2-6 weeks, to starting about 7 months later as I think that they opted to wait and get more systems into the game and ready for testing before opening the flood gates.

Not everyone gets the system and you'll always end up with those 'testers' that are there to play the game and not there to test and give feedback. So there's still a good bit of 'noise' and people that complain about systems 'not making sense' because the system is only partially implemented, or is in the midst of a revamp. While I think that SOE may be seeing a little more of that than normal because they're getting stuff to the testers earlier, any development method is going to have those issues at some stage.

In the end I'm hoping that some players are paying attention and learning something about the development process involved in these types of games. I'm pretty sure that other developers are taking notice and will be looking to see how Landmark / Next do in the end. If those projects are successful because of player/community involvement in the process then I really think that we will likely see more of this type of development cycle.