On Friday, April 8, the EVE forum debacle occurred. Late on April 11, CCP Sreegs wrote an 11-paragraph devblog on the subject. That devblog included two paragraphs about what the forum exploit could be used to do, plus two paragraphs about what it couldn't be used to do. It then contained several paragraphs about how "cool dudes" would report vulnerabilities that they discovered in CCP code. It contained zero paragraphs describing how such an embarrassing mistake could be made in the first place. It also contained zero paragraphs about the testing methodology that was used that could miss such a simple exploit.
This morning, CCP Rhayger of the web team wrote a 15-paragraph devblog on this same subject. It spends ten of those 15 paragraphs detailing some of the back-story, including why and how YAF was chosen to be the foundation for the new forums. One of the amusing memes on the subject of the new forums was the claim that 30 man-years of development time had been spent "re-skinning YAF". Rhayger tries to dispel this belief, saying "there was an awful lot more work necessary" and spending one of his ten paragraphs listing some of the tasks that were done. The phrase "lot of" is used four times in this one paragraph to describe the amount of work involved. ;-)
However, the new devblog contains zero paragraphs describing how such an embarrassing mistake could be made in the first place. It also contains zero paragraphs about the testing methodology that was used that could miss such a simple exploit. The new devblog didn't even address the very real question EVE players had about why CCP claimed to have built the new forums from scratch when this wasn't the case. It does say this:
I didn't address the significant flaws that made it into release and how that came to be, that will be for another dev blog to detail. We are doing a post mortem right now and doing some serious soul searching. We don't like making mistakes, let alone obvious ones we should have caught at various stages but much more importantly we do not want to repeat mistakes or gloss over flaws in process or skillsets that caused it.Given that the new forums have been down for nearly a month now, and we still haven't been told how these "obvious mistakes" happened, it's fairly easy to argue that "glossing over flaws" is exactly what's happening.
So instead of talking about what isn't in this new devblog, let's talk about what is.
Let's start with feedback. The new devblog uses this word six times. Of the 26 paragraphs in these two devblogs, 11 have to do with giving feedback to CCP. There's only one minor problem here: ignoring player feedback is practically a component part of CCP's DNA. The "CCP commit to excellence" thread in the Assembly Hall, which was the first call from players for CCP to iterate and improve on existing EVE Online features after their release, has been read 100,000 times since its posting last May. It has 2372 "supports"(1) and 2862 replies. None of them are from CCP employees.
During the testing of the new forums, the number of people who said CCP was ignoring player feedback about problems with the new forums was legion, and went all the way up to Estel Arador, a reasonably famous EVE player. CCP didn't respond in that thread, either, except to move it from "EVE General Discussion", where it would be seen, to (I still do not get this) "Out of Pod Experience", where it would not. Not only were these concerns ignored, but CCP Sreegs went on a number of EVE forum threads and claimed that such feedback had never been received. Estel has quit EVE over the experience.
The other problem with this new devblog that really, really bugs me is this:
Now some have worried that by choosing an Open Source solution we have to reveal the source code thus making your accounts and activities in EVE Gate vulnerable. This is not the case here as we have purchased a commercial license to YAF so we can properly protect our efforts. That said, we are big fans of Open Source initiatives and if we note issues in YAF that we come up with improvements for we will communicate that back to their project team to benefit the YAF community.I'm not even a software developer, and I can tell you this is a ridiculous reversal of the tenets of Open Source development. Releasing your source code does not make your code less secure. It makes it more secure. I'll grant you this is counter-intuitive, but it is true nonetheless. In any case, EVE players and non-players alike are going to find holes in CCP's code whether the source code is released or not. And bragging that they will share "issues in YAF that we come up with" with the community is rather hysterical in context.
Keith Neilson aka Mandrill, another well-known EVE player, wrote a rather long post to the EVE forums, Failheap, and evereport.com on this issue called "Loss of Faith". The post is (sorry, dude, but it is) extremely melodramatic and wildly unrealistic. The full piece is probably not worth your time. I debated whether I should link it at all. But it does call CCP out in one key area: leadership. There's a serious break-down in leadership at CCP that is becoming increasingly obvious. Information from the line programmers does not get through middle management to upper management. Information from upper management -- including stuff directly from Hilmar -- does not get through middle management to the line developers.
James Harrison wrote an excellent piece examining some of the underpinnings of CCP's hiring practices that points to some reasons why. Lack of leadership is practically being built into CCP's DNA, too. That piece is worth your time.
Without strong leadership and good internal communication, it's not at all surprising that CCP customer feedback is being ignored, and CCP external communication -- while frequent lately -- is empty.
(1) By the way, this is -- far and away -- the highest number of "supports" ever garnered for an Assembly Hall proposal. And it still is not enough to require this issue to be brought before the CSM according to the official CSM White Paper. This is one of the many amusing holes in the CSM White Paper. It was brought to the CSM (because the proposal was written by a CSM member), but it wasn't required to be.