Welcome to Jester's Trek.
I'm your host, Jester. I've been an EVE Online player for about six years. One of my four mains is Ripard Teg, pictured at left. Sadly, I've succumbed to "bittervet" disease, but I'm wandering the New Eden landscape (and from time to time, the MMO landscape) in search of a cure.
You can follow along, if you want...

Wednesday, July 20, 2011

Not all curves are pretty

On Failheap yesterday, Trebor from the CSM asked if someone could put together 7-, 30-, 90-, and 180-day rolling averages of the number of EVE players logged into the game based on data that's available on the EVE Offline website. That website's primary purpose is to act as a check-point so you can see -- if you can't log into the game -- whether that problem is just you or the server itself is down. But as a side effect, it tracks the availability of the EVE API by querying for the number of players logged in at any given time... and then as a second side effect, publishes daily averages and maximums.

Liang Nuren managed to download that data and put it into a CSV spreadsheet. On a whim, I took that spreadsheet, collated the data, and generated a graph of the rolling averages that Trebor asked for. Batolemaeus then took my graph and drew lines to indicate when various EVE expansions were released. Here's the final product (click to embiggen):


The raw data behind this analysis is available if you want it, too. You can ignore the places where the dark blue (7-day average) briefly bottoms out. Those represent gaps in the data provided by EVE Offline and don't materially affect the longer-term trend lines. And keep in mind, this data just reflects the number of people logged into the game, not the number of active accounts.

In particular, the big dip that takes place during the Apocrypha expansion, around June 24, represents the "Unholy Rage" implementation: CCP permabanned the accounts of about 6000 botters (logged in 23/7) that day.

The more I looked at this chart, the more it bothered me. I finally took some more time and put together a table that looked at the data in a different way. This time, concentrating on the 30-day rolling averages, I decided to look at three numbers: how much did the average number of players change 30 days after each expansion was released? How about 60 days? And finally, how about taking that metric from the day of that expansion to the release of the next expansion, whatever it was? I figured the first number would reflect people trying out the new expansion, and the second number would represent those same people sticking with the expansion, plus telling their friends "hey, look at this, this is pretty cool." The third number would at least partially reflect the overall success of that expansion: had it drawn in new players to the game, who in turn kept playing?

When I looked at the data in that way, I got this:

ExpansionFeature+30D+60D+NextResult
RMR/Bloodlines  T2s, Carriers-4.6%+7.0%+11.9%Success
Revelations IInvention, rigs  +4.3%-0.1%+17.6%Major success
Revelations IISov changes+0.8%+6.4%-2.6%Failure
TrinityNew graphics+2.4%+29.0%+19.5%Major success
Empyrean AgeFaction war+0.5%+7.1%-8.5%Major failure
Quantum RiseNano nerf+18.5%  +30.5%  +24.7%  Major success
ApocryphaWormholes+11.4%+10.7%+5.7%Success
DominionSov changes+13.6%+13.5%+6.4%Success
TyrannisPI+6.1%+2.6%-0.6%Failure
IncursionIncursions+8.9%+16.8%-5.0%Failure
IncarnaCQ-1.8%?????????

"Feature" is just a one or two word indicator of what that particular expansion is now best known for. By this metric, Trinity is one of the most successful expansions written for EVE, and it's not hard to see why: Trinity made the game much, much prettier. The +29% growth of average players logged in after its release shows the "hey, this is pretty cool" factor at work. The numbers for Quantum Rise are a bit tougher to justify. Three years later, this expansion is mostly known for being the infamous "nano nerf" expansion that cut the speed of every ship in EVE and every mod or rig that was being used to help players speed tank. However, Quantum Rise also introduced the new scram mechanic (scrams shutting down MWDs) which made PvP in EVE a lot more interesting, so that might have had something to do with the big increase in log-ins this expansion brought.  In addition, CCP did a lot more marketing (including TV advertising on SciFi in the U.S.) during this period, which probably brought in a lot of new players.  There were also a couple of fairly major meta news items affecting EVE at that time: Band of Brothers was disbanded in February 2009, for example.  That probably attracted a few more players still.

Apocrypha can also be regarded as very successful. Despite showing only a +5.7% long-term increase in players, Unholy Rage took place in the same period. Normalized for that, Apocrypha would have actually boasted about a +24% increase in logged-in players.

And by these metrics, Empyrean Age, Tyrannis, and Incursion have been failures. Empyrean Age brought in new players to try faction war, but the overall dip in players by the time Quantum Rise was released showed that very few players stuck with the game after trying it. Tyrannis had the same impact for PI. Incursion had the same impact for Incursions (though the number of players in high-sec Incursions apparently recently took a huge jump). The Incursion +60D number is probably an outlier; Incursion also released a lot of skill points from Learning skills back to the players and for a while there were a ton of people playing with the new ships these SP gave them access to.

Expansions like Revelations I and Dominion are in the middle. Dominion is mostly known for its massive changes to the 0.0 sov system (including system upgrades), but it also included a ton of other new features, such as pirate ship buffs, an auto-cannon buff, a super-carrier buff, and pirate faction mission arcs. Revelations I had pretty much no cool factor at all, but the combination of its incremental upgrades and the fact that players at this time were really getting into capital and super-capital warfare (introduced in RMR/Bloodlines, the previous expansion) likely kept players logging in.

But the overall worrisome indicator on this table? By this metric, EVE has not had a truly successful expansion since Dominion, almost two years ago. Which leads to...

One last thing: this chart seems to prove what a stagnant state EVE is in at the moment: other than the +60D peak after Incursion, the 30 day rolling average number of players logged into EVE has not changed from 30,000 in 27 months.

It will be interesting to see what this kind of metric shows for Incarna. We're only about +35D or so into Incarna at this point, and of course it's a long long way to the next winter expansion. However, the curve so far is not pretty.

23 comments:

  1. Interesting metrics indeed, and as a statistics addict I'd like to thank everyone that helped.

    I do hope that CCP catches on to these trends and tries to up their game a bit, but we'll see.

    ~Xeross

    ReplyDelete
  2. Very interesting stuff. I'm glad there are people in the community that have the patience and interest in doing little studies like this.

    It does seem a bit like Eve has reached a plateau. Also, if the 30 day indicator for Incarna is anything like accurate, it would seem to be the biggest failure to date - not only in numbers, but because of the simple fact that Incarna was supposed to attract hordes of players to Eve.

    ReplyDelete
  3. There is something beautiful about how quickly impromptu collaborations such as this one produce interesting results and analysis. I wonder how long it takes CCP to do the same thing--if they are even doing it.

    ReplyDelete
  4. Did not expect such a drope from an expansion that changed the new player experiance so much.
    Retention may have been the problem, but was it of old or new players they failed to retain?

    There has been no real expansion since WH to the actual space ship part of the game. Incursions are nice but not a lot of pilots particpate. Empire and 0.0 day to day life has not been made more intresting in a while.

    ReplyDelete
  5. As a matter of fact, I would expect an expansion specifically aimed at new players to bring in a few new players that actually play, yes. ;-)

    Then again, an expansion specifically aimed at new players should have been coupled with a new marketing push.

    ReplyDelete
  6. What makes you think that CCP is not acutely aware of this, but that in light of their historic preferential focus on "just" sales (acquisition to conversion target) that is exactly what pushes them into the strategic effort of "escaping forward".

    One comment on the side, Incursions and its recent boost is directly related to a limited number of people pushing a trend there. There's been some interesting focus on advertising the use of Incursions between nullsec events among some community groups in EVE. Mittani for example has been a major advocate of Incursions in exposure generation.

    But back to the point of "acquisition to conversion". Last year I made mention of if the pattern at the time of CCP in these matters would not change, we would see them seek and hook up with a major and mainstream advertising partner for designing media and market campaigns. Considering the continuity of the pattern, I hope for CCP's sake that they do not just blindly jump into that, as it would negatively reaffirm a focus on symptoms rather than causes. At quite considerable cost. But we are slowly getting to that point where it is not going to surprise me if it happens.

    An expansion tailored to players, as it is called? Not going to happen, remember CCP's fear of technical debt and their position in limitations of resource allocation...

    ReplyDelete
  7. Thanks for the post Jester - that's a great way to look at it. Glad to be of help.

    ReplyDelete
  8. Excellent analysis, Jester. I agree with Mynxee that the out of eve-o community, aka FHC and co, produce better data and predictions than CCP itself does.

    ReplyDelete
  9. It is rather shocking that CCP went and put the effort into the new player experience, then totally failed to advertise the game to new players.

    I shudder at the "Future Vision" video, and I watched "I was there" with a nostalgic tear in my eye for the days when sniper fleets were actually useful.

    If CCP could just produce a video that dramatised gameplay we were actually capable of delivering to players today, I'd be pushing EVE like it was maple syrup and chunky bacon.

    With the Winter expansion, I'm expecting to see multiplayer Incarna finally arrive. Which means that by Summer next year we'll be able to actually use it, with "stuff to use Incarna for" arriving in the next iteration.

    See what I did there?

    ReplyDelete
  10. Well I`d like to make a few points here. Wall of text incoming.

    First of all, I didn't buy anything off the NeX store. The WoW pets and mounts are enough of a strain on my credit card every 6-12 months or so :)

    Second, when Incarna came out, I was omgwtfbbqluff. I am used in many games to customize the appearance of my toon and 3 years ago when I created mine in Eve I went for the same stuff and was dissappointed to see no female buttocks on my screen after clicking "enter game" ^.^ Therefore it was a little sad.

    Incarna as it is today kindof... err... stinks. As lately I moved from nullsec back to hisec, I`ve been doing a lot of mission running and I`m thoroughly frustrated how my laptop overheats and shuts down if I spend too much time in the station with CQ on.

    On the other hand, the sheer possibilities Incarna brings are staggering. What about sending your enemy that has a jumpclone in your alliance station to the clone vat 50+ jumps away by shooting him in his quarters that are in YOUR station? And that is just an example.

    Third I really enjoy the new weapon turrets. I fly a torpedo boat but this made me train projectiles and lasers :)

    Fourth I managed to get a friend of mine addicted. Yes, A NEW PLAYER, people. So I can say here +1 Drake ;) And this friend of mine I was taunting with Eve for like 1 year. Incarna made him activate his subscription. It's his first MMO and knows almost everything there is to know about Eve mechanics. Yes, you lose your ship permanently when getting blown up. Yes, you can lose your clone and those +4s. Yes, you will rage. Yes, the gameplay stinks sometimes. No, you can't use wasd when in your ship.

    And still he likes it. Beats me why. I'm not complaining because I don't want him to quit and leave me running missions solo (/yawn).

    I strongly believe that all the players AND CCP should pull their collective heads out of their intergalactic nozzles and see the facts:
    - Incarna opens up a ton of possibilities;
    - Incarna does bring over new players and it's not the NeX store (mind you CCP) that did it;
    - the NeX store is not game unbalancing yet; do not anticipate. If they screw it up, rage then. Not before. Seriously, it's one of the most childish acts I`ve ever seen. At least could've given one's stuff away ^.^
    - Incarna should be followed by something really awesome this fall and CCP should bloody get rid of that "Soon©" which is between now and the end of eternity, more towards the latter;
    - gameplay with spaceships still sux; I managed to do a level 2 storyline in a crappy fitted rifter and the dogfight I had to put up with I believe has cut my G700 left click's life in half :lol: Yes, you're a commander. Pweaaaase giev thrust and yaw/pitch/roll controls that are controllable with a flight controller? Or at least revamp the interface. I'm clicking so much lately that I think I'm making a hamster jealous; 2011 aircraft carriers and large boats are controlled by a small joystick. Why can't a ship 215109871087 years in the future be controlled the same way? One of my favourite games of all times is the X series and that is why I started to play Eve in the first place (X + MMO = win). But in X, you can control your ship with a controller...
    - new players are brought in by what Eve is at its core. I can't stress this enough. Space barbies is not what this is, but infinite possibilities of beyond spaceship play [/yoda].

    I strongly believe the interface is one of the most important things to be revamped before the next expansion. The first time I started Eve I was "wth is this, Excel online?". Ofcourse later on I came to the conclusion that it is Excel online, but that is just a part of what makes this game what it is.

    I'm sorry for punching in the ideas in a rather awkward order and I hope my message gets through.

    ReplyDelete
  11. @Cosmin: uhhh... wow. ;-)

    Question, though: have you at least tried Black Prophecy? You should. Link at upper right.

    ReplyDelete
  12. Well WoW is something I've put time and effort and cash into so I'm not ready to give up on it. Plus you can do some really interesting stuff with a bunch of friends and it's all very relaxing.

    The same with Eve. I've come to invest a lot of time and effort and money into it and also feelings :-) I'm not ready to give up on it yet.

    Black Prophecy at the moment I can't try, as I'm abroad and the internet here is really bad. But as soon as I get home in about 2 weeks' time I'll give it a try and see how it's going. But I'm seriously in doubt of all free to play MMOs have anything but shallow stuff to offer :-)

    ReplyDelete
  13. Jester, I don't think this expansion happened to be aimed at new players so much as it had as a collateral effect that CCP could apply to new players. No matter how much they talk about all the walking in stations stuff, what it really is about is an attempt at getting their work in WoD to pay off twice.

    If you look at the analyses done on CCP's financials you will see that they are heavily invested in future projects (read WoD) and their work on that engine has pulled a huge number of their dev teams off of EvE. CCP has to know that they do best with "Spaceships blowing up and building" expansions, but they went ahead with Incarna, which didn't really have all that much for the average player outside of new turret graphics. They want to keep doing "2 a year" expansions with 1/4 of the staff working so they are forced to adapt content and technologies from WoD to EvE to fill the gaps.

    Don't think of Incarna as an EvE online expansion. Think of us as Alpha testers for WoD. Melted hardware and all.

    ReplyDelete
  14. Where on this graph is the sanctum nerf?

    ReplyDelete
  15. The sanctum/haven nerf was done the first week of April 2011, almost exactly where the yellow 90-day curve peaks.

    ReplyDelete
  16. LOL, they removed the datafrom the last 24 hours, week, month, and three mpnths from eve-ofline http://eve-offline.net/?server=tranquility What are they now hiding????? me wonders?

    ReplyDelete
  17. I don't think you can draw the conclusions you do when you claim one release was a success or failure.

    In order to tell if a mechanic is a success or failure you need to consider several things. Like how many players participate in that and does the game offer something similar to that. Is it the main reason people play the game or just a side thing.

    Currently I am showing there are about 17000 characters in FW. How many players are still running incursions?

    Apocrypha also had a apoc 1.5 patch that brought about sized rigs and a small boost to factionwar. This may have had lasting effects into dominion.

    In order to see how well the features work you need to consider how many participate in them over the long term.

    Consider also that after dominion (or tyranus Im not sure) and the insurance nerf there has been less pvp (at least fewer ships exploding) Less ships exploding may in the long run lead to fewer players.

    For those who say incarna opens so much up they are fooling themselves. Incarna will always just be dress up/micro transactions and walking. All of the combat is reserved for dust.

    ReplyDelete
  18. The Eve universe is overcrowded anyway. Many of the features are nearly not accessible.
    Exploration for example: Scanning for 2 hours and maybe find ONE site that has not been cleared already?
    Just boring.
    What we really need is more elbow room. More systems, more rats, more resources, more of everything.
    Then there will be more players.

    ReplyDelete
  19. Hey Jester

    Great blog ... Its past the 60day marker any chance of a update ...

    ReplyDelete
  20. @Anon1314: Thanks! And I'll see what I can do. The last time I checked, though eve-offline.net had "temporarily" taken its downloadable API data offline. I wonder why. ;-)

    ReplyDelete
  21. Jester ..any news yet on those figures ... based on EvE-Offlines graphs the is a rapid drop now visable from mid July...?

    ReplyDelete
  22. See the comment above yours. ;-) I'm looking to see if I can find another source for the data, or perhaps redo this chart with the data that IS available.

    ReplyDelete
  23. 1. you presume that the counter for eve players is actually real and un-tempered with by ccp. i have some doubts about it...
    2. CCP focuses A LOT of their resources on creating other games. The nex store is not just a "cool"feature, is a feature that could bring ccp some money or not, but at least they can say .. woohoo we did something new.. when this something new took very few resources and allowed the rest to focus on creating dust or whatever else they are working on... soon enough we will know if dust & co are great and eve will die and we will go play dust or it just sux and eve will get back to being #1 and treated as such by ccp

    ReplyDelete

Note: Only a member of this blog may post a comment.