Thursday, January 22, 2009

What can we learn from Warhammer population balance stats?

MMORPG companies are pretty notorious for never releasing statistics unless said numbers can be spun in a way that makes their product look good. As Mark Jacobs once put it, if the numbers are good, you release them.

For instance, Blizzard will happily mention that WoW has 11.5 million active players, but will never provide the breakdown of what portion of those players are located in Western markets that actually pay the full subscription fee. Funcom didn't release any subscriber retention figures for Age of Conan at all, they just yanked down more than half of their servers, suggesting that the numbers weren't all that good.

(Alternately, you can do what Turbine does, namely:
1. Release the number of characters created - yes, that number includes alts, characters on accounts that have since been canceled, and may or may not include characters from the FREE open beta
2. Hope some incompetent journalist thinks they can draw a line between that lone, misleading number and the number of unique accounts paying $15 a month
3. Sit back and whistle innocently about how you can't comment on the obviously inaccurate figures being posted in the press because you don't discuss numbers.)

Anyway, Mythic has chosen to release some numbers on faction population balance, a topic I discussed back in the days when the game wasn't out and us blog-types had nothing better to do then speculate on what would happen when it came out. What can we learn from the numbers?

Relative Balance
The closest to a hard number we get is for the server Monolith, described as having "the most noticeable population Faction difference", which is an "accounts active per faction" ratio of 44% order to 56% destruction. (In all fairness, I don't recall Blizzard ever releasing anything even this detailed for Alliance/Horde splits.) A few brief caveats apply. Mythic is defining "active" as "players who are currently gaining Experience and/or Renown Points". Also, players more familiar with WoW server rules should be aware that ALL Warhammer servers are one faction per server, so it's completely unambiguous whether an account is Order, Destruction, or Neither for a given server.

In practical terms, a 44:56 ratio would mean that, if 100 players showed up to a battle, Destruction would outnumber Order by 12 players, or two full 6-player groups. That's not bad for the worst server on the block, though it might still be enough to leave Order on the defensive nigh continuously. Unfortunately, the crucial second piece of the puzzle is the average number of player hours per account.

Let's say that a large number of the Order players are actually alts of Destruction players from other servers, who picked Monolith because the server select screen desperately requested that they do so. Those players might not show up for our hypothetical battle because they're off on their mains. There's also an underlying assumption that player faction choice is independent of player dedication - if more competitive players are choosing Order to get away from the incompetent unwashed masses playing on the Destruction side, for example, the trend could be dead even, as a smaller proportion of the total players who play Destruction might be online at any given time.

If that number looks murky, don't even bother with the reported 49/51 split for "accounts active per server" averaged across all US servers. We don't even know what that number is measuring. Does an account with an Order main on one server and Destruction alts on two other servers get counted once for Order (most /played), once for Destruction (more servers with active characters), once for Order and twice for Destruction (one point for each active server - I'm guessing this is the answer based on the post) or by some other method?

Incentive Balance
The other half of the numbers are a bunch of figures showing that Mythic has managed to balance things so that - both for the average and for the worst case server - the rates of exp/time are dead even. (On Monolith, the average exp/rp per character is slightly higher for Order, which may suggest a smaller number of more dedicated players, but the rate of exp gain over time is still even.)

This is one issue that I was a bit worried about - in WoW PVP, you really don't get much in the way of rewards if your side loses - and it's good to see that Mythic has managed to keep it from becoming a problem. Perhaps the game's much-maligned open-RVR-killing scenarios are saving the day by providing the out-numbered faction a safe haven in which they can continue to level and get all of the various good progress-related things that players of MMORPG's expect to be able to get as they play. Perhaps there's something else at work - between open-RVR influence and other changes that have gone into the game during the time since it's launch, I'm not 100% qualified to say whether Mythic has come up with some other means of helping the out-numbered side keep up. Either way, it's a good number.

What numbers should we see?
In the end, companies don't like to release these numbers because they don't want to reinforce any negative perceptions in the community. In Mythic's case, I'm sure that the churn rates for the game's first two months were not pretty. Reinforcing the idea that the game was not yet ready for prime time with a number that said that a majority of its players had left might have been a bad idea. Likewise, some players won't want to play on low population servers/factions for fear of not being able to find people to play with. Then again, refusal to disclose numbers, be they subscriber numbers or population balance numbers, can paint the devs into a corner.

For example, by many accounts, it sounds like the game desperately needed server merges to get enough players onto the same server to have real RVR battles. Unfortunately, server merges generate bad press by implying to the outside world that the server populations are low. (The inside world already knows that the population is low because they have a battle and no one shows up.) Instead, Mythic had to roll out a system that said "we really should be merging your server, but we don't want to be seen as the ones pulling the trigger, so here's a transfer tool". As I've noted previously, they're allowing inactive accounts to take part in server non-merge transfers, and Stargrace recently resubscribed to Warhammer only to be encouraged to leave her server before she even zoned into the game, presumably because its population was so low that Mythic did not feel it could be saved by encouraging new players to roll on that server.

Likewise, there is much speculation within the Warhammer community that, with many players having finished much of the WoW expansion, and with Mythic's efforts to add much-needed improvements to various parts of the game over the four months since it came out, the time is ripe for players to return. If this improvement to subscriber numbers does materialize, though, no one will know about it, because we don't know what the numbers were before things improved. (Mark Jacobs may actually have come around to this view of thinking, as he implies that EA will be releasing some figures at their next earnings call.)

In the end, numbers are just measures of reality. You might think you're doing yourself a favor by conceling the fact that half your servers are underpopulated, but it comes back to bite you in the rear end when new players blunder into a server, not knowing that it's deserted, don't find the level of community they want, and quit (in the process, missing out on your game). If things are bad, your problem is that things are bad, not the crippling fear that people might FIND OUT that things are bad. In an online world, they probably already know.

P.S. If anyone is looking to release more usage statistics to flesh out a post like the one Mythic put in their Herald post, the number I'd love to see is median hours /played per account (per week or whatever time unit you use). That plus the percentage Mythic did release, would probably have cleared up all the questions I asked. Thanks! ;)


DeftyJames said...

Assuming that what Blizzard says is true, they have publicly stated that all subscriber numbers are for active subscriptions (ie., paying customers.) I am not sure what you mean by that Asia comment since even in the USA what you pay varies on the type of subscription plan. There really is no such thing as a "full subscription fee" unless you mean by that the month to month fee but that is hardly representative of anything. Over Christmas Wal-Mart was offering a deal that gave you 180 days in pre-paid cards for $55. Does that not count? I really don't understand upon what basis you are dissing WoW. Generally speaking, WoW has the best publicly released numbers (of course, they do have something to brag about, a fair point).

Further, I am not sure there is one fair way to measure on-line games outside of raw subscription numbers. The wide variety of ways in which people approach the game makes any more detailed numbers obscure as much as they might reveal.

I do agree, however, with your general point that there really is no reason to hide the data. It's unhelpful to players. For example, I thought about switching to LOTRO pre-wrath but I chose not too simply because I could not figure out how the game was doing subscription wise and didn't want to invest time in a game that was failing. At least with WoW I know that someone is going to be around somewhere; it's better than not knowing anything at all.

Green Armadillo said...

WoW players in much of Asia do no pay for the software itself. Instead, they pay by use of hourly time cards rather than by monthly fees. Blizzard's numbers do account for this (their aggregate number only includes accounts that have used some time in the last month), but the entry barrier for having multiple accounts (e.g. a second account that can bid on auctions posted by your first) is an entirely different creature.

It's also worth noting that, especially in mainland China (which, by many estimates, though the figures are not available publicly, accounts for millions of WoW's subscribers), currency differences mean that Blizzard is not actually collecting very much money in those markets - the Chinese rate is literally pennies on the dollar, and Blizzard makes only a fraction of that on Chinese players because they have franchised a company called The9 to actually own and support the servers over there. So, not all of WoW's subscribers are equally valuable to Blizzard's bottom line.

Finally, WoW continues to open in new markets from time to time, and these numbers get added into the global total.

If you wanted to ask a specific question (e.g. "Has the lack of X content affected US subscriber numbers"), the answer would be masked by the fact that it's mashed in with all the other territories. Blizzard does announce a number, which is more than most companies do, but - as with everyone else - the really interesting data remains behind closed doors.

Daria said...

There is a guy who charts these things, while his numbers can't be 100% accurate they are still pretty interesting.

He estimates that only about 2.5 million of Blizzard's subscribers are in North America.

So when you hear that WoW has 11 million subscribers, it doesn't mean 11 million people in the US are playing the game.

And as Armadillo pointed out, the majority of that number comes from a less profitable Asian market.

And back on the subject of WAR, I also don't understand Mythic's reluctance to release numbers or actually merge servers. Anyone who wants to resubscribe is going to ask the community about which servers are popular and obtain the answer anyway.

DeftyJames said...

Green Armadillo. I understood the facts that you point out but I still don't see how they support your point. In your response you seem to have shifted the argument from players to profitability, which is a different kettle of fish. The players in Asia are paying. As I said, even people in NA pay different rates for their playing time. So the simple fact that people are paying different rates for playing time, while factually true, doesn't really get you where you want to go.

And arguing that players in NA are more profitable is a mind-numbing exercise that I doubt even the accountants at Blizzard bother to do. Sure, overall the subscriptions fee in NA are higher, but so are the costs of doing business. Despite the fact that the subscription fees in Asia are lower it wouldn't shock me if the net profit margins are actually higher there.

"So, not all of WoW's subscribers are equally valuable to Blizzard's bottom line." I don't doubt that this is a factually true statement, my point is that actually determining who those players are is not an easy task. How do you determine "valuable". Profit? In-game contributions? Playing time? You could make a cogent argument that the most valuable players for WoW are the professional PvP players, if by valuable you mean ROI.

At the end of the day I think you want to resolve some of these questions by using statistics, and I just don't think you can do that. "Has the lack of X content affected US subscriber numbers" is a silly question to ask. Even if you could demonstrate a coincidence between those two realties, you still wouldn't have causation. You should know this full well as a biologist. I think Blizzard themselves would be hard pressed to answer that question even with a rigorous regression analysis because of the huge externalities involved.

Green Armadillo said...

I will concede that this discussion has been somewhat diverted into profitability questions. These are of extreme interest to investors (who most certainly do expect that Vivendi find and hire accountants willing to be paid to address the "mind-numbing exercise" of tracking profitability) but limited DIRECT interest to players.

That said, the financial numbers have an indirect effect on players. Your own example of LOTRO is the most extreme case - time spent playing an MMORPG is an investment that earns you a character in a persistent online world, and that investment becomes very risky if you aren't confident that the world in question will be persistent. Players who invested that time in Hellgate: London, Tabula Rasa, or Turbine's own Asheron's Call 2 (on which it pulled the plug 3 months after releasing an expansion) might have been better off rolling yet another WoW alt instead.

Ignoring the case where the game actually folds, the financial numbers can influence the direction of game development. You're correct that a quarterly subscriber number (for example) alone would not be enough to draw conclusions about why that number is changing. Blizzard, on the other hand, has access to more detailed information.

For example, Blizzard actually has logs that will tell them that I used to be saved to various raidID's pre-TBC and that I never made it past Karazhan in the TBC era. If they see a substantial drop in the proportion of players like myself that are participating in raid content AND a drop in subscriber retention in that group, it would not be unreasonable for them to examine raid accessibility.

It's not a perfect measure - perhaps I stopped raiding for some other reason, such as preferring the non-raid schedule - but people don't cancel their subscriptions for literally no reason. If you're seeing a large enough trend - and Blizzard definitely has enough data for this sort of question - it becomes safer to presume that the changes are because of something you did, and less because of external factors (life events, etc).

I would argue that they must have observed something like that after patch 2.1, given how they now call the entry level TBC raids too hard (and commented at Blizzcon that they shouldn't be spending so much time on content that only 5% of players will see with reference to the Sunwell), have greatly expanded the role of non-raid content in the endgame in literally every update to the game from 2.3 forward, and greatly lowered the initial bar for entry into Wrath raids. It's a huge shift in philosophy from everything we'd seen prior to 2.3.

(Perhaps we are going to see the subsequent raids in Wrath tuned at a substantially higher difficulty in the face of current complaints that the content is easy. Then again, with comments that the number of raidID's being used has never been higher, the the current numbers might be good enough that Blizzard will stay the course.)

Finally, to bring this lengthy discussion back to the topic of my original post, take the example of the Warhammer servers. Mythic appears to have concluded that there is a certain critical mass of players - which at least some of their servers do not currently have - is needed to produce a positive experience playing their game. It would not make any sense to suck low population servers dry by actively telling players to leave them on login if Mythic had not reached this conclusion.

In a more transparent world, there would be no harm in simply merging some of the servers - the numbers would clearly show how the game's overall population isn't precipitously low, it's just poorly distributed. EVERYONE would win in that situation (I call the blog "Player Vs Developer", but the devs actually win when they make the players happy), as the devs solve their population balance issues and the players get to have a better experience playing the game for it.

Unfortunately for all concerned, fear that releasing the server population numbers would harm subsciptions keeps Mythic from admitting that what they're doing is closing servers, and, so, everyone suffers. That is how the financials behind the game can affect everyday players; even if the game doesn't fold and the devs don't change the direction of game development, there are always day-to-day management issues where lack of transparency gets in the way.