The numbers game...part I

Numbers...people like numbers, many people simply love numbers, numbers fulfill a deep rooted need in many of us to quantify, categorize and define things. We like to define things.

In the MMO space people often yearn for numbers and don't get them, and don't believe the numbers that they do receive. Then throw in the fact that in this marketing driven age many of the numbers thrown around are themselves quite often a touch ambiguous.

It is one of the questions people ask me the most - 'Why don't all MMO companies talk about their numbers?'

Personally I would much rather we could all just talk openly about numbers. There is nothing to fear from sales figures or subscription numbers...but...and it is a very significant but...only if it is exactly that, and we all talk about numbers, everyone, and we all used the same definitions. If it isn't something everyone does, then any given set of numbers lack context...and any lack of context allows for some very subjective analysis of what makes 'good' numbers. That is where people, in particular marketing people and such, start to get skittish.

...and I don't blame them...

Unless the context is defined, and clearly defined, there will always be a reluctance to discuss numbers.

You see, the main reason that people generally want to avoid giving numbers, is that more often than not they are taken woefully out of context, or used to attack or detract from a developer, or judged against a different set of numbers, that might themselves have been presented with a completely different set of definitions or in a different context.

That makes giving numbers scary, and in general that is not a good thing for your project unless the number is so undeniably good that it is unlikely to come back to haunt you.

Without standard definitions everything ends up in that vague, murky and entirely subjective middle ground...and a middle ground that very few could reasonably be expected to accept as being worthwhile exploring in a business sense.

Take for example an easy one...and the number you most often hear mentioned in press releases, the number of users who have played a game in any give period...

What is a 'player' exactly? is it just the users that are actually paying you, or does it include all the ones of trials and free play options as well? How many played for more than an hour? What percentage returned more than once? What if they have cancelled but are still in active subscription period? (and what does that mean for those on the various 'lifetime subscriptions' out there on the market?) What about people on time-cards who subscribe month to month but don't have saved credit card info, are they recurring? What does one company include in the definition that another one doesn't? What about the difference between public companies, who have to reveal more about their income, compared to those who are private and don't have to? Does it include multiple accounts rather than unique subscribers?

lots of questions...

...and many different definitions depending on the context.

It is safe to presume that people naturally use the most open number, after all that is the number of the people that have tried your game. It is also vital when we get to discussing the context of success later on...

What we lack in the genre is that standard set of definitions, a set of rules, or even just guidelines, by which consumers could at least get a decent impression of the big picture.

Ratings systems are far from perfect, I don't think anyone honestly believe they are. However most other mediums have a reasonably reliable ones, that at the very least give you a good indication of performance. TV has it's own ratings system (although that is struggling to keep pace with the times too, but it is still somewhat indicative of performance). Cinema has box-office figures, with DVD and Blu-ray sales figures combining to give the complete picture over the commercial lifespan of a movie (although again, the budgets for movies often being at least slightly secretive, muddies the water a little, but again, as long as you know that up front it can accounted for when considering things). Books have bestseller lists. Newspapers and magazines have circulation figures...and non-subscription games have sales figures and a pretty decent count of what shifts boxes.

If a developer makes a single player boxed game, or an online download title, there are generally two decent standards by which they can judge the merit of their work in reviews scores (which of course can be subjective and don't guarantee success) and box sales, the number of units shifted...the issue with MMOs is that the number of units sold only marks the beginning of a very long term lifespan.

So what we lack are a set of standards by which we could all be judged. There is a lack of transparency that means that even if any one company wanted to give numbers, it would probably do itself a disservice (unless it was the market leader) as without context you have little way of knowing whether that makes it a success or not...

You can't really argue you were a success if your XX million dollar single player game only sells 10,000 units, and you also can't be denied if it ships several million...but with an MMO there is a vacuum of standards as to how the games are judged after release.

Personally I would love to see some kind of industry body established to monitor and agree on standards for online subscription or user levels.

So a ratings system of some kind would be a great first step to establishing that all important context so that we can be sure we are comparing the same things. I don't think an MMO needs to have ten million, one million, half a million, a hundred thousand, ten thousand, or less, in order to be a success as different games can have different budgets, goals and ambitions, but until we have some agreed way of assessing that it makes judging success or failure a very difficult proposition.

Alas, I don't think it will happen, at least not in the short term. Right now there doesn't seem to be a significant desire amongst developers to be judged in a formal or official capacity...it is a natural reluctance, but personally I believe that in the long term some kind of ratings or subscription definitions that are official enough to be meaningful (there will always be some 'shades of grey' hidden away in any such system as discussed above) would be beneficial for the genre.






Addendum: Of course it is important to remember that numbers in and of themselves aren't the only part of defining a success (although they do of course define the viability of your projects, in particular in an online space), in so far as there are quite a few games down the years that are not massive bestsellers, but are critically acclaimed and beloved by many. Likewise an MMO title can succeed with a limited or small audience...depending on how you define 'success'. I have talked about that before in terms of scale of ambition, but this touches more on that all important definition...

...the definitions of 'success' for games in this genre vary so widely that it is another matter entirely...and the subject for the next post. I started writing more as part of this post, but I think this one is long enough as it is...so next up I'll be musing on the different definitions of 'success' being thrown around for MMOs and the impact, good and bad, that has on the genre...

Comments

Kendrick said…
In my own humble opinion, the only people that need to be concerned with numbers are investors/shareholders and the studio itself. As long as those groups are happy, the rest don't matter.
Anonymous said…
Well "success" does have a direct impact ingame.
Do I struggle to find enough people to team-up with? , to do some PVP activities?...
Success is not also defined by the number of max level players in game, but also the attractivness of the game to newcommers so lower level content can still be enjoyed by rerolls & new players.
Unknown said…
Ah the numbers game. People surely do like to speculate about how many people are playing any given game. I think that the root of the issue is that they know that a company needs to be financially viable for their favorite game to keep running (let alone moving forward). I think that the guessing of the number may come from a fear that this game that they are invested in will continue or not.

Like you say, though, the context is part of the problem. Even if there was a standard measure for subscriptions/players, people still wouldn't understand the meaning behind the number.

A given game may say that it has 10,000 active monthly subscriptions. If the game costs $15 a month, that's $150,000 a month. Now the company behind the game may know that they can support the current servers, a team of developers and support staff and still make their profit margin that they want to have as a company with those numbers.

But the players don't understand that. They see "10,000 players! OMG! That's NOTHING!" (of course they're comparing it to WoW or a Facebook social game). They don't understand that the business side of game has already figured out that they can operate at those levels indefinitely.

Then the fear on the company side also kicks in. If people *think* that there is a problem with the business, they may speculate that the game is shutting down soon, and decide to "get while the getting is good". If subs drop too much, then there is a real financial problem caused by players imaginations.

I, for one, am not as concerned with the raw numbers of people paying for the game. Tell me the business is solid and that the team is still working on enhancements and bug fixes, and I'll stick around.

The only time I think that numbers matter is in server populations. I know that deciding to merge servers isn't trivial, but when individual server populations drop to the point where a player can't find team mates for quests/raids or opponents for PvP, then the number of players truly is an issue which is impacting the players.
Craig Morrison said…
@Karl - definitely part of the issue, and something I'll elaborate on in the next post...

...because while you are right @Kendrick, it doesn't matter to players individually (you shouldn't make a decision on anything other than whether YOU like a game or not) it does have an impact on what games you might see in the future, as making good business cases is much harder for newer developers, again something for the next post :p
Niko said…
For me at least, as player, the most important part of the numbers is the population level logging for the server. The level needed varies a lot depending on the virtual world's layout and desired activities. In some game world amount of people logged in might be much larger than the next one, but if the players are spread out much sparsely, it will appear as less. Also, if amount of population interested in activity I want to do (be it PvE, PvP, RP...) is so low that activity is hard, unsatisfying or impossible, then it does not matter to me personally if the server is bursting with people.

As an example, having played NWN in the past, some persistent worlds had quite low populations, but they felt alive because the way they were constructed to guide players so they would meet each other... without feeling railroaded. Also people playing in the server were likeminded enough that with small population we had still crowd that felt large. And, I have felt utterly alone in large MMO server. So... effect of the numbers do vary from situation to another.
Krystal said…
It would be good for us as consumers to see who is actually successful, stable, small, medium or large. Sometimes I feel I actually want to find a smaller but stable MMO, and it can be hard to establish which ones are. A system like you describe might at least help with that.

I seriously doubt it would silence the trolls / fanboys however. Even with real numbers they would still be there saying the game was dying / thriving based only on their own point of view :p
Livestring said…
Personally I don't care as long as I enjoy the game in question BUT I do think it would be a good think to add transparency and prevent the only numbers we ever hear being those pseudo-fictional creations of a marketing mind.
Furiousbeard said…
Numbers do help a player population guage what is going on around them and also provides a sense of "life" to your game. With the recent expansion in AOC, the game has become a lot larger and now "feels" more empty to a lot of people.

Since most publicly traded companies don't talk numbers, perhaps a more visual approach to it would be better. Perhaps a graph that acts like a pulse to a server not releasing numbers but at least showing "hot" times to be on.

Numbers can tend to be a more "in house" success factor, and the numbers your paying customers expect may be different from what you expect. Since you can't share your numbers with the public, they will make assumptions based on what they perceive through feeling. For example mini-games and other activities that prove to them that other people are online.

Give a community a sense of community and numbers will take care of themselves, even if you can't tell anybody what they are.
eps server said…
I agree. Numbers play a great role in our lives that is why everybody loves numbers. Number games seem to be exciting especially if the server is good.
Anonymous said…
Craig is lost in space. Success is defined in the business world by growth, not the death of revenue and sales. AOC has been slowly and consistently dying since the drastic and sharp drop-off they shot themselves in the face with when they release AOC in the state they did when they did.

Since then they've tried everything, shills, marketing, free play, EVERYTHING to steady, grow, or at least stabilze their playerbase and every quarter it is consistently shrinking with more and more players disinterested in what they offer.

Players come and go is what he says and yes they do, but when they return to AOC they leave again much faster much sooner than in other MMOs with a quality and content level that is better.