Scaling in Games - creating and removing demand for multiplayer servers cost efficiently

Summary

Games present some unique challenges and one of these is efficient scaling for multiplayer games (or indeed any games with a server based element).

Let's take a look at why game scaling is hard, and what makes it different to scaling business apps.

Multiplayer Games

To allow multiple gamers to play a game and have a consistent version of the universe requires a single version of the truth; we call this a "game server".

Some (mostly legacy) games do this by having the game server run on one of the player's PCs/consoles.  However this can lead to cheating, at worst, and poor performance for some, at best.

Usually we would hope that these game servers would be hosted in data centres and clouds and managed centrally.  This, largely, stops cheating and provides a consistent performance to all.  However it also means that the game developer now needs to manage fleets of game servers effectively and efficiently.

Scaling?

Scaling is the practise of creating/removing game servers as demand rises/wanes.  

The cost of running the service is directly related to the number of game servers we have running so we need to ensure we have the right amount at all times.

And.. because the speed of light is constant we need to ensure that we have game servers near the players so they get the fastest response times available - making the game flexible and "flow" well.

What's special about Games?

In a usual scaling process we'd use a general rule to increase or decrease capacity.  Either based upon the utilisation of the set of services or an individual server within it.

For example; a simple rule for increasing capacity could be "when the existing servers hit 60% of utilisation then add another server", and when they hit 40% then remove one.  In this instance everybody's performance decreases while the new server is provisioned/created and set to run, but for most business applications this won't materially affect the end-user.

However in games any kind of performance problem immediately has implications for the players and it's almost always negative.

Secondly, for non-gaming workloads we work hard to make the calls "stateless" so any server can process the query/command.  But for games this isn't often true as a game server reflects a set of players playing a single game (or round or mode).  So games are often, very "stateful".

Adding new game servers only works where a new set of players is allocated to it.  Adding a new game server without players doesn't make the player's experience any better at all, while still costing additional money.

So.. how do we scale Games?

Scaling games needs to consider two key priorities: the player's uninterrupted experience and financial management and control.

- Scale Up - Creating Game Servers needs to be done in advance to avoid queueing/creation time for new Game Servers - in this way new games are allocated to Game Servers without waiting around

- Persistence - Game Servers need to persist for (at least) the duration of a single game/round so all players get a good game from it and the universe they play within is coherent, consistent to all players and (most importantly) fun

- Scale Down - To avoid massively increased costs we must remove Game Servers as soon as we possibly can - the easiest way to lose a lot of money is to run servers unnecessarily as the hourly cost per server may be low but the cost of hundreds or thousands can make even the slightest delays an expensive folly

Ideally the resource allocation should mirror the player needs; the closer the two are the better.  However you need to decide if you're a "create over capacity to enable best player experience if there's a peak in demand" or "map closely to the curve and accept that player peaks won't be the best experience" game.

Options

There are two off-the-shelf systems we love:

- Multiplay.com  - we had the privilege of working with these guys on their latest incarnation of the platform and it's an excellent cross cloud + bare-metal solution for almost every game.  

- AWS GameLift - our partnership with Amazon (and our experience in this area) means that we understand the GameLift proposition very well.  If you're going AWS-centric then GameLift is an excellent example

However, if there's custom setup that you need then we've worked with a number of developers/studios to customise off-the-shelf or even build bespoke scaling platforms for their specific needs.

Whatever you need we're pretty sure we can help so drop us a line