Torque MMO Kit - Server Architecture

Server Architecture Version 1

This is a diagram of our original server layout. It's simplified in terms of (temporary) connections, the number of zone servers, etc. Though, it should give a reasonable overview.

Here's a short description of the various servers:

Master Server - This server handles registration of players and community operated servers. It handles authentication of "Free Editon" and "Premium Edition" accounts and routes players to worlds. Additionally, it holds a Character Server connection which is used in negotiating the Character Server to the World Server.

Character Server - This is a central repository for characters. Players can use the same characters on any world server. This also allows "Free Edition" players to seamlessly upgrade to a "Premium Account" and not lose any progress.

World Server - This server runs all of the database and RPG logic for a world in a single process. Every zone available to players is connected to this process. Additionally, every player in the world is also connected and executes commands on this server.

Zone Server - a modified Torque Game Engine dedicated server. It communicates closely with the World Server for positional, animation, movement speed, spell effects, etc. It also holds connections to players for simulation in the 3d world. Players are routed by the World Server to the zone servers.

Client - The client is routed from the Master Server to a World Server. Once the player has chosen/created his character, a connection to a destination Zone Server is established and the player "zones into" the world. The player remains connected to the World Server via TCP and the Zone Server via UDP. The World Server connection is used for "RPG" type stuff and the Zone Server connection is used for simulation, selection of objects in the world, etc.

Server Architecture Version 2 (Current)

Here are the changes of note:

Zone Cluster Server - The single world process of the old system has been broken up into distributed Zone Cluster processes. This allows multiple CPU's (and machines) to be utilized and concurrent processing of players, zones, RPG, and database logic. A cluster can be configured to serve an arbitrary number of zones and these can be intelligently chosen. For instance, coupling a busy zone with a zone that gets little traffic is a good strategy. I also added additional checks based on individual player CPU time. This way someone who is macroing expensive operations can't tie up the server, etc.

World Daemon - This is a single point of connection between a World and the Master and Character servers. It coordinates the Zone Clusters that make up a discrete world and routes players between them. It contains logic for chat, and other game elements that must span clusters. The World Daemon is also used as a data replication service for backing up character data. Furthermore, it can be configured to reboot itself at off peak times to help keep the servers running smoothly.

Client - The client now jumps between Zone Cluster servers. This posed a number of issues with features like player alliances. Though, it was mostly smooth to implement.

Summary - We're able to support many more players with our existing servers by distributing the World Server process across multiple processes, CPU's, and even machines. The Zone Cluster architecture is scaleable and critical for adding new zones to the game's existing content. It was also relatively painless to implement these changes to the backend.