Part II: Design and Development Considerations
Chapter 7. Digging Deeper into Development and Design Issues
KEY TOPICS
Technical Considerations Where to Start?
Building the Right Tools Host Hardware and Bandwidth Player Hardware and Software
Customer Support: Dude, Where's My Tools?
After some six months, plenty of skull sweat, and arguing with stakeholders about the necessity of this or that feature, the team is finally ready to settle in and begin building the game. At this point, the routine should be fairly easy because the design process has laid out a guide map of what needs to be built when and how many people it will take at each stage.
Naturally, there will be problems; nothing in this industry ever goes exactly according to plan, and changes will have to be made along the way. With the completed game and technical designs to guide you, however, these dislocations should be solved easily so that development can proceed.
As you've probably noticed by now, everything about designing one of these games is a tradeoff between creative design and the plain technical capability to pull off that design. We'll start with the technical considerations, work our way into design issues, and finish up with the testing process and preparing for the launch.
This chapter is not designed to be an exact roadmap of how design and development work. Experienced online game developers will sneer at most of what you'll read as basic and, in truth, it is. As we note elsewhere, this chapter alone deserves its own 350-page book, so it is impossible to be complete and definitive in the space allotted. This chapter, at first glance, may look woefully incomplete to those in-the-know. Thankfully, three of the most experienced people in the industry, Gordon Walton, executive producer for The Sims Online, Dr. Richard Bartle, co-creator of the first MUD, and Scott Hartsman, technical director for Verant's EverQuest, agreed to look over the book and point out any mistakes we'd made. Any errors remaining in this chapter belong to the authors, not the technical reviewers.
A Note on the Buzzword "Community"
Throughout the remainder of the book, you'll be seeing the word "community" quite a bit. There is a misconception in the online game industry and, indeed, in the online/Internet services industry as a whole, that a product or piece of content builds a community around it. Nothing could be further from the truth; the community already exists and may gather at your game, if you provide the proper tools.
This may seem like nitpicking, but understanding the difference is crucial to success in the online game industry. Many initial game portal efforts, such as TEN (now pogo.com) and Mplayer, failed by assuming they could create a community by just slapping together a portal and some interface software. This was the "If we build it, they will come" mentality.
What they discovered, after throwing nearly $100 million collectively down a rat hole, was that they should have been
building tools to facilitate the needs of existing communities—the community center, if you will. A site must build a community center with a suite of tools that is easy to learn and use, and then make sure the customers know they exist.
When the customers know that the tools and center exist, whole communities of gamers—be they guilds, teams, or squads from other games or one of the broad market niches—will migrate to see if your community center is easy to learn and use and provides the communications and information tools they require to maintain their existing communities. If it does, and some of the community leaders pick up on them and use them, the community begins to migrate; if it doesn't, they migrate to the best alternative.
Among the current portals and individual online games, Microsoft's Zone (or The Zone, found at www.zone.com) is one of the better examples of providing the necessary tools to facilitate a community. The Zone should be closely studied with an eye toward improving the tools and adding new ones, such as web page hosting for teams and role-playing guilds.
One of the less stellar examples is Ultima Online (UO); the chat tools built into the game at launch were so bad that the vast majority of regular players used AOL's ICQ instant messaging software to communicate during play instead.
What we really hope to accomplish is something that has yet to be done, and that is just list the general, minimum requirements for getting through design and development, show where the quicksand pits are, and give some advice on how to avoid the problems others have experienced.
[ Team LiB ]
In this day and age, when most programmers recognize the need to comment their code so that those coming after them know why any particular piece of it was written and what it is supposed to do, you'll find many game developers who don't comment their code at all. This is even more prevalent in the online game industry; there are hideously complex PWs in operation today without a single comment anywhere in the source code. There is a school of thought in the game industry, in fact, that says commenting code is just a time-waster, for sloppy programmers who can't write elegant code that is easily understood. This ignores the fact that most programmers can't agree on what "elegant" code is, much less write an example of it.
It is insanity of the first order not to insist on comments. Ours is one of the most mobile areas of the job market. People change jobs and move to new companies every couple of years, on average. Much of this has to do with the fact that publishers are constantly laying off employees at the end of the year, scheduling many projects to finish just in time to ship for the Christmas selling season. They tend to staff up again for new projects in the spring, and one has to wonder if the reason why so many projects are late to ship is an
unconscious (or not) attempt by some programmers to extend the paycheck cycle a bit.
Whatever the reason, it is a safe bet that not every programmer who starts a project will finish it. If that programmer's code is not commented, whoever the replacement is could spend a considerable length of time figuring out the code and its effects on other pieces of code. In projects that can easily exceed 800,000 lines of code, knowing how the pieces interact with each other is a requirement, not a luxury. There is nothing worse than having to replace a critical coder at a touchy stage of development and then finding out it will take three to six months for the replacement to fully understand the legacy code because it is uncommented.
So, a cardinal rule for the team must be comment the code. It is up to the team leaders to establish the standards (Comment algorithms, data, or both? Comment by line or by blocks?) and follow up on them by checking the code and ensuring that the team is, indeed, commenting the code.
[ Team LiB ]
[ Team LiB ]
Where to Start?
In many ways, choosing technology for an online game starts at the same place as a retail home unit: with the basic libraries and commercial development tools available on the market today. Since you'll be working with both a client and a server complex, there are some interesting additions:
Base (data structures and utilities) Database
Encryption
Input/Output (I/O) and sockets
Network messaging (TCP or UDP, with UDP preferable in most cases) 2D/3D sound
2D/3D graphics
Note that the encryption, I/O and sockets, and network messaging are going to require a different skill set and core competencies than most game development teams normally feature.
What Database, Database Structure, Client View, and Language Will You Use?
Let's approach these topics one at a time.
Database and Data Structures
If you aren't planning to have your online game be database-driven, you're already about to make a huge error. The easiest way to add content over time is to make sure the database and structures support easy, on-the-fly additions of new objects, classes, and art.
Objects and Content Addition
Online games must be refreshed with new content constantly to keep the game alive and exciting for the players. The easiest and most convenient method for this is simply downloading the new content onto the player's computer via the patching process.
If you are going to sanely add content to your game over time, using a 3D client, objects, and a flexible database on the backend are pretty much necessities. Take, for example, UO's situation from launch in 1997 until late 2000/early 2001, when UO: Third Dawn, the 3D upgrade, was shipped. Until Third Dawn, UO's structure for the art and graphics files used what can only be called the "monoblock"
approach: one huge, inherent file that was several hundred megabytes in size. This meant that to add even one piece of new content, the
whole file had to be reinstalled, thus producing an impossible patch size for any player. If the live team wanted to upgrade or add content on the client side, it had to convince EA to ship an expansion pack; this hindered content addition for years, until the 3D upgrade was mandated by then Online Services VP Gordon Walton and ramrodded by live team producer and experienced 3D hand Rick Hall.
In most shops, unless the developers are building a database from scratch, a commercial database product such as MS SQL Server, MySQL Server, or Oracle is the main product in use. Either option (build or buy) has its advantages, but in general, it makes the most sense for a team new to online game development to pick a commercially available product; there are more people out there who know MS SQL Server or Oracle than your home-grown alternative.
C/C++, Object-Oriented Language
Is there any PC development group not using C/C++ these days? Most of the shops we queried were using Microsoft C++ and objects.
The general consensus seems to be that the use of objects gives far more flexibility in design and makes online games easier to manipulate and change after launch.
The Client
One of the key choices the team will need to make at the outset is whether to go 3D, first person with the interface, like EverQuest, or isolinear, like Lineage: The Bloodpledge. There are a number of considerations that will go into the decision.
3D First/Third Person Versus Isolinear 3D or Flat 2D ("God's Eye View")
Both interfaces are in use today, but there are more 3D first/third-person online games than isolinear, especially in the PW niche. The 3D first/third-person solution is most popular with developers in the US and Europe (EverQuest, Asheron's Call, and Dark Age of Camelot); the isolinear interface style is very popular in Asia (Lineage and numerous imitators). See Figures 7.1 and 7.2 for an example of each view.
Figure 7.1. A 3D, first-person interface, from Funcom's Anarchy Online. Most first-person interfaces allow the player to set the camera view to third person (which means the player can
be viewed on screen).
"raked position," which means the view is not from directly overhead, but from approximately ẵ to ắ of the way from ground level to directly overhead.
Almost every developer working with IBM-compatible PCs today (about 95% of the total market) is working with DirectX, or at least some portion of it. For the uninitiated, DirectX is an applications programming interface (API) that allows programmers to access PC hardware directly in Windows without having to write hardware-specific code. It is somewhat loosely based on the company's component object model (COM), the same technology that drives ActiveX controls. The API includes various modules for dealing with discrete PC functions, such as DirectGraphics (version 8 combined Direct3D and DirectDraw), DirectShow, DirectSound, DirectMusic, DirectInput, and
DirectPlay.
While there have been complaints about the clunky, bloated code, numerous updates to learn (eight major version marks in fewer than seven years), and lack of OpenGL support, in general, the API has been well accepted.
Pretty Pictures, Polygon Counts, and Graphic Accelerators: Your Living Hell Has Arrived
Graphic accelerators have become standard issue with new PCs, and there is no doubt they have allowed the creation of ever more realistic and beautiful graphics in all computer games. They have also greatly increased production and development costs because players expect developers to take advantage of the ever-rising on-screen polygon counts, frame display rates, finer textures and meshes, and every bell and whistle imaginable to make a game as visually stunning as possible.
There are two main problems here:
In an online game, the more graphic data you have to load, the more latency the player is likely to experience and the more complaints about lag your customer support (CS) staff will receive. For example, at the time of this writing, Anarchy Online (AO) was probably the most visually stunning online game currently on the market. However, when a player zones into a city area, even the most buffed-out PC can take as long as a minute to finally finish loading all those stunning textures and 3D objects—and these are all static objects, built into the terrain. One can imagine the delays if they were dynamic objects that the server had to first identify for the player, then load as he/she came in range.
While most accelerators use the same two or three basic chipsets, mainly being NVidia's Quadro and GeForce and ATI's Radeon, accelerator board makers such as Matrox, ATI, and NVidia have different standards, configurations, and sometimes wildly different software drivers. Toss in Microsoft's DirectX API, which allows developers to access hardware directly through Windows, and the mix can be even more volatile. Since it is virtually impossible for the manufacturers or game developers to test the graphics chipsets and their software drivers with even a significant number of PC hardware and software
configurations, there are always going to be conflicts that cause some games and applications to react weirdly or not work at all. The fact that the drivers included with new PCs are probably six months out of date by the time a PC is sold to the consumer (one or more updates have already been issued) also increases the potential for software conflicts.
During the later test phases, accelerator conflicts are likely to become one of the banes of your existence, so be prepared for it.
After launch, conflicts are generally easily fixed by downloading an updated software driver for the chipset. In the meantime, however, it also probably means that your player relations staff will be bombarded with complaints and demands that the problem be fixed.
[ Team LiB ]
[ Team LiB ]
Building the Right Tools
"You will need tools for everything! Hire lots of tool and library coders. Use money to develop brilliant tools. Make decent and efficient GUIs. And force the coders to work with their own tools before they ask anyone else to work with them."
—Thomas Howalt, Funcom
The tools and their capabilities should be fully specified in the game and technical design documents. This is another case where the lead designer and his/her team and the technology builders must work closely together to ensure that nothing escapes notice during the design phase; after all, your designers will be doing quite a bit of the scripting of the game mechanics. It can be hell to realize during development that you forgot to specify the scripting capability to modify a weapon's effectiveness by a character's inventory weight load and then have to try to retool the scripting language to match.
Where most teams get into trouble is in not building the tools to handle those minimal capabilities right at the outset of the project.
World-Building Tools and Editors
The world-builders/editors are probably the most important tools you'll build first. These are what your designers and scripters (collectively called "world-builders") will use to actually construct the world that your players will romp in.
These aren't just graphics tools to lay out terrain and buildings; they comprise a suite of tools to build and edit items, spells, quests, and non-player characters (NPCs), plus a sound editor for sound effects and music.
These tools are no secret in the industry, and their capabilities are fairly well-known. Some of those capabilities are pretty obvious and include the following:
Laying out ground terrain, such as forests, lakes, deserts, and mountains
Basically manipulating and deforming terrain, such as raising mountains, making lakes and rivers, and so forth Placing man-made terrain features, such as buildings and roads
Placing triggers, such as NPC spawns and weather effects Creating and editing players/characters
Creating and editing player-usable objects, such as weapons, armor, books, food, gold, and so forth
Bear in mind that these are the minimal capabilities your team will need going forward with the project.
Scripting Tools
What will the designers use to build the world? What capabilities will they need? Most, if not all, PWs support a scripting language. This is mainly used by the designers and world-builders (in conjunction with a home-brewed graphic-oriented tool) to actually construct the
game world and mechanics. The whole purpose of using a scripting language is so you don't have to hard-code every mechanic, NPC conversation, or event into your game. It also avoids the need to recompile the executable for the game each time you make a change.
Note that this is not the same scripting you may be familiar with in hybrid games like Quake; it can be far more detailed and time consuming and require a language and tool that are far more flexible. The old joke about Ginger Rogers being a better dancer than Fred Astaire (she did everything he did, backward and in high heels) applies here. Bear in mind that an online role-playing game has on the order of 10 times the actual content and complexity of the average hybrid, in terms of player-controlled objects, terrain, and the way inventory objects correlate with a player's skills and attributes. If each change in a PW had to be hard-coded into the executable, none would ever be finished enough to launch, and not many changes and content/feature additions would ever be done.
Python
There are many scripting languages available that work with C/C++, including PERL, JavaScript, Visual Basic, and SmallTalk.
However, many shops are now using Python: an interpreted, interactive, object-oriented programming language that is extensible in C or C++ and portable, meaning it works with several brands of UNIX on the Macintosh and on PCs under MS-DOS, Windows, Windows NT, and OS/2. It is also free and has virtually zero compiling time. See www.python.org for details.
Other Tools You'll Need
The other tools include those you'd need in any project with multiple builds and versions:
Code building Version control Task tracking
In addition, you'll need a set of tools that just about every online game has neglected to design and build during development and was forced to build at or after launch. These are tools that monitor and log into record files what goes on in the game, including the following:
Player activities:
Where do they go?
What do they do?
What do they look at?
What do they buy, where, and how often?
How long does it take them to do it?
Client and server statistics:
Who is where, when?
What machines in the server cluster are empty for significant portions of the day?
Which machines are full for significant portions of the day?
Chat logs
Crashes and errors