by Matt Firor, Producer, Mythic Entertainment KEY TOPICS
The Community The Beta Starts
Server Backend Configuration The Business Arrangement Lessons Learned
AUTHOR NOTE
Anarchy Online (AO) was a poor launch that made a turnaround; Dark Age of Camelot is that other elusive beast in the industry of persistent worlds (PWs): the flat-out success from day one. Dark Age of Camelot had the smoothest launch of any PW in recent memory and has since seen unfettered growth. In the eight months from its launch in October 2001 until May 2002, the game acquired 200,000 subscribers. If Mythic can sustain that pace, it will have 300,000 subscribers by Dark Age of Camelot's first anniversary, taking the crown from EverQuest (EQ) and making it the fastest-growing PW in US history.
This success should really come as no surprise; Mythic has been around for over 18 years and has developed a number of PWs in that time. That's why we asked producer Matt Firor to write a post-mortem for this book, noting what went right and what went wrong. As you'll see, experience may be a key to a smooth launch and management, but there are shoals to navigate even so.
When Dark Age of Camelot hit the market on October 9, 2001, it was the culmination of about 18 months of development. About 25 developers worked on the game in that time, evenly divided among programmers, artists, and world developers.
The genesis of Dark Age of Camelot came about when we at Mythic saw the immense popularity of the new generation of online RPGs, such as EQ, Ultima Online (UO), and Asheron's Call (AC), and realized that we could do that type of game. In the past, we'd created many text-based MUDs as well as graphical action games. It made perfect sense to us to combine the two technologies and make a graphical MUD.
But which MUD to make graphical? We'd created many different types of RPGs over the years, from standard player versus monster games to games where players were actively encouraged to fight one another. At that time, the most recent online RPG developed by Mythic was Darkness Falls: The Crusade. This text-based MUD featured realm-based combat, where players chose to be one of three different teams. Each team (realm) was in conflict with one another and was in a state of constant warfare, with the goal being to steal the other realm's idols. We liked the dynamic that realm versus realm (RvR) combat gave the game because it gave a sense of purpose to PvP combat that was missing in other PvP-centric games. So, we decided to base our new graphical online RPG on three-team,
realm-based RvR conflict.
This decision was not taken lightly; we had many discussions where we brainstormed about what players wanted to play. EQ, the most popular graphical online RPG to date, did not even really offer PvP combat in any meaningful sense. Many players touted that feeling of safety as one of the primary reasons why they played EQ, instead of other, more PvP-oriented games such as UO. We wanted our RvR combat to be compelling enough to attract players who wanted to play such a game, but we also didn't want to alienate potential players who were looking for a more sedate game style. Hence, we hit upon the plan of having safe zones where players could play the game solely against monsters—where they could group up with other online players in a cooperatively, friendly fashion and adventure together.
Because the game would have three realms, we needed to have three different home areas—one for each realm. However, there would also be a frontier area that was RvR-oriented; players would make the choice of whether they wanted to fight other players or not by traveling (or not traveling) to the frontier areas.
Darkness Falls: The Crusade was developed around a good versus evil system where the three realms were Evil, Light, and Chaos.
This was compelling enough for a game of its type, and we started designing the game and working on the game's graphical engine.
Soon after development started, Mythic's president, Mark Jacobs, hit up on the idea of making the game based on the legends of King Arthur. Having a recognizable background to the game would go far toward making the game more appealing to players. He decided the game should be situated after the death of King Arthur, in the times not really covered by the legends, and that the game should be appropriately entitled "Dark Age of Camelot." The only task left was to add the other two realms, and soon Viking Scandinavia (Midgard) and Celtic Ireland (Hibernia) were added to Arthurian England (Albion).
At the time, we were far ahead of the technological curve. We had already created 13 online games by that point—most of which built up on the code base of the previous games. In essence, we had been testing our technology—by having it in our older games—for five years by the time Camelot's development started.
In general, Mythic's pre-Camelot games were separated into two distinct categories: text-based MUDs such as Dragon's Gate and Darkness Falls: The Crusade, and first- (and sometimes third-) person shooters like Rolemaster: Magestorm, Aliens Online, Godzilla Online, and Spellbinder: the Nexus Conflict. As designed, Camelot would fuse the two technologies together—use the graphical frontend from the shooters and the server backend from the MUDs.
The first versions of our shooters were software only; most were created before 3D-accelerated video cards became prevalent.
However, our most recent—Spellbinder— was our first 3D-accelerated game. It was built using the Netimmerse 3D API from NDL, Inc.
Because of this, we had a great head start on developing Camelot's client; we had a stable base to start from, which already supported our messaging and art requirements.
Spellbinder's engine, while accelerated, still needed some major work in order to bring it up to the level that today's online gamer demands. First, we had to add the concept of "outdoors" to the engine—Spellbinder's gameplay took place exclusively inside castles or caves. We also needed to add rolling, organic-looking terrain, which again did not exist in Spellbinder. Rob Denton, Mythic's primary programmer, made it his first task to upgrade the engine.
While Rob was hard at work on the client, Brian Axelson, the main programmer on Darkness Falls: The Crusade, was working on Camelot's server, ensuring that the database, editor, and in-game tools from Darkness Falls: The Crusade would work when accessed from the new graphical client. Eventually, both would be done and would work exceptionally well. With some other code integrated from Spellbinder (lobby, login, player authentication), we were ready to start content development on the game.
I cannot overstate the importance of this fusion of existing technology on Camelot's timely and stable development. Basically all parts of the game's underlying systems had been tested for years by their inclusion in other Mythic games. All artists and programmers were familiar with the tools needed to create content and new features. Once new versions started rolling out for testing, they were stable, and we could test game balance and other features almost immediately without having to worry about stability.
We assigned co-lead artists Lance Robertson and CJ Grebb on the game. Lance was assigned management and technical art duties, and CJ was in charge of the game's look and feel. Lance and Rob worked out a scheme for creating player and monster models, animating them and putting armor/outfit textures on them. Their team then geared up to create figures and outfits, as well as other in-game models such as houses, trees, rocks, and the myriad of other landscape features that are in the game.
Soon after getting the initial versions of the game running, our lead world developer, Colin Hicks, started assembling a team that would place the art provided by the artists into the game. His team was responsible for quest development, terrain creation and object placement, and the general background/stories that defined the player's interaction with the game.
I was responsible for class and race creation, which was made interesting by the fact that we had three separate realms, each of which had separate races and classes. We didn't want to make the classes the same across all realms, so eventually I (with a lot of input and help from everyone on the team) came up with 32 different classes and 12 races across the three realms, none of which were exact
copies.
[ Team LiB ]
The Community
Soon after Camelot's development started, we got serious about developing a fan base for the game. We enlisted the aid of the Vault Network, who provided us with some message board space and a moderator. This gave us an outlet to the Internet community—we could post ideas and updates on the game, as well as entice potential customers with screenshots.
The mood on the fan boards was initially quite dubious—no one had really ever heard of Mythic before, and few thought we could create Camelot at all. Many other online RPGs had been announced, and most never saw the light of day. The gaming community was frustrated by this and viewed any announcement of a new title with extreme skepticism.
However, after the game started Beta testing and word got out that the game was actually functional, the boards turned more and more into a place for players to express their hope that Camelot would turn out well and for them to give us their ideas and comments on the game design. Eventually we hired an Internet relations manager, Sanya Thomas, who was put in charge of managing the community and for keeping all of us developers appraised of what the community was thinking.
[ Team LiB ]
The Beta Starts
After about six months in development, it was time to start testing the game. We identified the fans we thought would make the best testers and invited them into Camelot's first Beta test. The test would eventually have four stages and would run for almost a year; the first versions tested had only one realm and a limited number of classes, but as time went on, the game got more and more content.
The Beta test was very important to Camelot's maturation; it allowed players to tell us what was going on in the game, how the different classes performed, and how the world was designed and set up. We made many changes and tweaks to all aspects of the game during the Beta period; most were made because of tester feedback.
[ Team LiB ]
[ Team LiB ]
Server Backend Configuration
From our experience with our other online games, we had a rough idea of how many servers we'd need and how much bandwidth would be required for Camelot. We standardized on Linux as our server operating system, as it was our standard server operating system for other games. Linux was rock-solid and we had lots of in-house expertise with it. Because Linux runs so well on Intel-based boxes, we shopped around and settled on Pentium 4 dual-processor Dell boxes. We contacted several PC manufacturers, but Dell clearly rose above the others in terms of understanding what we were trying to do, as well as providing a great price/service combination.
We had planned on using Oracle or Microsoft SQL Server as the character/account database for Camelot. I had a lot of experience developing and maintaining Oracle applications from my previous career in the IT industry, so I contacted Sales departments of both companies and solicited bids. Oracle came back with the ridiculous price of about $975,000, while SQL Server, at $30,000, was more reasonable. It was still far outside our price range. So, we instead settled on a hybrid of MySQL—a freeware Linux database—and an in-house-developed flat-file character database. This solution cost us only the development time it took to create the databases; no licensing fees were required. MySQL manages all account and customer service (CS) records, and the flat file handles all character information. This combination has worked essentially without a hitch since launch.
[ Team LiB ]
The Business Arrangement
Almost as soon as Camelot started to be developed, we began to try to interest other companies in publishing it for us. The traditional relationship between developer and publisher is that the developer handles all design and implementation of the game, while the publisher handles business arrangements; distribution, advertising, marketing, and so on. Typically the publisher finances the game and then takes a cut of all profits. For the first year that Camelot was in development, we simply could not interest a company in publishing it, which left us in a financial bind; we couldn't afford to develop Camelot.
We then turned to the only source of revenue available to us: We sold a minority share of Mythic's stock to Abandon Entertainment, a New York-based entertainment company. With this sale of stock, we financed Camelot's entire development, as well as some of its marketing and advertising. It also paid for a booth at E3.
By the time E3 rolled around, we had two of the three realms done and were well on our way to hashing out the RvR combat system. In fact, our primary focus at the show was to demonstrate our RvR system. By that time—five months before release—it was obvious that the game was going to be done on time and that it was good. Publisher interest was heating up, but by that time, we didn't need one—we had the game financed and produced. What we needed was a distributor, so we used E3 to set up meetings with many large game companies interested in distributing Camelot for us.
One such company, Vivendi Universal, impressed us more than all the others, and we signed up with them to promote and distribute Camelot. Vivendi has a great system of regional sales; through their sales chain, Camelot has been distributed to just about any retail outlet that wants to sell it. Vivendi has been a stable, reliable, and overall a great partner for Mythic.
[ Team LiB ]
[ Team LiB ]
Lessons Learned
It wasn't all roses, of course, during the time that Camelot was being created. We ran into problems just like everyone else who makes an incredibly complex game like Dark Age of Camelot. Here's a sample:
We launched without nearly enough in-game and support tools for our CS department to be able to do their jobs effectively. It took almost a month before we got them the tools they needed in order to answer questions, figure out who was having problems, be able to see logs, and a myriad of other small utilities. Not having tools of this type made players wait far too long for help.
We had big plans for in-game cities that would be "alive" with NPCs and would serve as hubs for in-game commerce and player community building. Sadly, it took us far longer to create the cities than we estimated, so we had to launch with only one city per realm.
Also lacking at launch were dungeons. We wanted many more dungeons than were included with the game at launch, a problem we are still attempting to overcome. Because cities and dungeons are basically large models that don't use the terrain system, we ran into many monster AI problems, as well as players getting "stuck" on geometries and even falling through the floors.
In general, Camelot's development was a wonderful experience. Sure, we made some mistakes, but all in all, the game came out beautiful, stable, and most of all, fun to play.
[ Team LiB ]
[ Team LiB ]