What follows is several excerpts from Braving Britannia: Tales of Life, Love, and Adventure in Ultima Online (http://www.bravingbritannia.com). In addition to speaking with dozens of players about their experiences in the grandfather of MMOs, the book features interviews with several of the game’s early programmers, designers, and producers, including Chris Mayer (Fallout 76), Raph Koster (Star Wars Galaxies), Starr Long (Shroud of the Avatar), Gordon Walton (Crowfall), Rich Vogel (DOOM), and others.
Before Chris “Faceless” Mayer went to work at Origin Systems as lead programmer, he experimented with Ultima Online as an early adopter. Outside of the game, Mayer worked as a software engineer in Austin, Texas, utilizing his employer’s faster-then-average internet connection to play Ultima Online during lunches or after hours.
Mayer had always had an interest in creating games and hoped one day to do so in a professional capacity. In fact, he’d been preparing for the opportunity throughout his youth.
“I’ve always wanted to make games since I was a kid,” Mayer said. “It’s what got me into programming. One of my favorite games was Adventure Construction Set, where I even spent a week while on vacation with my family designing the game on graph paper and notebook pages before coming home to my computer and coding it all in.
“I taught myself Applesoft Basic in fifth grade and took a class in 6502 assembly in seventh grade. In college, I learned various flavors of assembly, C, C++, and a few other languages I have never used since.”
Until Mayer could figure out how to get his foot in the door with a game studio, he was content to simply play the games being released. Throwing himself into UO whenever he had a free moment, Mayer said he and played “religiously,” and became an active member of the community, both inside the game and through the Vault Network message boards, a popular website of the era dedicated to roleplaying games.
In the summer of 1998, Mayer’s close friend Scot Salmon received an email from Origin Systems. Having seen many of Salmon’s helpful posts on the Vault Network’s forums—and knowing of his background with computers—a representative of the company invited him to apply for an open programmer role.
While Salmon wasn’t interested in becoming a programmer, he knew someone who was, and pointed the recruiter in Mayer’s direction. Following an interview with three programmers at the Origin Systems offices, Mayer was offered the job by producer Rich Vogel.
Mayer said that shortly after Ultima Online’s launch, the game’s original programmers—who had been contracted to build the UO codebase—were assigned to other games.
"For the first few months of my employment, I came into work every day thinking the entire service would combust into flames...that was, to this day, one of the most exciting jobs I’ve ever had."
“When I walked in the door at Origin, Rich told me that only one of the original programmers remained on the project and that he was primarily a client guy,” Mayer said. “Kirk ‘Runesabre’ Black was already working, and it turned out Kirk and I were the only ones who could work on the backend.”
And just like that, Mayer was in over his head. Not only did he not understand the game’s codebase, but with thousands of players inhabiting UO’s servers, everything seemed poised to explode on a regular basis.
“For the first few months of my employment, I came into work every day thinking the entire service would combust into flames,” Mayer laughed. “We somehow kept it together while slowly making improvements.
“That was, to this day, one of the most exciting jobs I’ve ever had. MMOs were so new, and no one really knew what they were doing at EA when it came to online games, so Kirk and I pretty much got to do whatever we wanted.”
While players across UO’s virtual world of Britannia were swinging swords and casting spells without a care in the world, Mayer and Black were putting out fires and solving problems as fast as they’d appear—the equivalent to riding unicycles across tightropes over a forest of razor blades.
“It was hard,” Mayer said. “I was also unfamiliar with game architecture in general. Couple that with the fact that the UO team was under stupid amounts of pressure to ship the game when they did, and they had to take many, many shortcuts.
“That made the codebase very fragile and a frightening tangle of dependencies between systems. We had to deal with service outages practically every night for a long time until we could stabilize the codebase.”
The first priority for Mayer and Black was stabilization as a whole, fixed through a branching solution that allowed the programmers to make hotfixes to certain sections of the game without risking unintended changes across the entire system.
“A branch is a line of source code that is independent of any other branches of development,” Mayer explained. “We might put a branch out on Live while a different branch is being tested by QA, while perhaps a third branch is being worked on by engineers. All these changes are independent, so if we have to fix something [on the live version of the game], we only need to touch the Live branch and test the fixes there.”
Once the servers were stable, Mayer was promoted to lead programmer. Now thoroughly “in the fire,” he set to work on Origin’s next project, the game’s first official expansion called The Second Age (“T2A”). With the project came new challenges, including a new landmass, two new cities, and a ton of new monsters for players to battle. And with all those changes, of course, came more stability issues.
“I remember getting my first client bug trying to ship T2A,” Mayer said. “I was very comfortable working in the backend because it resembled the kinds of systems I had to work on in school and at National Instruments.”
But while Mayer knew programming, UO was his first gig involving 2D and 3D graphics. New problems reared their heads including rugs falling under floors and even player houses disappearing altogether.
“I had no clue what I was doing, but managed to get it working by walking through the code methodically,” he said. “Kirk and I worked tirelessly to keep the servers running, including having to miss the launch party for that expansion.”
Soon, Mayer was tasked with tracking down exploiters—players who abused faults in the game’s code for their own gain. Exploits were often used to make duplicates of powerful objects, or raise a player’s skill in a way that went against the game’s design principles.
Mayer recalled a time that one of the game’s servers crashed repeatedly with little explanation. Putting on his detective hat and looking at the code, Mayer saw no logical reason for the recurring problem.
Night after night, Mayer watched as one of the servers suddenly blinked offline. In turn, it caused a chain reaction, causing many of UO’s other game servers—even those stationed in separate data centers across the world—to crash shortly thereafter.
“It made no sense whatsoever,” Mayer said. “Unless… someone was doing it on purpose. Then, such a pattern could be explained.”
Trying to find some method to the madness, Mayer asked the employees in charge of the game’s authentication services to correlate the crashes with players who had logged on before it occurred. It didn’t take long to find a match.
“One player was always on all of the servers before they crashed,” Mayer said. “But what was he doing? We had to know.”
Mayer sought assistance from the customer service department and consulted with a Game Master about tracking the player. With one conversation, Mayer had successfully set up a sting operation in a virtual world.
“I wanted to track this player’s movements in-game,” he said. “When that player logged in, they called me downstairs and took me to a GM’s computer where he was following the player invisibly.”
Whereas any other day, Mayer did everything possible to keep the game’s servers from crashing, on this day he needed it to happen.
A plan was put into motion for the Game Master to dress up as a new player and follow the person of interest. While in disguise, the GM would annoy the player to the point where the suspect would kill him. This death would subsequently cause the player to turn into a murderer, or “go red.”
If all went according to plan, Mayer believed the suspect would crash the server in order to revert it back to its last save point—roughly 30 minutes before he became a murderer—and reveal how his trick was performed.
“Now, I want you to imagine for a minute what it’s like to be a GM, having to listen to every conceivable kind of abuse from players who were pissed enough to call customer service, and take it like a professional,” Mayer laughed. “Then, being given permission by your boss to just unleash on a player.
“This GM went off on this guy and became the most annoying player ever to play UO. The guy couldn’t take much of it and finally killed our poor GM. Now, it was time to kill him. The GM went off screen, turned himself into a poison elemental, and one-shot the guy.”
Just as they’d planned, the suspect had become a murderer. While the suspect was resurrected by his friends, Mayer and the GM sat invisible, hoping he’d divulge his secret to crashing the server.
They didn’t have to wait long.
“After watching this guy curse to his friends about how crappy that was, he said something like, ‘It doesn’t matter. Watch this,’” Mayer recalled. “I saw a stream of unfamiliar characters come into the chat space above his head, and it clicked. T2A shipped with localization for the first time (which attempted to convert foreign languages into English), and clearly there was a problem with the chat message that used Unicode—16-bit instead of 8-bit ASCII—characters.
“That network message was being sent to a buffer without a size check, and that caused a buffer overrun and corrupted the stack.”
The server crashed just as it had over the nights prior. Mayer and the GM had caught their guy red-handed. Mayer coded a fix for the issue and the player was banned from the game.
Despite the constant need for troubleshooting, Mayer said his experience at Origin helped him understand the MMO technologies that ran countless games over the next two decades.
“In 1998, this stuff was largely the Wild West, especially at Electronic Arts,” he said. “EA was the first big game company to make an MMO, and there really wasn’t a lot of standards or best practices developed back then.
“Nowadays, this stuff has matured. Not only have the game companies buil
No tags.
