AI and Games is a crowdfunded YouTube series that explores research and applications of artificial intelligence in video games. You can support this work by visiting my Patreon page.
In collaboration with Ubisoft and Massive Entertainment, I present three blogs exploring the AI behind Tom Clancy's The Division 2, including excerpts from my interview with the Lead AI Programmer of the franchise, Philip Dunstan.
Part 1 of this series, where we discuss enemy AI design can be found here.
Meanwhile part 2, which explores open-world and systemic design can be found here.
Building a live-service game such as Tom Clancy's The Division, comes with all sorts of challenges. Ensuring the game is stable for players in a variety of online connections, handling the different ways players move through the world and exploring interactive systems and gameplay challenges, but more critically, checking that the game plays as expected so that players aren't getting frustrated because world events don't trigger or missions don't register as complete. While this is most certainly a challenge for quality assurance and testing teams, as the scale and complexity of these games increase, the workloads of QA teams explode in scale. And that's where artificial intelligence can help not just create in-game functionality, but change how the game is being developed.
In this final blog on the AI of The Division 2, I'm going to take a look at the secret AI players that playtest Tom Clancy's The Division with insights from Philip Dunstan: the lead AI programmer at developers Massive Entertainment. I'll be looking at the custom AI bots that are deployed to assess specific parts of the game and how the first games post-launch DLC changed how the game would be tested moving into Division 2.
The Testing Frameworks
Tom Clancy's The Division has not one, but two types of bots that are used to help test the games of the franchise: the server bots and the client bots. The Server bots - as the name might suggest - run natively on the server and don't interface with the game like a player would. As I'll explain in a minute, these bots behave completely different from real players and are designed to stress-test the limits of the Division servers. Meanwhile the client bots run as if they're playing a build of the game client-side. They assume control of the game instead of the player, adhering to all the same rules as a regular player, to a point that all the in-game systems genuinely believe that this player character is being controlled by a human. They don't have any special modifications that allow them to manipulate or cheat the game and are built to run on any platform, be it PC, Xbox or Playstation. Their job is to test how the game actually works for players: testing the main story missions, wandering the open world and gathering all sorts of performance stats along the way that help give the developers a stronger understanding of how the game will perform for players when they log into Washington.
The demand for these types of tools is ever increasing. As in-game worlds in live-service games continue to increase the number of potential bugs explodes exponentially. Even if you consider both Division games, it's not just the map of Washington DC being larger than Manhattan, but each update to both games not only introduces new content - which might have bugs in it - but it also can change or impact a lot of the existing content in the game - meaning even more bugs because you broke something that was already working. This is only made worse by the reality live service games need to be updated fairly frequently to maintain player engagement and these updates need to work, so the word of mouth continues to be strong. This is a problem that exceeds the capabilities of human testers: as more content is being built and existing content modified, not only does quality control need to be maintained on all the new content, but everything else that already exists in the game. This is thousands upon thousands of play hours and is increasingly difficult to balance. And sometimes, the requirements of testing exceed the number of available staff who can even sit down and play the game...
Server Bots
Philip Dunstan: "As you can imagine we're building servers that host a thousand players, but it's really difficult to get a thousand players to play at the same time. And especially if you want to know if your servers can stay up for a week it's difficult to find a thousand players that can play continuously to test the stability of your server while the game is in such an early stage of development.
As mentioned in a previous blog, the original Tom Clancy's The Division runs with what is known as a Server Bot: it's an AI player that logs into a Division server and plays around in the game. This is being used to test whether or not the games systems are operated as expected. As Philip explained, while the develop team really benefit from this, the actual AI they built was really simple and, well... it cheats a lot.
Philip Dunstan: "So very early on in the Division 1, we had these server bots that would connect to a game, they would... you know they're actually really stupid. They're not trying to mimic player behaviour at all. They just teleport around the world, they find NPCs to kill, they shoot the NPCs and then they teleport off to a different part of the world. And they've got god mode turned so they can't be killed and they just do this continuously and then every now and then they disconnect from the game and they reconnect or they group up into co-op sessions and they disconnect. We're testing our ability to you know group players, to create all the different phases for the players to join and disconnect. And then surprisingly it's extremely performance metrics out of these bots. Their performance metrics actually very closely matches the type of metrics we see in players, even though they're not trying play like a player."
"We had those in the Division, we honestly would not have been able to ship a stable Division 1 or Division 2. I mean Division 1 and Division 2 were both extremely stable games you know considering how many players we had after launch. If you look at this last year type thing, the number of like significant downtime causing issues that we've had has been extremely low. And we're able to do that because we're able to test it to an extent that we're satisfied through an automated method."
Client Bots
While the server bots were conceived from the very beginning, the client bots are a different story altogether and emerged from an interesting problem during the development of the first Division. But not at launch, rather with the second DLC update for the game The Underground.
The Underground opens up a whole new game mode in the Division accessible from the basement of your base of operations: the James A Farley post office building across the street from Madison Square Garden. In the underground players would complete procedurally generated missions comprised of different enemy factions hiding out in the tunnels underneath New York. And this is where it introduces a new problem: unlike the rest of the Division, if a mission is going to be procedurally generated how do you test each possible permutation to know it's going to work?
Philip Dunstan: "The client bots were interesting, the Underground, because it is procedurally generated had a sort of problem which had been unique up until that point. Up to that point we'd be able to test whether a level could be completed by having QC run through that level and see if we can complete it. We have a large test team at Ubisoft that is constantly playing through the levels testing things like 'is this level completable'. And that worked perfect fine for the launch of the Division and survival mode. But for underground, we had you know hundreds and thousands of different variations of the level. It no longer became possible to test this manually. We had a technical problem at the time as well that our navmesh generation wasn't consistent enough, that when we generate the navmesh for the underground level one of the variations might be playable, but later on when someone had moved some props around and we may have had a navmesh break on a subsequent generation. So it became not just impossible from a practical sense of how many testers you need, it just wasn't even feasible at all to manually test."
The Client bots were headed up by one Massive's partner studios working on the Division: Ubisoft Reflections based in the UK. As mentioned earlier, the team opted for the more challenging task of creating an entirely new system. The AI players are not based on the existing AI archetypes, instead it's a custom built AI layer that directly controls the players inputs. This helps keep all the development of these tools isolated from main enemy AI but as mentioned, it means every system in the game still believes that a human player is playing the game. The system was subsequently interfaced into the debug console and tools, allowing for a variety of game actions to bypass the controller layer and be processed by the player character. This means that just like a human, the actions it's trying to execute only work if the current game state permits it.
One of the first priorities for the bots was to the test navigation mesh coverage. Navigation meshes are the data structure baked into a level that tells AI characters where in the world they can move around. Without a working nav mesh, no friendly or enemy AI would be able to walk around the map. Hence if any of it is broken this needs to be identified immediately for designers and programmers to test. In addition, follow bots were built that allowed for AI players to follow human ones, once again checking how AI characters might be able to use the navigation mesh to move through complex environments and combat arenas. Plus simple combat behaviours that - while they didn't pay attention to their health or avoiding any hazards - would eliminate targets simply by turning the in-game camera towards an enemies head and then pulling the trigger.
But in-time this scaled up from the more low-level tests of movement space and simple combat, to being able to take on an entire mission. This requires a lot more complexity and interfacing with the separate mission system built into The Division's codebase given it needs to know what the objectives are at any point in time and naturally these shift throughout a given story mission. This requires a more nuanced process, whereby the bots kill all the enemies in an area, follow the path to the objective marker, destroying specific objects if expected to and triggering any and all interactions it finds within a radius of itself.
With the client bots successfully implemented, they could not only test hundreds of permutations of the Underground missions over a weekend, but they could also run tests overnight on all the main story missions from the first game.
