Since the beginning of my AI and Games series on YouTube I've explored a variety of AI techniques to solve specific problems within video game design that require both immediate, reactive behaviour in conjunction with long-term and strategic decision making. First-Person Shooters (FPS) are certainly a common genre from which interesting AI can emerge given its predicated on fast, reactive and engaging gameplay. But if we're to consider the need for effective systems that balance both short and long term decision making in games, there is no greater problem space to explore than Real-Time Strategy (RTS) games.
Both the RTS and FPS genres have continued to innovate the state of the art in AAA video gaming in an effort to satisfy their design. But RTS games have much larger considerations for the long-term impact of quick and reactive decisions, as well as the need to manage resources at varying levels of abstraction and complexity over multi-hour gameplay sessions. When considering the challenges and innovations faced in AI for strategy games, as a player, developer, or researcher there's one franchise you simply cannot go without discussing, Creative Assembly's Total War.
Total War has grown from strength to strength since the launch of the franchise in the year 2000. As each game becomes more ambitious, the challenges faced by the AI systems continue to increase in scope and complexity. This has resulted in a development team that continues to push the state of the art in game AI unlike any other in video game history. But the journey to reach more recent entries at the time of this video, such as Total War: Attila and Warhammer has had more than a few bumps in the road along the way. So in this first in a series, I will be looking at not just how the core tenets of the AI of Total War, but how each entry in the series aims to solidify, tweak or completely rewrite parts of these systems. In addition, I'll look at the response to these changes from the fanbase, the modding community that has thrived within Total War and how The Creative Assembly adopted groundbreaking academic research to help innovate from the very first entry all the way to more recent releases. .
RTS Games and AI
But first let's look at the Real-Time Strategy genre and understand where Total War sits within this space. Real-Time Strategy games are focused primarily on the control of territory and assets: whereby two or more opposing forces will appear in the same map in an effort to assert dominance by securing resources or locations within the existing defined area. Often in order for this to be achieved, each force will build collections of structures and units that either improve the resource collection within a given area, defend existing structures and land or scout and secure areas of the map by force. The latter often requiring the destruction of opposing forces units in the process. This process typically requires players to explore a previously hidden environment by piercing what is referred to as the 'Fog of War' in order to establish enemy and resource locations. In addition, over time each faction is expected to build upon the structures and units previously established by enhancing their existing capabilities and resource acquisition.
While arguably not the first RTS game ever made, Westwood's 1992 release Dune II: The Building of a Dynasty acted as the archetype for future games in this genre such as Westwood's own Command and Conquer series, but also Age of Empires, Total Annihilation, Dungeon Keeper, Homeworld and Blizzard's Warcraft and Starcraft franchises. The genre has continued to evolve and branch off since the late 1990's, with the likes of real-time tactics games like Dawn of War and Star Wars: Empire at War emphasising tactics over resource management. Meanwhile games such as 1994's UFO: Enemy Unknown have explored a balance between the real-time and turn-based decision making, with it still proving popular to this day with the franchise being successfully rebooted as XCOM: Enemy Unknown in 2012. Arguably the largest deviation is the the rise of the Multiplayer Online Battle Arena or MOBA genre - popularised by the Warcraft III mod Defence of the Ancients and typified by games such as Dota 2 and League of Legends - where players assume the role of hero units on the ground to define and assist the larger strategies playing out during a match.
AI applications within real-time strategy games have existed since the inception of the genre, with players responsible for managing the movement of faction units by telling them where to go, but without direct control of their actual movement. As such, players are expected to interact with these systems in order to yield desired outcomes. In addition, AI is critical to the single-player aspects of the game, given that players play off against opposing commanders responsible for managing their own units and construction strategies. The role of the AI player is one that is tremendously challenging and is considered to be one of the larger challenges facing academic research. This has led to the forming of numerous academic benchmarks and projects, including the Starcraft AI Competition hosted at the AAAI conference on Artificial Intelligence and Interactive Digital Entertainment. This is a topic in and of itself, but don't worry: I'll be talking about that on AI and Games in the near future.
Total War
So what makes Total War so special and challenging for an AI system? Creative Assembly's strategy series is interesting in that it combines multiple modes of strategic gameplay into a single game: with players managing turn-based resource management and political strategy alongside real-time combat with large scale armies. In addition, the combat juggles between micro and macro levels: requiring not only AI control of individual units, but more abstract control of unit types and managing the combat formations and troop deployments on the battlefield at runtime.
The first entry in the franchise, Shogun: Total War, balances the combat simulations that strive for realism and authenticity, alongside the political strategy that is aimed to give context and stakes to each conflict. The original Total War is set in 1530, during the Sengoku Jidai period of feudal Japan: a time largely popularised in contemporary fiction by the works of Japanese film director Akira Kurosawa, with films such as Kagemusha, Seven Samurai and Ran proving influential on the design and development of the game - with clips of the latter being used as part of the games cinematics. Both the player and opposing AI assume the role of the 'Daimyo': local-lords who control provinces of Japan with a need to conduct both diplomatic strategy alongside military movements. When rival factions are drawn into conflict, players take control of the 'Taisho' (General) and move hundreds if not thousands of troops across the battlefield. Sengoku Jidai made for an ideal period of history for the game, given the politics and even the economics of the period was built around the logistics of fielding armies in defence of the Daimyo and his ambitions.
Total War deviates from many traditional real-time strategy games in that it removes mechanics such as Fog of War from combat gameplay as well as the need for resource management outside of unit counts from combat decision making. Given the nature of the game itself, the AI player is comprised of two distinct AI systems:
The Campaign and Diplomacy AI manages the turn-based strategy of the game and is responsible for moving armies around the map, conducting diplomacy (be it by sending envoys or assassins to forge alliances or eliminate opposition) and building the agriculture and infrastructure of owned provinces.
Meanwhile the Combat or Battle AI dictates combat unit formations, strategies and attack patterns. Much like human players, this is only responsible for managing the strategy of specific groups of units, given the units themselves are already controlled by AI techniques.
So to get to grips with the AI of Total War, I'm going to explore the AI systems from lowest to highest level: from individual troop control systems all the way strategic systems that seek to conquer all of feudal Japan.
Unit AI Behaviour
Total War combat is driven by units: groups of specific troop types that can be deployed in formation ranging from melee types to archers to cavalry. These combat units are not only expected to keep formation, but to move around and conduct combat as a unit. Movement is often a tricky task, especially when asked to navigate through or around a variety of terrains such as mountains and forests.
![A standard neural net formulation [from https://stats.stackexchange.com/questions/154879/a-list-of-cost-functions-used-in-neural-networks-alongside-applications]](https://aiandgames.com/wp-content/uploads/2018/01/neural-network-2-1.png?width=640&auto=webp&quality=80&disable=upscale "A standard neural net formulation [from https://stats.stackexchange.com/questions/154879/a-list-of-cost-functions-used-in-neural-networks-alongside-applications]")
Neural networks act as a weighted summation function of incoming data. By customising the weights between each simple compute node or 'neuron', we can change the result of the data being pass to the layers on the far right. [IMAGE SOURCE]
To achieve this, Total War adopts artificial neural networks for managing individual units. Neural nets are a fast and effective means to achieve quick and reactive responses to a pre-defined objective. A neural network processes data via 'neurons': simple processing units that take incoming data via 'synapses' that are weighted. By changing the weights, we change the outcome of the data being passed through the neural net. We typically train these weights using machine learning methods but they can also be tweaked by hand if the network is small enough. What makes this so effective is that once they are trained, the processing time to make decisions is lightning fast. In addition, a well-trained neural net is able to generalise its decision making process: meaning it can recognise the similarities in numerous individual circumstances and in each case give a similar answer.
Despite this, neural networks are typically not great when being given multiple objectives to complete at once, especially when they oppose one another. In this instance, the troops have different neural nets they can trigger in depending on the objective they're trying to solve, whether it's moving formation, avoiding oncoming enemy fire or taking position to attack themselves, be it up close or from afar. Each of these networks is pre-defined and is not tweaked or optimised during gameplay, so it's not learning to be better at the game as you're playing it. These Unit AI systems are in effect regardless of whether a human or AI is in control of the army, as each requires these neural net systems to manage the individual troop behaviours.
Combat AI Behaviour
The combat level is the AI system that controls opposing forces akin to players. This AI needs to decide the group formations and actions of units. Moving segments of its army together in order to achieve a specific tactic or micromanage movement of units given the terrain or weather that is currently in play.
To achieve this, Shogun employs a logic system that allows for specific decisions to be made based upon the current state of the world. These decisions are derived from 'The Art of War': a collection of strategies by Chinese military strategist Sun Tzu that dates back to around 500 BC. The use of a logic system is well suited to adapting the texts, given that in many instances a set of conditions are in play that should result in a particular response from the system. These rules considers the type, condition and size of both our units and our opponents, the current terrain, the weather and state of troop morale. Some simple examples derived from The Art of War are listed below:
When you surround an army, leave an outlet free.
On open ground, do not try to block the enemy's way.
If you outnumber the enemy 10:1, surround them.
