Prototypes in The Machinery

Most game engines have some way of creating reusable entities — i.e., template entities that can be placed multiple times in different levels. Sometimes, these templates are called prefabs.

An important part of any template or prefab system is that if you make changes to a prefab, those changes are automatically reflected to all placed instances of that prefab. Prefab systems let you start building levels with placeholder entities and then later update the template with the real graphics.

Advanced prefab systems can support mixing and matching templates in various ways. For example, maybe you can create an “advanced enemy” template which is identical to the “basic enemy” template, except it is tinted red and has 50 % more hitpoints. Changes made to the basic enemy would automatically carry over to the advanced enemy and its instances. For example, if you changed the hit points of the basic enemy to 100, all the instances of the advanced enemy would get 150 hitpoints.

Or, you might be able to create a house template with windows and doors and then, in one placed house instance, swap the default wooden door for a brass one. If you later changed the windows of the template, this instance would get the new windows but keep its brass door.

In The Machinery, we call our templates prototypes and in this blog post, I’ll describe how our prototype system works and try to explain why we made the choices we did.

The trouble with templates

Designing a template system can be difficult because it requires you to manage your data at a higher level of abstraction. The user needs a way to edit, not just the data itself, but also the rules for how that data relates to other data.

There are no fixed rules for how this should work. A template system can be anything you can dream up. The Slide Master concept in Powerpoint/Keynote and the reusable page layouts and text styles in InDesign are simple template systems, while the class or prototype hierarchy in object-oriented languages can be seen as an advanced template system.

When you design a template system for a game engine, you have a bit of a dilemma.

On the one hand, you want the template system to be as powerful as possible. “Powerful”, in this context, means that the users can express complicated data relationships and use that to automate as much of the workflow as possible.

On the other hand, you want a simple and easy-to-use UI.

There is an inherent conflict between these two goals. The more powerful the abstractions are, the harder it is to create a UI for them (other than a basic text editor).

Let’s see how this plays out in a few examples, and then we’ll look at the path we took in The Machinery.

1. Source code

The most powerful way of expressing data relationships is with code I.e. the “configuration file” is just a program, written in some general-purpose programming language, that you “run” in order to generate the data for the project.

Since the “configuration file” in this case is a program, written in a Turing-complete programming language, it can automate anything that is possible to automate.

Creating the “advanced enemy” template could look something like this (in pseudocode):

advanced_enemy = basic_enemy
advanced_enemy.hitpoints *= 1.5
advanced_enemy.meshes.torso.tint = rgb(255,0,0)

But the automation could go much further than this. We could write a function that placed lamp posts along a road. We could procedurally generate stairs based on the step width, step height, etc. Since it’s general purpose code, anything is possible.

As you can see, this approach is super powerful. The drawback is that since the data is code, only programmers will be able to edit it. And even for programmers, writing code is not the most natural way to edit visual things, such as meshes and levels.

For some applications, this can still make sense. For example, the Premake build system (which we use to build The Machinery) uses regular Lua programs as its configuration files. In this case, the lack of a UI is not a big deal, since only programmers are interested in build systems (in fact, even programmers are not that fond of them).

A way of making the data-is-code approach accessible to more users would be to use a visual scripting language, instead of a textual one, but this comes with its own set of compromises. To programmers, visual scripting tends to be more tedious than just writing code. To non-programmers, visual scripting is more complicated and less visual than a regular GUI. While it makes sense for advanced tasks, such as gameplay scripting or rendering, it’s typically not something they would want to use for the majority of their data, such as building levels.

Another drawback of this approach is that if your data is a program written in a general-purpose, Turing-complete language, you can run into things like the halting problem. I.e., you can’t reason about the data in any way without first running the code that generates it. This can be problematic if it takes long for the program to run. Or, if the program has a bug and never completes — now you have no data at all, just an empty scene, until the error is fixed.

But again, there are ways of making this approach work. For example, you could argue that Houdini is essentially a visual scripting language that you “run” in order to generate your scenes, just more high-level than your typical visual scripting language. Note that the immediate visual feedback that Houdini gives when you edit the graph is key to making it usable.

2. Markup language

The next step in expressive power below code is a “markup language”, such as CSS or HTML. Markup languages are parsed, not run, and can express complicated relationships, without being Turing-complete.

A markup language implementation of the example above might look something like this:

advanced_enemy : basic_enemy {
    .hitpoints = calc(base.hitpoints * 1.5)
    .meshes.torso.tint = (255, 0, 0)
}

I’m making up both the syntax and the object model here.

Markup languages come in many shapes with varying degrees of expressive power. For example, in the code above, I’m assuming that (similar to CSS) the language can calculate expressions such as base.hitpoints * 1.5. In a more basic language, this might not be possible, and we would instead have to set the hitpoints as a hard number:

advanced_enemy : basic_enemy {
    .hitpoints = 45,
    .meshes.torso.tint = (255, 0, 0),
}

In this case, the hitpoints wouldn’t automatically scale if we change the hitpoints of the basic_enemy — they would remain at 45 until we explicitly changed them to something else.

Markup languages have the advantage of being more limited and contained than full programming languages. This makes it easier to create some kind of UI/editor for them and since we don’t “run” the data, we don’t have to worry about “bugs” or infinite loops.

But beware, markup languages are often subject to feature creep — as time goes on, the developers add more and more useful “features” to the language until it approaches the complexity of a general-purpose programming language — just a shitty and haphazardly designed one. I’m looking at you CMAKE!

This is the code/data cycle that I’ve tweeted about before.

The Code/Data Cycle.

The Code/Data Cycle.

What kind of UI can we create for our markup language? Again it depends on the expressiveness. The more expressive the language is, the harder it is to create a sensible UI. Here’s an attempt:

Possible UI for the declarative language.

Possible UI for the declarative language.

Yay, we’ve made a UI… but actually, it is really just a glorified text editor. To use this UI, the user still has to know that he needs to enter the magic incantation base.hitpoints * 1.5 in the textbox. Not very user friendly and doesn’t seem like a nice way to work, even if autocomplete, syntax highlighting, and drag-and-drop could lessen the burden somewhat.

If you want a markup language to work with a UI, I think you have to carefully design it with that in mind. Otherwise, the text editor will end up being the default (as happened with both HTML and CSS).

An example of the markup language approach is the Universal Scene Descriptor format (USD). USD allows resources to be reused and remixed in lots of powerful ways. But it is hard to imagine what a UI editor for it would look like — it’s predominantly a format geared for text-editors. Nothing inherently wrong with that — but it’s not what we wanted for The Machinery.

3. “User-friendly” UI

Finally, let us imagine what a user-friendly UI might look like:

User-friendly UI.

User-friendly UI.

There are no magic text incantations. The hitpoints are set with a regular slider. The tint color is set with a regular color picker (sorry for my limited ability to draw one).

We have the nice UI that we want, but no way of expressing something like “50 % more hitpoints”. Also, the relationship between the advanced enemy and the basic enemy is unclear. We can’t easily see how the one affects the other.

This approach doesn’t look like it gives us enough expressive power.

In my opinion, making templates work in a graphical UI, and not just a text editor, depends on finding a good “cut”: A more limited set of features — less than what code or a markup language could do, but still powerful enough to handle most of the users’ needs — that we can put together a sensible UI for.

Prototypes in The Machinery

We had three main goals with the prototype system in The Machinery:

Since our template system works on any object, we don’t use the word Prefab, instead, we call our templates Prototypes and the places where they are used Instances.

The Machinery Data Model

Our Data Model is based on Objects with Properties. Objects have property Keys that can be assigned Values. Each object has a Type and the type of the object determines which properties it has: