Shader deep-dive #3: Obtaining an animated look from a hand-drawn image

In case the story in your game is more on the minimal side, here’s a technique to make it look just a bit different. In this article, we’ll go from this:

to this:

First of all, credit is due to A-ha and the video of “Take On Me” that somewhat inspired this approach (https://www.youtube.com/watch?v=djV11Xbc914&t=94).

My game Semispheres features an organic/fluid art style where everything is slightly moving so I felt just dropping in a plain image would clash with its existing art style:

If you’re interested in how to achieve that look, check out my previous two shader deep-dives here and here.

Now onto this shader. Similar to my previous articles, we’ll go step by step and see how everything combines together into the final result. All of this code will work in Unity and the concepts will translate to other shader dialects.

First off, we’ll start with a plain shader that just does a texture lookup, where the fragment function looks like this:

fixed4 frag_mult(v2f_vct i) : SV_Target{
    return tex2D(_MainTex, i.texcoord);
}

This will return an example similar to the first image in the article.

Now let’s replace the hand-drawn lines. For that, we’ll use an additional hatch texture. You can just pick one by image-searching for “hatch texture”. Always be mindful about the image’s license depending on the type of project you’re using it in. It may help if the texture is seamless, meaning it won’t show stitching artifacts if you tile it.

Once you’ve located a hatch texture and added it to your project, you can add this to your “Properties” group at the top of the shader:

_HatchTex("Hatch Tex", 2D) = "white" {}

And the corresponding variable:

sampler2D _HatchTex;

If in doubt, just find your main texture and follow that as an example (in our example _MainTex).

Now we can sample this texture to change the output

return tex2D(_HatchTex, i.texcoord);

This is where choosing a seamless texture comes in handy. In the end result it will not matter much though. Assuming the texture is set to repeat, we want to make the lines more dense, so let’s try this:

return tex2D(_HatchTex, i.texcoord * 10);

This will have an effect of making the lines 10 times denser. The way it works is that instead of having the texture coordinates go from 0 to 1, they will go from 0 to 10 – effectively meaning that instead of fitting your original hatch image in the sprite, it will fit it in 10 times.

Here’s how that looks:

That’s a bit too dense, so let’s settle on somewhere in the middle using 3 instead of 10:

That’s very much acceptable. A brief aside, when playing with values inside a shader, try to follow the “Double or halve” rule to make sure you’re noticing the change and understanding the effect of your tweaks (http://www.gamasutra.com/view/news/114402/Analysis_Sid_Meiers_Key_Design_Lessons.php).

So while this is what we wanted for now, it’s not very useful, now is it? In order to get something nicer out of it, let’s combine the two images. We do this by alpha-compositing (or alpha-blending) the two images: https://en.wikipedia.org/wiki/Alpha_compositing.

You can find various variants on the formula, but here’s a function that I’m currently using:

fixed4 alphaBlend(fixed4 dst, fixed4 src) {
    fixed4 result = fixed4(0, 0, 0, 0);
    result.a = src.a + dst.a*(1 - src.a);
    if (result.a != 0)
        result.rgb = (src.rgb*src.a + dst.rgb*dst.a*(1 - src.a)) / result.a;
    return result;
}

With this function added to the shader, we can change our fragment function to the following:

fixed4 original = tex2D(_MainTex, i.texcoord);
fixed4 hatch = tex2D(_HatchTex, i.texcoord*3);
return alphaBlend(original, hatch);

Let’s segment the image a bit based on the original. This will vary a bit on how your image is stored (whether the black background is baked into the image or there is a transparent background). We want to output the hatch only if there is something in the original image.

We’ll use the alpha of the original pixel to determine what we should draw on it, let’s take this step by step:

fixed4 original = tex2D(_MainTex, i.texcoord);
fixed4 output = fixed4(0, 0, 0, 1);
fixed4 hatch;
if (original.a > .1) {
    hatch = tex2D(_HatchTex, i.texcoord * 7);
    output = alphaBlend(output, hatch*.8);
}
return output;

This will output the hatch texture wherever there was something (with alpha greater than 0.1) in the original image.

Before we move on, let’s dig into some of these numbers to make sure we understand what their contribution is.

The .1 in "original.a > .1" determines the cutoff, let’s use another value, for example .5:

Notice how less of the image is being drawn.

The 7 in "i.texcoord * 7" determines the frequency of the lines as described before, using a smaller value like 3 yields this:

And the last value of .8 in "hatch*.8" determines what the contribution of this will be, higher values will mean output is more intense. Here’s what using 1.3 looks like:

Now that we understand how all of these things work, let’s layer things a bit up. Adding another layer for alpha greater than .4, with a different line frequency (9) and higher contribution (1.2):

if (original.a > .4) {
    hatch = tex2D(_HatchTex, i.texcoord * 9);
    output = alphaBlend(output, hatch*1.2);
}

Looks like this:

Ok, starting to look better. One more segment like this:                    

if (original.a > .8) {
    hatch = tex2D(_HatchTex, i.texcoord * 1.2);
    output = alphaBlend(output, hatch*1.5);
}

Just to catch up, here’s how the whole fragment function looks right now:

fixed4 frag_mult(v2f_vct i) : SV_Target{
    fixed4 original = tex2D(_MainTex, i.texcoord);
    fixed4 output = fixed4(0, 0, 0, 1);
    fixed4 hatch;

    if (original.a > .1) {
        hatch = tex2D(_HatchTex, i.texcoord * 7);
        output = alphaBlend(output, hatch*.8);
    }

    if (original.a > .4) {
        hatch = tex2D(_HatchTex, i.texcoord * 9);
        output = alphaBlend(output, hatch*1.2);
    }

    if (original.a > .8) {
        hatch = tex2D(_HatchTex, i.texcoord * 10);
        output = alphaBlend(output, hatch*1.5);
    }

    return output;
}

Ok, so far so good. This looks interesting but it’s still rather static. Let’s address that.

First, let’s animate the hatches by changing them every once in a while. To make this easier to follow, we’ll go back to just showing the hatch texture:

return tex2D(_HatchTex, i.texcoord);