How to Become Numbers-Driven With Profile Analyzer

Every game should get a profiling session. Optimally, you should make it happen regularly by using the archifamous Unity profiler tool. But this tool will often give you misleading results for a reason I'm about to explain. Let's see how to be more accurate using the new Unity Profile Analyzer.

Thumbnail

When you profile your game, a few things can happen.

You can find that everything is alright and there's no further action needed. This is rarely the case with any ambitious game.

Often enough, you'll find out that the new feature you implemented is just too expensive for a portion of the hardware you're targeting.

However, there's something I discovered over the years while working with Unity...

Here's my learning: the Unity Profiler is not accurate... by far.

Let me explain why and what you can do about it.

In this blog post, you'll learn:

Quick Navigation

Why the Unity Profiler Is Not Enough

Unity Profile Analyzer: How to Become Numbers Driven

The New Performance Analysis Strategy

The Unity Profile Analyze Package

Example: The Unity Profile Analyzer Workflow

Why the Unity Profiler Is Not Enough

I started with a blunt affirmation: using the unity profiler alone will not give you the accurate results you're seeking.

But wait, don't start writing hateful comments just yet. Let me elaborate on what many would consider blasphemy.

The unity profiler is itself accurate, but almost every game developer is making a dangerous assumption that will make these numbers very misleading.

Instead of getting lost in words — I'm pretty skilled at it — let me show you an example:

void Update()
{
    image.rectTransform = image.rectTransform + Vector3.one * Time.deltaTime;
    character.transform.position = character.transform.position + Vector3.forward * Time.deltaTime;
}

You see what I'm doing there?

It's very basic. It's just a few instructions that I apply on a thousand game objects. This function shouldn't cause performance issues, but let's profile it anyway.

Unity-Profile-Position-Update

Unity Profiler: Position Update Cost

This is the cost of running this function: 2.74 milliseconds. Acceptable.

However, if we stop our profiling session there, we'll make the mistake 95% of the developers do.

You see, the profiler will surely give you the accurate cost of executing your function.

But here's the key: we're missing to measure the cost of the side-effects this function causes.

Profiling your function will not show you its total cost on the game performance, but only its internal execution cost.

Let's zoom out of the function. Here's what the profiler looks like now.

Unity-Profile-Position-Update-Zoom-Out

Unity Profiler: Position Update TOTAL Cost

The panorama the profiler paints now is somehow... different.

The thing is, it's easy to see the side effects of such code in an empty project like above. In this case, 

changing the rectTransform incurs in a canvas rebuild that takes place just before rendering.

But we rarely work with empty projects, which makes these side effects go unnoticed 95%+ of the times as I have seen with almost every game developer I coach.

Now that you're aware of the hidden cost of the code that you were probably not measuring, the question turns to: how can you and I become more accurate at diagnosing the real game performance impact of a change?

Unity Profile Analyzer: How to Become Numbers Driven

As we saw, using the Unity Profiler is clearly not enough to get the total performance cost of a function, much less of a feature or a change.

If your function interacts with the Unity API, chances are high that it might have side effects later on the profiling road.

Luckily for us, fellow developers, we have a fairly new tool at our disposal: the Unity Profile Analyzer.

Let me tell you what this tool will do for you.

The Unity profile analyzer is a package that excels at giving you statistical information about the performance of your game.

And how does that help us?

With the Unity profiler, we capture about 300 frames and then analyze per-frame costs.

Instead of doing this, we can use Unity profile analyzer to statistically analyze the performance metrics of the entire profile session.

This per se is great, but it doesn't solve the problem.

So here's the key: on top of that, Unity engineered  the possibility of comparing different profiling sessions.

Aha, now we're talking.

Do you see where I'm heading to already?

Let's have a look at a more accurate performance analysis strategy.

The New Performance Analysis Strategy

The strategy to get accurate measurements shifts from profiling your function to:

This is what I call A/B game performance test.

Does it sound like more work?

Yes, it is indeed more work.

But it'll be rewarding. That I promise to you.

If you're a professional game developer, you need to know about this performance analysis technique.

Why?

Because judging performance is all about using meaningful numbers. And these numbers have to include all costs, not just a possibly small portion of the whole picture.

Let's have a look at the Unity profile analyzer tool.

The Unity Profile Analyze Package

Let's have a brief look at the Unity profile analyzer.

Start by installing it through the package manager. As of today, it's still a preview package so you'll have to tick the display option in the advanced menu.

Unity-Profile-Analyzer-Install

Unity Profile Analyzer: Installing Package

You can then open the Unity Profile Analyzer through Window → Analysis → Profile Analyzer.

The unity profile analyzer's main interface is divided in four main sections as I show you below (click to zoom in).

Unity-Profile-Analyzer-Single-Overview

Unity Profile Analyzer: Overview (Single)

Based on the numbers I wrote in the picture: