Welcome to Performance Testing

Estimated Lesson Time: 20–25 Minutes

Lesson Overview

Throughout the previous modules, you’ve learned how Microsoft Flight Simulator works, how your hardware affects performance, and how to optimise Windows, your graphics drivers and in-simulator settings.

However, there is one important question that every professional optimiser asks before making any changes:

“How do I know this actually improved performance?”

This is where performance testing begins.

One of the biggest mistakes Flight Simulator users make is changing several settings at once and assuming the simulator feels better.

Without proper testing, it’s impossible to know whether:

• A change genuinely improved performance.
• Another setting caused the improvement.
• The weather changed.
• The airport was less demanding.
• The improvement was simply psychological.

Professional optimisation is built on evidence—not assumptions.

This module will teach you how to measure performance correctly, compare results fairly and build your own repeatable benchmarking process.

Lesson Objectives

By the end of this lesson you will understand:

Why benchmarking is essential.

Why guessing wastes time.

The principles of scientific testing.

Why controlled testing produces reliable results.

The importance of repeatability.

How to create your own benchmark routine.

1. Why Benchmarking Matters

Benchmarking is the process of measuring your simulator’s performance under controlled conditions.

Rather than relying on how the simulator “feels”, benchmarking provides objective data that can be compared over time.

Professional benchmarking allows you to answer questions such as:

• Did this graphics setting actually improve performance?
• Did the new graphics driver make things better or worse?
• Did upgrading my GPU increase FPS?
• Has a Windows update affected my simulator?
• Is my latest optimisation worth keeping?

Without benchmarking, every change becomes guesswork.

2. The Problem With Guessing

Many users optimise Microsoft Flight Simulator like this:

• Change Terrain LOD.
• Update the graphics driver.
• Install a new airport.
• Change DLSS mode.
• Enable Frame Generation.
• Restart the simulator.

Then they say:

“It feels smoother.”

But why?

Which setting made the difference?

Was it the driver?

Was it the airport?

Was it simply because the weather changed?

Without proper testing, there’s no way to know.

Guessing often leads users to keep unnecessary settings or remove optimisations that were actually helping.

3. Think Like a Scientist

Professional benchmarking follows the same principles used in scientific experiments.

The key idea is simple:

Change one variable at a time.

If you change multiple things simultaneously, you can no longer identify which change produced the result.

For example:

Poor testing:

• New driver.
• New graphics settings.
• New weather.
• Different airport.

Result:

Impossible to interpret.

Good testing:

• Same airport.
• Same aircraft.
• Same weather.
• Same route.
• Same graphics settings.

Only change:

Terrain LOD.

Now you know exactly what changed.

4. Controlled Testing

Controlled testing means keeping every possible variable consistent.

Examples include:

• Same airport.
• Same aircraft.
• Same parking stand.
• Same weather preset.
• Same time of day.
• Same season.
• Same AI traffic settings.
• Same graphics settings.
• Same camera position.
• Same flight route.

The only thing that should change is the setting you’re testing.

This allows you to make fair comparisons between results.

5. Why Repeatability Matters

Imagine you benchmark Heathrow today.

Tomorrow you test at Innsbruck.

Next week you test at JFK.

Can you compare the results?

No.

Every airport places a different load on your PC.

Repeatability means you can recreate the exact same test whenever you need it.

This allows you to compare:

• Driver updates.
• Hardware upgrades.
• Simulator updates.
• Graphics changes.
• Add-on performance.

Professional reviewers use repeatable testing because it removes uncertainty.

6. Building Your Own Benchmark Routine

Every Flight Simulator user should develop a benchmark routine that can be repeated whenever major changes are made.

A good benchmark routine should include:

Airport

Choose one airport and continue using it throughout the course.

Avoid changing locations.

Aircraft

Use the same aircraft every time.

Different aircraft have different system complexity and CPU demands.

Weather

Use the same weather preset.

Avoid Live Weather when benchmarking, as conditions change constantly.

Time of Day

Keep lighting conditions consistent.

This ensures shadows and reflections remain comparable.

Route

Follow the same taxi route, departure and flight path.

Consistency is the key to meaningful results.

7. Why “Feeling Smooth” Isn’t Enough

It’s perfectly normal for performance to feel different from one day to the next.

However, your perception can be influenced by:

• Expectations.
• Recent updates.
• Different scenery.
• Weather.
• Fatigue.
• Frame pacing.

That’s why benchmarking uses measurements rather than feelings.

A simulator may feel smoother even if the average FPS hasn’t changed because frame pacing has improved.

Likewise, higher FPS doesn’t always mean a better experience if frame times become inconsistent.

Later in this module, you’ll learn how to measure these factors accurately.

8. Benchmark Before You Optimise

One of the most common mistakes users make is optimising before recording a baseline.

Without a baseline, you have nothing to compare against.

Always benchmark your simulator before making significant changes.

Your baseline becomes your reference point for every future optimisation.

Common Benchmarking Mistakes

Testing at a different airport every time.

Using Live Weather during benchmarks.

Changing multiple settings simultaneously.

Comparing results from different aircraft.

Judging performance after only a few seconds.

Relying solely on average FPS.

Forgetting to record system specifications.

Professional Benchmarking Workflow

Whenever you optimise your simulator, follow this workflow:

Step 1

Create a repeatable benchmark scenario.

Step 2

Record your baseline.

Step 3

Make one change only.

Step 4

Repeat the exact same benchmark.

Step 5

Compare the results.

Step 6

Keep the change only if it provides a genuine improvement.

Student Exercise

Today you’re going to create your own benchmark routine.

Step 1

Choose:

• One airport.
• One aircraft.
• One parking stand.
• One weather preset.
• One time of day.

These settings will become your permanent benchmark scenario throughout the Performance Academy.

Step 2

Plan a simple flight.

For example:

• Start at the gate.
• Taxi to the runway.
• Depart.
• Climb for five minutes.
• Return to land.

The route doesn’t need to be long—it needs to be repeatable.

Step 3

Write down your benchmark routine and save it.

You’ll use this exact routine throughout the remainder of the course whenever you need to compare performance.

Student Checklist

Benchmark airport selected.

Aircraft selected.

Weather preset chosen.

Time of day fixed.

Route planned.

Benchmark routine documented.

Lesson Summary

Congratulations.

You’ve completed the first lesson of Module 5 and taken the first step towards becoming a professional performance tester.

Remember:

• Benchmarking is the foundation of every successful optimisation.
• Guessing wastes time and often leads to incorrect conclusions.
• Scientific testing means changing one variable at a time.
• Controlled testing removes unnecessary variables and creates fair comparisons.
• Repeatability allows you to compare results across hardware upgrades, driver updates and simulator versions.
• Your benchmark routine will become one of the most valuable tools you use throughout the rest of this course.

Knowledge Check

1. Why is benchmarking more reliable than judging performance by feel?
2. Why should only one variable be changed during a benchmark?
3. What does controlled testing mean?
4. Why is repeatability essential when comparing performance?
5. Why is Live Weather generally unsuitable for benchmarking?
6. What should be included in a benchmark routine?
7. Why should you record a baseline before making optimisations?
8. What is the biggest mistake users make when testing Microsoft Flight Simulator performance?