Optimisation & Robustness

A backtest answers one narrow question: did this exact set of rules make money over my history? But every system has dials - a fast period here, an ATR multiplier there - and so far you have set those numbers by feel. Optimisation is the disciplined way to turn the dials and watch what happens, all without editing code by hand. Done well, it tells you whether your edge is broad and stable. Done badly, it is the quickest route to fooling yourself into trading a curve-fit fantasy.

This chapter teaches both halves: the mechanic of sweeping parameters, and the judgement to read the results honestly. Get the judgement right and optimisation becomes the bridge from "I have a backtest" to "I have a strategy I trust" - exactly the leap the playbook chapters ahead ask you to make.

From Param to Optimize

You already know Param - it makes a number adjustable from the Parameters window, but it is still a single value you choose. Optimize looks almost identical, yet it hands the choice to the backtester:

// A Param is one value you set by hand:
Fast = Param("Fast EMA", 10, 5, 30, 1);

// An Optimize is a RANGE the backtester can sweep:
Fast = Optimize("Fast EMA", 10, 5, 30, 1);
//                  name   default min max step

The five arguments are the same shape as Param: a name, a default, a minimum, a maximum and a step. Here is a complete crossover system written for optimisation - notice it is just an ordinary system with Optimize in place of Param:

_SECTION_BEGIN("EMA Crossover - Optimisation");

// Two dials to sweep. Outside an optimisation each returns its default.
Fast = Optimize("Fast EMA", 10,  5, 30, 1);   // tries 5, 6, ... 30
Slow = Optimize("Slow EMA", 20, 20, 60, 2);   // tries 20, 22, ... 60

emaFast = EMA(Close, Fast);
emaSlow = EMA(Close, Slow);

// Only trade pairs where the fast line is genuinely faster
Buy  = Cross(emaFast, emaSlow) AND (Fast < Slow);
Sell = Cross(emaSlow, emaFast);

// Non-repainting: decide on the close, act on the next open
Buy  = Ref(Buy,  -1);
Sell = Ref(Sell, -1);
Buy  = ExRem(Buy,  Sell);
Sell = ExRem(Sell, Buy);

SetTradeDelays(0, 0, 0, 0);
BuyPrice = SellPrice = Open;
SetPositionSize(100, spsPercentOfEquity);

_SECTION_END();

Running an optimisation

Open the Analysis window, pick your symbol and date range, then click the small arrow beside the Backtest button and choose Optimize. AmiBroker works out how many combinations exist - here 26 fast values times 21 slow values, minus the pairs the Fast < Slow filter discards - and runs a full backtest for each one. Every combination becomes a single row of results.

Keep your step sizes sensible while you are here - sweeping an EMA in steps of 1 is reasonable; in steps of 0.1 is just noise.

Exhaustive vs smart optimisation

By default AmiBroker runs an exhaustive optimisation: it literally tests every combination of your parameters. With two dials of, say, 100 values each, that is 100 x 100 = 10,000 backtests - quick and thorough. The catch is how fast that number grows, because each new dial multiplies the count:

By four or five dials an exhaustive sweep would run for days - it simply does not finish in sensible time. That is where smart (non-exhaustive) optimisation comes in.

A smart optimiser does not test everything. It starts with a scattered handful of parameter sets, backtests them, keeps what worked, and evolves the next batch toward the promising regions - a little like natural selection. In a few thousand tests it homes in on the good areas of a space that would take billions of exhaustive runs to cover.

You do not lose anything that matters by going smart. A smart search might miss a single razor-thin profit spike hidden in the space - but you do not want that spike anyway. As you are about to see, a lone peak is a curve-fit trap; the broad, robust plateaus you actually want to trade are exactly what smart optimisers find well.

Switching one on is a single line at the top of your formula:

OptimizerSetEngine("cmae");          // use the CMA-ES smart optimiser

Fast = Optimize("Fast EMA", 10,  5,  60, 1);
Slow = Optimize("Slow EMA", 20, 10, 120, 1);

AmiBroker ships three smart engines, all selected the same way:

• "cmae" - CMA-ES, the most capable; a great default that converges fast. Leave its settings alone and it just works.
• "spso" - Standard Particle Swarm, inspired by how a flock or swarm searches together.
• "trib" - Tribes, a self-tuning particle-swarm variant with nothing to configure.

To push a hard problem harder, OptimizerSetOption raises two dials - "Runs" (how many times to restart the whole search from scratch, which guards against getting stuck in a merely local best) and "MaxEval" (the maximum backtests per run):

OptimizerSetEngine("spso");
OptimizerSetOption("Runs", 5);        // restart the search 5 times
OptimizerSetOption("MaxEval", 5000);  // up to 5000 backtests per run

A useful rule of thumb: multiply your test budget by about ten for each parameter you add. And one caveat - smart engines love continuous dials (periods, multipliers) but stumble on on/off switches; keep binary parameters out of a smart run and set those by hand.

Reading the result table

When the run finishes you get a table where each row is one parameter combination alongside its full statistics: Net Profit, CAR (compound annual return), Max. system % drawdown, CAR/MDD, win rate, number of trades and more. The instinct of every beginner is to sort by Net Profit and crown the top row the winner. Resist it.

Also glance at the trade count. A combination showing a spectacular return on six trades has told you almost nothing - there is no statistical weight behind it. You want settings that produce a healthy sample and good risk-adjusted numbers.

Two parameters at once: the heatmap

When you optimise exactly two parameters, AmiBroker can draw a 2D heatmap of the results. One parameter runs down the rows, the other across the columns, and each cell is shaded by your chosen metric - bright green for strong results, red for poor ones. This single picture is worth a hundred table rows, because it shows you the shape of the result, not just its peak.

What you are hoping to see is a broad green island - a whole neighbourhood of settings that all worked. What you are dreading is a lone bright cell surrounded by red.

For that same two-parameter run, AmiBroker can also tilt the heatmap into an animated 3D surface - click the Optimize drop-down and choose View 3D optimization graph. It is the same data drawn as a landscape: a robust plateau becomes a broad, flat tableland you could stand on, while a curve-fit peak is a lone, needle-thin spike. Rotate it with the mouse and the difference between a real edge and a fluke is unmistakable.

Robust beats peak

Here is the single most important idea in this chapter: a broad plateau of good results beats one tall spike. If Fast = 11, Slow = 34 is dazzling but Fast = 10 and Fast = 12 next to it are terrible, that peak is almost certainly a coincidence of your particular history. Trade it and the market will hand you the neighbours' results, not the peak's.

A setting sitting in the middle of a green plateau is robust: it says "anywhere around here works", which is what an edge looks like. Pick the centre of the island, not the highest pixel. You give up a little backtested profit and buy a great deal of real-world reliability.

Avoiding over-fitting

Over-fitting is the act of tuning a system so tightly to past data that it learns the noise instead of the signal. It is seductive because the backtest looks magnificent - right up until live trading, where it falls apart.

Try it yourself

• Run the system above on RELIANCE, sort the table by CAR/MDD, and note how different the "best" row is from the top Net Profit row.
• Optimise just the two EMA periods and view the 2D heatmap - can you spot a green plateau, or only isolated spikes?
• Pick the centre of the plateau, set those values back as plain Param numbers, and run a normal backtest to confirm the report.
• Split your date range in half, optimise on the first half, then test the chosen settings on the second half. Did the edge survive?

Recap

• Optimize("name", default, min, max, step) replaces Param and lets the backtester sweep a range; outside an optimisation it just returns the default.
• Run it from the Analysis window's Optimize button; each parameter combination produces one row of results.
• For two or three dials an exhaustive sweep (every combination) is fine; beyond that, switch on a smart engine with OptimizerSetEngine("cmae") that samples and evolves instead of testing every point.
• Sort by a risk-adjusted metric like CAR/MDD, not raw Net Profit, and check the trade count for statistical weight.
• A 2D heatmap reveals the shape of the results; a broad green plateau beats a lone bright spike.
• Choose robust settings from the middle of a plateau, use few parameters, hold out data, and re-test in sandbox to avoid over-fitting.

With optimisation in hand, we are ready to build real systems end-to-end - and the playbook opens with the classic that started it all: the EMA crossover.