Worked path · DATA

Data hygiene for StrategyQuant X: gaps, sessions and costs before you build

Edge Finder, the Builder and the retest all read the same two things: minute history and an instrument description. If either differs from what the broker’s terminal trades, every later number inherits the difference. This page lists the checks in the order to do them. It uses SpecSmith for the instrument description and GapSmith for the history.

Data hygiene orderCheck gaps, fill short ones into a separate symbol, compare the instrument description with the terminal, apply the rows you choose, then measure in Edge Finder.01GapsAnalyze data quality02FillFix gaps into _patched03ContractCompare with the terminal04ApplyTick rows, apply, undo05MeasureEdge Finder
Fig. 1: The order of the checks.

The checks, in order

  1. How much of the M1 history is missing. Select the symbol on the Data tab and open Tools, View and Analyze, Analyze data quality. Look at the gap list. The GapSmith dialog adds four columns: Bars, Missing, Fillable gaps and Left as is.
  2. Which gaps are fillable. Any gap that is not a weekend, a holiday or a scheduled closure, up to the maximum gap length you set (blank: no limit, with a safety ceiling of 100,000 bars). Weekends, holidays and closures are recognised and left alone. Stretches stored at a coarser resolution than M1 are never touched.
  3. Fill them into a separate symbol. Fix gaps writes a <symbol>_patched symbol with flat, zero-volume bars at the previous close (an optional, marked synthetic fill for genuine outages is described in the GapSmith manual), imported through StrategyQuant’s own import so it inherits the original’s spread, commission and swap. The original is unchanged and an audit log lists every filled minute.
  4. Decide which symbol you build on. Use the same one for Edge Finder and the Build task. If a result looks too good around a patched stretch, test on both.
  5. Export the broker’s specifications. Drag AWSpecExport onto a chart in the terminal logged in to your broker. It writes two CSV files to Common\Files\AWSpecs and changes nothing in the terminal.
  6. Compare with a broker profile. Open the SpecSmith tab, choose the export and the profile, and press Compare with MetaTrader. Rows show Differs, Matches or Not in SQX.
  7. Apply what you want. Tick rows and press Apply ticked rows. Swap is never positive unless you allow it, spread stays off unless you ask, and a backup is saved before anything is written.
  8. Set the broker’s timezone yourself. SpecSmith never guesses it, because MetaTrader gives only today’s UTC offset. Its timezone check tests your value against the weekly reopening of a currency pair and reports whether it fits.
  9. Measure. In Edge Finder, the Data quality panel still checks the symbol each time you open the module.

What each step is for

Holes matter because tick-precision and M1 simulation build the price path from M1 bars: a stop or limit inside a missing stretch fills at the next price that exists, not where it would have triggered. A higher-timeframe chart rarely notices, and a tick-precision backtest does. Contract details matter because the same strategy costs one thing in the tester and another on the account when tick size, point value, stop distance, swap, commission or session differ.

What to check by hand afterwards

Run the exported EA in the Strategy Tester and compare its trades with the SQX backtest on the same history. The parity checklist says what to compare: instrument description, history, clock, files, indicators, closed bars, frozen versus live data, costs and fills, licence.

Limits

  • Filled minutes are flat bars by default, or marked synthetic bars if you choose them; neither is what the market did.
  • GapSmith patches M1 only.
  • SpecSmith copies what the terminal reports at one moment. Projects created earlier keep their own copy until you redeploy them.
  • Neither tool downloads price history or merges data sources.

Questions

Which should I do first, SpecSmith or GapSmith?

Either. They fix different things and do not depend on each other.

Do I need MetaTrader 5 for this?

GapSmith does not. SpecSmith exports from a terminal.

Will GapSmith change my strategies?

No. It writes a new symbol and leaves the original as it was.

Risk

Trading leveraged instruments carries a high risk of loss. Software output is not investment advice and says nothing about future results. Validate every strategy yourself before risking capital.