[Quant Trading System 3/5] Why no_trade Baselines Made Positive Validation More Suspicious

The Purpose Of The First Baselines

Once the training matrix exists, the next temptation is a model. In this quant trading system, I ran deterministic baselines first. The reason was simple: if human-readable rules all fail after costs, putting ML on top of the same surface can become more dangerous, not less.

Baselines can search for profit, but their more important role is to provide a stop signal. If common rules fail, feature IC sends strange warnings, and candidate selection keeps choosing no_trade, the conclusion should not be “train a more complex model.” It should be closer to “do not train yet.”

The first baseline set was deliberately conservative.

test split excluded
chronological train/validation only
no overlapping positions
max 5 trades per day
cost-adjusted net return
no_trade always available as a candidate

The no-overlap and max-trades-per-day rules matter. A research signal may look active every minute, but real trading decisions are sequential. If a position is already open, the next trigger cannot be traded. Trading unlimited signals in a day is also unrealistic. So the baselines included some execution-like constraints from the beginning.

Walk-Forward Baseline

The first walk-forward baseline evaluated simple deterministic scores sequentially inside each day. The hold period was five minutes, each strategy could trade at most five times per day, and the test split was excluded.

The result was cold:

selected trade rows: 46,879
validation folds: 10
test trades: 0
maximum trades in one day: 5
top validation baseline: no_trade

I liked this result because the harness did not force an active rule to be selected. Many backtesting scripts present the least bad active rule as if it were the best strategy. In a trading system, doing nothing may be the best choice. The baseline harness has to allow that choice naturally.

Candidate Policy: The First Feature-Only Failure

The next step was a feature-only candidate policy. It used training-matrix features to create mean-reversion, momentum, and trend-pullback rules, then checked whether any policy was positive on both train and validation.

The rules were simple, but the boundaries were strict. Policies must not reference label columns, must not load the test split, and must respect no-overlap plus the daily trade cap.

POLICY_FEATURES = (
    "mes_return_5m",
    "mes_bb_20m_zscore",
    "mes_rsi_14m",
    "mes_vwap_session_diff_pct",
    "es_return_5m",
    "mes_es_return_5m_spread",
)

def validate_policy_features() -> None:
    label_overlap = sorted(set(POLICY_FEATURES) & set(LABEL_COLUMNS))
    if label_overlap:
        raise RuntimeError(f"Candidate policies reference label columns: {label_overlap}")

The first candidate policy also selected no_trade.

Metric	Value
Active policy rows scanned	95,856
Test trades	0
Max trades per day	5
Overlap count	0
Best active validation policy	trend_pullback_score_2p5
Best active validation net points	-360.97

A result this poor is at least easy to interpret. The feature-only heuristic did not beat costs. If I trained a model at this point, it would probably find some noisy feature combination. But that would be closer to the start of overfitting than the start of a strategy.

The Warning From Feature IC

I then built a feature IC report. In the spirit of Alphalens or Qlib, it looks at rank correlation, quantile spreads, turnover, and decay between features and forward returns. The test split stayed excluded.

The numbers were interesting:

input rows: 281,145 train/validation rows
features evaluated: 47
daily IC rows: 49,491
strongest validation mean absolute IC: mes_sma_5m, 0.214682

But this should not be read as an alpha claim. The strongest features were mostly raw price, SMA, and VWAP-level features. That can be a normalization warning. Futures rolls, price levels, and session behavior can make rank correlations look useful without implying a tradable signal.

So the feature IC conclusion was not “I found a good feature.” It was closer to “feature normalization and benchmark design need more suspicion before modeling.” The better a feature looks, the more I need to explain why.

Strategy Zoo: Rebuilding External Ideas Locally

After that, I built a strategy zoo. The goal was to reimplement common GitHub strategy families on top of the local data contract: SMA/EMA crossovers, MACD, RSI reversals, Bollinger mean reversion and breakout, Donchian breakout, and opening-range breakout/failure.

The important thing was not that these ideas came from outside. It was that every outside idea had to pass through the same local gate. GitHub stars, README backtests, and attractive charts are not evidence in this system. Everything must be rebuilt on local MES/ES data, costs, no-overlap rules, and train/validation splits.

The strategy zoo remained conservative:

Metric	Value
Input rows	281,145
Strategies evaluated	46
Strategy trades scanned	141,528
Validation strategy trades	24,409
Test trades	0
Selected strategy	no_trade
Best active validation strategy	vol_expansion_breakout_short
Best active validation net points	-21.166

The fact that the best active validation strategy was still negative is a useful stop signal. One could say, “Maybe XGBoost can still find it.” But when deterministic families all fail to beat costs, any model-found pattern needs much stronger evidence.

The Lookahead Audit Passed, But The Strategies Failed

An interesting detail is that the data leakage audit looked clean at this point. The lookahead audit recomputed selected feature families from raw OHLCV prefixes and compared them with stored feature values.

audited features: 26
sampled rows: 216
feature observations: 5,616
mismatches: 0
max absolute difference: 4.96548580031e-10
forbidden future/label/rank/net feature-name tokens: 0

This matters. When baselines fail, there are two possible explanations. Either the code/data is broken, or the signal is weak. The audit is not final proof, but it does show that the features were not accidentally looking into the future to produce attractive results. And the results were not attractive anyway.

That combination leads to an honest conclusion: the data contract was getting cleaner, but a tradable edge was not visible yet.

no_trade Is Not A Lazy Answer

The results at this point all pointed in the same direction.

walk-forward baseline: no_trade
candidate policy: no_trade
strategy zoo: no_trade
test split used: 0 trades
model training: not started

no_trade is not a lazy answer. It is closer to evidence that the baseline system is working correctly. The system should not be forced to pick something. If active rules cannot beat costs and validation, doing nothing should win.

What I gained from this stage was not a strategy. I gained a standard. Any future candidate must at least beat this baseline set. And even if it looks good on validation, it still has to survive train, cost stress, concentration, and execution realism.

The model still had to wait. When baselines return a negative conclusion, ignoring that conclusion and escaping into a more complex model is not research. It is wishful thinking.

That clear stop signal did not end the research. It changed the job of the next experiments. Sensitivity analysis became less a search for something to promote and more a pressure test of whether the first positive validation rows could still be rejected under the rules already in place.

When The First Good-Looking Numbers Appeared

This was the first point where attractive validation numbers appeared. A 30-minute horizon row produced 175.124 validation net points, and exit-label sensitivity reached 256.048 validation net points. A session-extreme short row from the parameter sweep also showed a good-looking profit factor.

This is the dangerous moment. When a research loop that had been finding nothing finally prints a positive result, the human wants to explain it. Maybe the signal fits 30 minutes better than 5 minutes. Maybe the entry is fine and only the exit is wrong. Maybe a session filter can preserve the edge.

But the point of this phase was not to find positive validation rows. It was to test whether the system could keep rejecting them through train, concentration, cost-stress, no-overlap, and test-locking rules.

Horizon Sensitivity: 30 Minutes Looked Better, But Train Collapsed

The first horizon sensitivity pass kept the current roll rule and recomputed labels at 5m, 10m, 15m, and 30m. It also checked that the recomputed five-minute labels matched the strict matrix labels exactly.

horizon label rows: 1,082,461
strategy trade rows scanned: 160,845
test labels: 0
test trades: 0
5-minute label recomputation mismatches: 0

The most visible result was a 30-minute ES/MES spread row.

Metric	Value
Best active validation row	30m mes_vs_es_spread_gt_0p0001
Validation net points	175.124
Same row train net points	-2,894.096

On validation alone, this is interesting. But with train failing that badly, it is not a candidate. A train loss of -2,894.096 points is not a small weakness. It is much closer to a signal that happened to survive in validation.

The horizon sensitivity conclusion was therefore simple: the 30-minute horizon may be a research clue, but it is not ready for training or test evaluation.

Roll-Rule Sensitivity: Contract Selection Did Not Rescue The Structure

The next pass was roll-rule sensitivity. It compared the previous-volume leader, front contract, and same-day-volume leader. Same-day-volume leader was marked as a lookahead diagnostic only, because it uses full-day volume.

The result was even more conservative:

roll-rule input rows: 1,014,039
strategy trade rows scanned: 121,315
test rows used: 0
front-contract differs from previous-volume leader: 53 trade dates
same-day-volume differs from previous-volume leader: 17 trade dates
best non-lookahead active validation net: -140.334

Changing the roll rule did not make the five-minute baseline family work. Contract selection matters, but it is not a magic knob that turns a bad signal into a good one.

This also blocked a tempting mistake. Same-day volume can look useful, but if the full day’s volume is used for that same day’s entry decision, it is lookahead. A tradeable rule must choose contracts using only previous information.

Parameter Sweep: The Good-Looking Row Broke On Train

The parameter sweep reused strategy-zoo signals and varied trading windows plus score floors. It made the rule stricter by allowing at most one trade per sweep variant per day, and the test split remained excluded.

This produced the first plausible validation-positive row.

selected sweep: no_trade
sweep variants: 192
sweep trades scanned: 114,967
validation sweep trades: 20,117
best active validation sweep:
  session_extreme_reversal_short__skip_open_1030_1500__max1__score_ge_50p0
best active validation net points: 67.576
best active validation profit factor: 2.408655
same sweep train net points: -151.662

The validation result is tempting. Net points are positive and profit factor looks strong. But train is -151.662 points. This combination requires caution. A rule that fails on train and works only on validation may reflect regime change, but it may also be a selection artifact.

I did not promote it at this stage. I sent it into a deeper diagnostic pass. The important thing was that the validation winner did not immediately become a candidate.

Exit-Label Sensitivity: Exit Changes Almost Looked Like A Rescue

The most dangerous result came from exit-label sensitivity. For the existing OHLCV robustness leads, it tested fixed time exits, stop/target first-touch exits, and train-derived regime filters.

The code deliberately forced train-derived thresholds:

REGIME_FILTERS = (
    RegimeFilterSpec("no_filter", "All source trades."),
    RegimeFilterSpec("session_early_or_middle", "Session progress before late-third cutoff."),
    RegimeFilterSpec(
        "train_high_mes_volume_20m_zscore",
        "MES volume z-score is at or above the source-lead train 67th percentile.",
        "mes_volume_20m_zscore",
        0.67,
    ),
)

The result looked strong:

Metric	Value
Source exit rows	22,925
Variant trade rows	98,903
Active variants	147
Best validation variant	horizon_30m...time_exit_30m...train_high_mes_volume_20m_zscore
Best validation net points	256.048
Best validation profit factor	2.652178
Best validation top-5 day share	0.750012
Same variant train net points	-1,242.548

If I looked only at validation PnL, it would feel like a discovery. But a top-five day concentration of 0.750012 means too much of the result came from a few dates. And with train at -1,242.548, this is diagnostic evidence, not a candidate.

Another variant was subtler. After a direction-aware rebuild, a session-extreme short 15-minute exit looked positive on both train and validation under configured costs.

configured train net: 27.230
configured validation net: 34.558
dominant-active p99 stress train net: -1.520
all-clean p99 stress train net: -145.270

The important number is not 27.230. It is -1.520. The average train edge was so small that a p99 spread-stress adjustment pushed it negative. If a result cannot survive one tick of extra cost, it is not live-ready, and it is not even model-training-ready.

Validation Is A Question, Not An Answer

This phase changed how I read validation. Validation is not an answer. It is a question generator. Why did this work only here? Why did train fail? Did a few dates create the whole result? Does cost stress erase it? Is entry the problem or exit? Was this filter chosen after seeing validation?

A good research system should ask those questions before promoting a validation-positive row. Most rows will not survive them.

The final selection in this phase remained no_trade.

horizon sensitivity: no_trade
roll-rule sensitivity: no_trade
parameter sweep: no_trade
exit-label sensitivity: no_trade
test split touched: no
model training started: no

Why This Failure Was Useful

These results may look like failures, but they made the next research direction clearer. The problem was no longer just “the baselines are bad.” I could now describe how they were bad.

The 30-minute horizon works in validation but collapses on train. Changing the roll rule does not rescue the structure. Session-extreme short looks good on validation but is negative on train. Exit changes briefly improve results, but concentration and cost stress reject them.

After decomposing the failures this far, not training a model becomes easier. ML is not a magic rescue tool for weak deterministic signals. If I train without understanding why a weak signal is weak, I risk wrapping a validation artifact in a more convincing function.

So the conclusion stayed the same. Do not trade yet. Do not open the test split yet. Do not train yet. Build better questions.