Of 10 validated tools, only 3 are clean, non-marginal, non-degenerate wins — and even those are relative to a base signal that itself loses money. This page gives the honest reading of every result that looks good, every rejection worth remembering, and the engineering lessons the project paid for.
All three are risk transforms on an already-chosen book of directional bets, not standalone money-makers. They prove a tool can add real value by changing how much and in which direction you express an edge — not by manufacturing an edge where none exists.
An uncapped 8-symbol book of independent per-asset MA-crossover signals reaches 7.8× mean gross exposure and a −97.6% out-of-sample max drawdown. Capping gross/per-asset exposure cuts the drawdown to −35.3% with Sharpe/Sortino essentially unchanged — matching the tool's own hypothesis ("minimal drag on Sharpe") almost exactly.
The same 7.8× book, sized down by a rolling correlation-weighted netting factor instead of a hard cap. Drawdown cut from −97.6% to −39.0% and Sharpe/Sortino improved (e.g. −0.25 → −0.01 median) — the only validated tool that improved returns alongside risk.
Vol-scaled sizing beats fixed full-size sizing on Sharpe, Sortino and max drawdown simultaneously — the only clean pass on all three legs. Caveat: the underlying MA-crossover base signal is itself unprofitable, so this shows sizing helps relative to fixed sizing on a losing signal, not that it makes money outright.
| Tool | Label | The honest story |
|---|---|---|
| HMM regime filter | marginal | Classification accuracy beats the naive baseline by only 1.7 points (0.756 vs 0.739), and gating made Sharpe worse (−0.29 vs −0.24). It passes only because the regime-filter bar is an OR across Sharpe/drawdown and drawdown improved a hair. |
| ATR (Volatility) | marginal | An ATR stop beats a fixed-% stop calibrated to the same average width — but both stop mechanisms are far worse than no stop at all (unstopped Sharpe +0.34 → about −0.39 stopped). The narrow "ATR beats fixed-%" claim is true; "ATR stops help" is not. |
| ATR Stop-Loss (Risk Mgmt) | duplicate | A literal re-run of the Volatility element's ATR tool under its own catalogue row. Same finding twice. |
| Volatility Targeting (Pos Sizing) | duplicate | Mechanically identical to the validated Realized-Vol tool (same formula), with the target-vol level added to the swept grid. |
| Kelly Criterion | degenerate | The position is nonzero only ~6% of the time, average leverage ~0.035 when active. The rolling win-rate/payoff estimate on the losing base signal is negative almost always, so the rule clips to flat. "Validated" because it mostly doesn't trade — the safest possible way to not lose money. |
| PCA stress filter | user override | Fails the accuracy-first rule (0.622 vs 0.778 naive baseline) — a hard Rejected under normal logic — but gating an equal-weight basket with its label more than doubled Sharpe (0.32 → 0.80) and cut drawdown (−20.2% → −16.8%). Shown both numbers explicitly; the user chose Validated on the economic effect. A deliberate, reviewed exception, not a licence to drop the accuracy bar (the very next tool, Rolling Correlation, failed accuracy and showed the opposite economic effect). |
| Spread & Commission Tracking | nuanced | Real, measurable gross-vs-net divergence — but its materiality is turnover-dependent. The project's own funding-cycle result (Sharpe −5.46 on 539×/yr turnover) is the corroborating proof that costs dominate mechanically-triggered high-frequency rules. |
The funding-history bug cost the most: an endpoint returning an empty frame (start date predating the listing) was silently treated as "no funding data" for the entire project's early backtests — ~12.9%/yr of real cost never applied. It was caught by building the funding-dependent Carry tool and sanity-checking the fetch. The fix also guarded a second truncation trap (short mid-history batches). All 16 affected tools were re-run; zero conclusions changed.
Kelly (position on ~6% of bars) and the Max-Drawdown Circuit Breaker (nonzero ~6–11% of bars) both looked passable or near-passable on pass-bar output alone. The breaker's underlying signal is so persistently drawdown-prone (−83.5%) that once triggered it never recovers, locking in near-permanent de-risking. A one-line (position != 0).mean() check catches this in seconds — now routine for every risk-input tool.
Exposure Caps' per-asset cap of 0.25 on 8 symbols exactly equalled the tightest gross-cap grid value (2.0), making all three gross-cap cells produce identical results. Sanity-check that fixed values in a grid don't coincide before trusting the sweep's apparent variation.
Benjamini-Hochberg across all 17 p-value-bearing tools drew the exact same significance line as the flat p<0.05 check (6 tools significant either way, 11 either way). More importantly, no tool was ever rejected on the significance criterion alone — every rejection also failed Sharpe/IC/regimes independently. The correction closed the last unexecuted spec item with zero status changes.
The Market Impact test used next-bar |return| as a realized-impact proxy — hundreds of bps of ordinary daily volatility standing in for single-digit bps of real per-trade impact. Both the sqrt-law model and the flat baseline were inflated ~100×; the rejection is therefore "proxy-limited" rather than a verdict on the square-root law.
The Sortino sign-inversion fix flipped Realized Vol from a false Rejected to Validated. The project's rule: re-run everything affected, state the change, and let the user re-review. Data accuracy > speed, always.
Classic directional signals on liquid crypto perps do not survive costs out-of-sample — if you are trading them, you are trading noise and fees. What does survive is risk management: capping exposure, netting correlated bets, and targeting volatility turned a catastrophic book (−97.6% drawdown) into a survivable one with the same or better return profile. And 17 catalogue entries are honest unknowns — tools that deserve a test the day tick data, point-in-time fundamentals, or options data enters the pipeline.
Every number on this site is a grid median with a caveat attached. The few "validated" labels that look too good (Kelly) and the ones that look too bad (funding cycle) both repay reading the footnotes.