RSJ Portfolio Portfolio Optimization

← All posts

2026-11-04

How to Check That a Portfolio Optimizer Is Not Overfitting

How to Check That a Portfolio Optimizer Is Not Overfitting
portfolio optimization modern portfolio theory portfolio correlation portfolio optimization hierachical risk parity minimize volatility minimize semivariance maximize sharpe

How to Check That a Portfolio Optimizer Is Not Overfitting

You ran the optimizer, you got weights, and they look wonderfully precise: 11.4% here, 7.8% there, a volatility figure with two decimal places. That precision is exactly the problem. A mean-variance optimizer will happily fit the noise in a historic end-of-day price series and hand you back a portfolio that describes the past beautifully and the future not at all.

This article is about the decision that comes after the run: how to check that a portfolio optimizer is not overfitting before you commit capital to its output. Not with a single number, and not with a promise from the vendor, but with a short, repeatable verification workflow. We use RSJ Portfolio as the concrete tool throughout, because its workflow exposes the objective, the risk estimator, and the weight constraints — the three levers where overfitting hides.

Why Verification Matters Before You Trust an Allocation

An overfit allocation is not a wrong answer to the question you asked. It is a correct answer to a different question: which weights best explain this particular sample of prices? Modern portfolio theory will solve that problem whether or not the sample contains signal.

The symptoms are easy to miss because the output looks professional:

  • Weights are concentrated in a handful of names for no reason you can articulate.
  • The reported risk is low, but only on the exact date range you happened to choose.
  • The result is attractive under one configuration and unremarkable under every other one.

Verification is a workflow, not a metric. Four things have to hold at once: the inputs are fixed and known, the method matches the claim, the weights are stable under small perturbations, and the result survives a period the optimizer never saw. Skip any one of them and you are guessing.

The reason a desktop tool beats a notebook here is that a notebook lets you silently change three things between runs. RSJ Portfolio separates configuration, selection, and recommendation into distinct steps, so you can change one parameter at a time and still know what you changed.

What a Non-Overfit Result Looks Like

Before testing, define the target. A defensible allocation has four properties.

1. It is stable under small changes. Perturb the date range by a quarter, swap the risk model, tighten a weight constraint — the portfolio should drift, not flip. If 40% of the capital migrates from one stock to another because you moved the window by three months, the optimizer is describing noise.

2. The method matches the claim. If you asked for minimum volatility, the result should be explainable by the correlation structure of the candidates. You should be able to point at the weights and say “these names correlate weakly, so together they move less.” If instead the answer is “these three stocks happened to have a good run in 2021,” the objective was not really minimum volatility; it was hindsight.

3. It survives a holdout period. Fit on one window, evaluate on a later window the optimizer never touched. This is the closest thing to a proof you will get from historic data alone.

4. It is interpretable in the product’s own logic. RSJ Portfolio allocates to candidates that correlate weakly — or whose downside correlates weakly — which reduces the volatility of the resulting portfolio. If your result cannot be described in those terms, treat it as suspect until you can explain it. (For the underlying idea, see How to Reduce Portfolio Volatility with Low Correlation Stocks.)

Step-by-Step Verification with RSJ Portfolio

The verification below assumes you already have a candidate list. If you are starting from zero, the getting started guide lists the recommended starting parameters; set those first, then come back and stress them.

Step 1 — Fix the inputs and freeze them

In the configuration step, set the data provider, the local currency, the date range, and the cleaning settings. Then write them down and do not touch them for the rest of the exercise.

This matters more than it sounds. The data and currencies page documents the price cleaning pipeline, the local cache, and the conversion path for portfolios quoted in several currencies. Each of those is a legitimate source of variation — and therefore a legitimate way to accidentally fool yourself if you change it mid-test. If you have already automated the price side, the workflow in Automate Portfolio Optimization with Cached End-of-Day Prices is a good place to keep the input layer boring.

Keep your runs in dated folders so an export is never ambiguous:

mkdir -p runs/2026-11-04/baseline \
         runs/2026-11-04/objective-semivariance \
         runs/2026-11-04/risk-model-2 \
         runs/2026-11-04/tight-constraints

Step 2 — Run the baseline and record everything

Build the candidate list, choose a stated objective, and produce a recommendation. For a first pass, minimum volatility is the least ambiguous choice: the claim is narrow and easy to falsify.

Do not just screenshot the pie chart. The recommendation page reports the optimizer parameters alongside the resulting allocation, which is precisely the pairing you need. Record it in a run log:

run_id,objective,risk_model,date_from,date_to,max_weight,volatility,notes
baseline,minimum_volatility,<model 1>,2019-01-01,2025-12-31,0.10,<reported>,frozen inputs
objective-semivar,minimum_semivariance,<model 1>,2019-01-01,2025-12-31,0.10,<reported>,same candidate list
risk-model-2,minimum_volatility,<model 2>,2019-01-01,2025-12-31,0.10,<reported>,same candidate list
tight-constraints,minimum_volatility,<model 1>,2019-01-01,2025-12-31,0.05,<reported>,cap halved

Use the exact labels the application shows; the placeholders above are only there to keep the schema honest.

Step 3 — Stress one parameter at a time

Now re-run the same candidate list and the same date range while changing exactly one thing per run:

  • The risk estimator. RSJ Portfolio offers eight risk models, and they do not agree with each other on the same prices. That disagreement is the test.
  • The objective. Minimum semivariance, conditional value at risk, and hierarchical risk parity are all available, and they weight the tails differently. Minimum Variance vs Hierarchical Risk Parity: Which Fits? covers when each is the better fit — here you only care whether the allocations resemble each other.
  • The weight constraints. Halve the per-name cap and see whether the portfolio rebalances smoothly or collapses into the same three names.

Write the test matrix down before you run it, so the exercise stays a test rather than a search for a configuration you like:

# test-plan.yaml — your own checklist, one change per run
candidate_list: core-40
date_range: 2019-01-01..2025-12-31
runs:
  - id: baseline
    change: none
  - id: risk-model-2
    change: risk_estimator
  - id: objective-semivariance
    change: objective
  - id: tight-constraints
    change: weight_constraints

Step 4 — Test out of sample

Shorten the fitting window — say, end it two years earlier — and run the recommendation again. Then apply the resulting weights, unchanged, to the excluded later period and see what actually happened.

You can do this with the exported weights and any price series you already trust:

import numpy as np
import pandas as pd

prices = pd.read_csv("holdout_prices.csv", index_col="date", parse_dates=True)
returns = prices.pct_change().dropna()

weights = pd.read_csv("runs/2026-11-04/baseline/weights.csv", index_col="symbol")["weight"]
weights = weights.reindex(returns.columns).fillna(0.0)

portfolio = returns.mul(weights, axis=1).sum(axis=1)
print(f"holdout annualised volatility: {portfolio.std() * np.sqrt(252):.2%}")
print(f"holdout cumulative return:     {(1 + portfolio).prod() - 1:.2%}")

The comparison that matters is fitted-window volatility versus holdout volatility. A portfolio whose risk roughly doubles once the optimizer loses sight of the data was fitted to that data, not to the market.

Step 5 — Compare the weight vectors directly

This is the step most people skip, and it is the cheapest one. Export the weights from each run and measure how far they moved.

import pandas as pd

base = pd.read_csv("runs/2026-11-04/baseline/weights.csv", index_col="symbol")
alt = pd.read_csv("runs/2026-11-04/risk-model-2/weights.csv", index_col="symbol")

joined = base.join(alt, how="outer", lsuffix="_base", rsuffix="_alt").fillna(0.0)
joined["abs_delta"] = (joined["weight_alt"] - joined["weight_base"]).abs()

print(f"one-way turnover between runs: {joined['abs_delta'].sum() / 2:.1%}")
print(joined.sort_values("abs_delta", ascending=False).head(10))

If you have ever built a covariance matrix and a quadratic program by hand — the hard way — you already know why this manual comparison beats rebuilding the whole pipeline in a notebook: the run-to-run budget stays reserved for the actual question, not for plumbing.

The concept chapter of the user guide lays out the statistical background you are testing against, which is worth reading once so your thresholds are grounded in the method rather than in intuition.

Interpreting the Results and Edge Cases

Turnover between runs needs a reference point. My rule of thumb: a change in one parameter that moves less than a fifth of the capital is unremarkable; a change that moves more than half means the allocation is not identified by the data, it is identified by the estimator.

Watch for these patterns:

  • Gradual drift — weights shift in the same direction as the constraint tightens. Consistent with a robust allocation.
  • A cliff — one small change and the portfolio is unrecognisable. The optimizer is fitting noise.
  • Concentration after a minor change — if a few names dominate once you swap the risk estimator, the result is being driven by that estimator’s quirks rather than by the correlation structure.
  • A single-model result — attractive under one of the eight risk models, mediocre under the rest. Treat the attractive version as the outlier.

Data configuration deserves its own re-check. Portfolios quoted in several currencies follow a conversion path, and different cleaning choices produce different input series. Any change there invalidates your earlier stability comparisons — re-run the matrix rather than assuming the conclusion still holds.

Finally, know what not to blame on overfitting:

  • Very short histories. Not enough observations to estimate a covariance matrix; every optimizer will look unstable.
  • Illiquid candidates. Stale prices manufacture fake low correlations.
  • An internally correlated candidate list. If your universe is twenty semiconductor stocks, no method will produce a diversified answer.

These are data limitations. Labelling them overfitting sends you tuning parameters that were never the problem.

FAQ

How many parameter variations do I need to run before I can say a portfolio optimizer is not overfitting?

There is no fixed number. A practical check is to vary the objective, the risk estimator, and the weight constraints one at a time and confirm the allocation changes gradually rather than flipping. In RSJ Portfolio you can re-run the recommendation with a different objective or risk model while keeping the candidate list and date range fixed, which makes the comparison direct.

Is a single out-of-sample test enough to rule out overfitting?

A single holdout period is a useful first check but not a proof. It tells you whether the weights fitted on one window still make sense on a later window. Combining it with parameter stability checks gives a stronger signal, because an allocation that is both stable under parameter changes and reasonable out of sample is less likely to be an artifact of the historic price series.

Does the choice of risk model in RSJ Portfolio affect whether the result looks overfit?

Yes. The product offers eight risk models, and different estimators react differently to the same historic prices. If the allocation is only attractive under one specific risk model and collapses under the others, that is a warning sign. Re-running the same candidate list with a different risk model is one of the simplest stability checks available in the workflow.

Can RSJ Portfolio tell me directly whether my portfolio is overfit?

No tool can certify that. RSJ Portfolio builds a statistically optimized portfolio from historic end-of-day prices and exposes the objective, risk estimator, and weight constraints so you can run the verification steps yourself. The user guide’s concept chapter explains the statistical background, and the recommendation page reports the parameters and resulting allocation you need for the checks.

The Verdict Is Yours to Produce

Overfitting checks are not a feature you buy; they are a discipline you apply. Freeze the inputs, run a baseline with a stated objective, perturb one parameter at a time, hold out a period, and compare weight vectors rather than charts. Four runs and one holdout will tell you more about an allocation than any single risk number.

RSJ Portfolio is built for exactly this loop: a Windows desktop application that turns historic end-of-day prices into a statistically optimized portfolio, with the objective, the risk estimator, and the weight constraints all selectable — and the parameters reported next to the result. Install it under the single-user license, set your starting parameters from the getting started guide, and run your first stability matrix on a candidate list you already understand. The optimizer will still be the same one; the difference is that you will know what its answer is worth.

Related posts

EU label: AI-generated content