Abstract

Financial machine learning faces a fundamental challenge: markets are non-stationary, and models trained on historical data inevitably degrade as market dynamics shift. Traditional approaches of periodic full retraining discard valuable learned representations and require substantial computational resources. This paper presents a production-validated 3-Phase Transfer Learning (TL) Protocol specifically designed for cryptocurrency signal prediction, addressing the dual objectives of maintaining predictive performance across market regime changes while preventing catastrophic forgetting of proven alpha patterns.

The protocol implements a frozen OLD model baseline combined with an adaptive NEW model ensemble, achieving weekly incremental updates in approximately 60 minutes (full 8-step pipeline) with the core training step completing in ~20 minutes. Through a combination of Boruta-based feature selection, exponential sample weighting, and warm-start tree addition, the system maintains stable performance across diverse market conditions. Walk-Forward Validation with stationary bootstrap provides robust out-of-sample estimates with lower variance than simple holdout approaches. The methodology has been deployed in production cryptocurrency trading since November 2025, demonstrating consistent Information Coefficient (IC) values in the 0.05-0.10 range across multiple instruments while operating with sub-5ms inference latency.

Key contributions include: (1) a formal 3-phase protocol separating validation, production training, and weekly updates; (2) model-specific transfer strategies for tree ensembles, gradient boosters, and linear models; (3) locked feature sets preventing subtle drift; and (4) a 4-tier decision framework for adaptive deployment based on comparative holdout validation.

Implemented in Trade-Matrix

This section documents Transfer Learning capabilities deployed in production as of November 2025.

Three-Phase TL Protocol

Phase 1: Validation (75/25 holdout, ONE-TIME)

  • IC threshold: ≥ 0.05 required for approval
  • p-value: < 0.15 (relaxed for small samples)
  • Boruta feature selection (locks 9-13 features per instrument)
  • 200-bar purge gap prevents data leakage
  • Stationary bootstrap (100 samples) for robust IC estimation
  • Decision: APPROVE or REJECT for production deployment

Phase 2: Production Training (100% data, ONE-TIME)

  • Full training with locked features from Phase 1
  • No holdout (validation already completed)
  • MLflow model registration with comprehensive metadata
  • Ensemble weighting: 0.4 × OLD + 0.6 × NEW
  • OLD models frozen (trained on 2022-09 to 2025-07)
  • NEW models trained on full dataset with sample weighting

Phase 3: Weekly Updates (~62 min automated pipeline, ONGOING)

  1. fetch-ohlcv-data (~3 min) - Retrieve latest market data from Bybit
  2. build-features-only (~5 min) - Feature engineering with locked feature set
  3. train-tl-only (~20 min) - Transfer Learning training step 3.7. retrain-regime-models (~2 min) - MS-GARCH regime model retraining (v1.9.5)
  4. run-boruta-only (~5 min) - Validate features unchanged (skip training)
  5. train-rl-only (~15 min) - RL position sizing agents
  6. generate-signals-only (~3 min) - Generate trading signals
  7. run-backtest-only (~5 min) - Performance validation
  8. export-mlflow-only (~1 min) - Export models to MLflow registry

Automated Execution: GitHub Actions workflow every Sunday 6 AM UTC

Workflow Integration: TL and Regime Models

The weekly workflow maintains two independent model pipelines that combine at inference time:

Sequential Training Order:

  1. TL Models First (Step 3): Direction prediction models train on full OHLCV history
  2. Regime Models After (Step 3.7): MS-GARCH volatility models train on rolling 1200-bar window
  3. Both outputs used at inference: TL predicts direction, regime adjusts position sizing

Why Sequential, Not Parallel:

  • TL models require complete feature store (built in Step 2)
  • Regime models use the same 4H OHLCV data but with different preprocessing
  • Step 3.7 outputs (models/regime/*.pkl) must exist before signal generation (Step 6)

Independence Property: The models are mathematically independent - TL models do not use regime state as a feature, and regime models do not use TL predictions. This separation:

  • Prevents circular dependencies
  • Allows independent validation and debugging
  • Enables fallback to Kelly baseline if regime detection fails
# At inference time (simplified)
tl_signal = tl_model.predict(features)           # P(up) in [0, 1]
regime = regime_model.classify(volatility_data)  # BULL/NEUTRAL/BEAR/CRISIS
kelly_fraction = KELLY_MAP[regime]               # 0.67/0.50/0.25/0.17
position_size = base_size * kelly_fraction * tl_signal

Production Transfer Learning Strategies

1. Warm-Start (Tree Ensembles) - Primary strategy for RandomForest

  • Freeze OLD model (100 trees)
  • Add 50-250 new trees trained on recent data
  • Grid search n_new_estimators on Tune block
  • Total ensemble: 100 (frozen) + 50-250 (new) trees

2. Booster Continuation (XGBoost/LightGBM)

  • Extract OLD booster state
  • Continue training with reduced learning rate (lr_factor: 0.1, 0.3, 0.5)
  • Add 25-75 new boosting rounds
  • Early stopping on Tune block prevents overfitting

3. Elastic Weight Consolidation (Linear Models)

  • Ridge/Lasso/ElasticNet support
  • Regularize toward OLD parameters: L=MSE+λ2∣∣θ−θOLD∣∣2\mathcal{L} = \text{MSE} + \lambda_2 ||\theta - \theta_\text{OLD}||^2
  • Grid search λ2∈[0.1,0.3,1.0,3.0]\lambda_2 \in [0.1, 0.3, 1.0, 3.0] on Tune block
  • Prevents catastrophic forgetting for linear models

Production Feature Engineering

Boruta Selection (Phase 1 only, locked thereafter)

  • Shadow feature algorithm for statistical significance testing
  • 9-13 features selected per instrument (BTCUSDT: 9, ETHUSDT: 13, SOLUSDT: 11)
  • Features locked after Phase 1 to prevent drift
  • Stored in locked_features.json with exact order

Sample Weighting Strategy

  • Exponential decay: w(t)=exp⁡(−λ⋅agedays)w(t) = \exp(-\lambda \cdot \text{age}_\text{days})
  • λ=0.005\lambda = 0.005 (139-day half-life)
  • Recent month: ~85% weight, 3 months: ~64%, 6 months: ~41%, 1 year: ~16%
  • Alternative: regime-based weighting (historical: 1x, 2025: 2x, post-break: 5x)

Production Validation Metrics

Key Production Metrics (ranges for IP protection):

  • IC Range: 0.05-0.10 (typical for cryptocurrency markets)
  • Inference Latency: <5ms (sub-millisecond critical path)
  • Training Time: ~20 min per instrument (Step 3: train-tl-only)
  • Full Pipeline: ~60 min (8 automated steps)
  • Ensemble Weights: 0.4 × OLD (stability) + 0.6 × NEW (adaptation)

October 2025 Regime Recovery Case Study:

  • OLD Baseline alone: IC degraded to <0.02 (never recovered)
  • Full Retrain: 4-6 weeks recovery time
  • TL Ensemble: 1 week recovery (IC 0.05-0.10 restored)

Production Integration

MLflow Model Registry:

  • Comprehensive experiment tracking
  • 4-tier resilient loading (Registry → Run ID → S3 → Local)
  • Locked features logged as artifact (CRITICAL for feature order validation)
  • Model versioning with production promotion workflow

Feature Order Validation:

  • Sklearn validates feature names AND order at prediction time
  • Mismatch causes silent incorrect predictions
  • locked_features.json enforces exact training order
  • Validation checks OLD/NEW feature compatibility before ensemble

4-Tier RL Position Sizing Integration:

  • Tier 1: FULL_RL (100% RL) - High confidence (≥0.50) + IC (≥0.05)
  • Tier 2: BLENDED (50% RL + 50% Kelly) - Medium confidence/IC
  • Tier 3: PURE_KELLY (100% Kelly) - Low confidence or IC failure
  • Tier 4: EMERGENCY_FLAT (0% position) - Circuit breaker OPEN

Regime-Aware Position Sizing

The system integrates TL signal predictions with MS-GARCH regime classification to compute final position sizes. This two-model architecture separates concerns: TL models predict direction, while regime models adjust exposure.

Model Responsibilities:

Model Output Role
TL Ensemble P(up) ∈ [0, 1] Direction prediction
MS-GARCH Regime ∈ {BEAR, NEUTRAL, BULL, CRISIS} Volatility classification

Kelly Fraction by Regime:

Regime Volatility State Kelly Fraction Rationale
BULL Low volatility, positive trend 67% Maximize exposure in favorable conditions
NEUTRAL Stable volatility, no clear trend 50% Balanced exposure
BEAR Rising volatility, negative trend 25% Reduce exposure during drawdowns
CRISIS Extreme volatility, correlation breakdown 17% Preserve capital

Position Sizing Formula:

position=base_size×fKelly(regime)×ML_signal\text{position} = \text{base\_size} \times f_\text{Kelly}(\text{regime}) \times \text{ML\_signal}

Where:

  • base_size: Maximum allowed position from risk parameters
  • f_Kelly(regime): Regime-specific Kelly fraction (0.17 - 0.67)
  • ML_signal: TL ensemble output scaled to [-1, 1] for direction

Real-Time Regime Detection:

The RealtimeRegimeDetector actor uses single-step Hamilton filter updates for sub-millisecond regime classification:

# services/regime_detection/realtime_regime_detector.py
class RealtimeRegimeDetector(Actor):
    """
    2-regime MS-GARCH with 4-state Kelly mapping.
    Hamilton filter update: O(K^2) where K=2, &#x3C;15 microseconds.
    """

    def on_bar(self, bar: Bar) -> None:
        # Update regime probabilities
        regime_prob = self._hamilton_filter_update(bar)

        # Map 2-state MS-GARCH to 4-state Kelly
        if regime_prob[0] > 0.7:  # Low-vol regime dominant
            kelly_state = KellyState.BULL
        elif regime_prob[1] > 0.9:  # Extreme high-vol
            kelly_state = KellyState.CRISIS
        elif regime_prob[1] > 0.7:  # High-vol regime dominant
            kelly_state = KellyState.BEAR
        else:
            kelly_state = KellyState.NEUTRAL

        self._publish_regime(kelly_state)

Weekly Regime Model Retraining (Step 3.7):

# Retrain MS-GARCH parameters on rolling 1200-bar window
make retrain-regime-models

# Output: models/regime/{btcusdt,ethusdt,solusdt}_msgarch.pkl
# MLflow: Logs to 'regime_detection' experiment

Training script: scripts/ml/retrain_regime_models.py

What Is NOT Deployed (Research Only)

The following are documented in research sections but not implemented in production:

  • ❌ Multi-task learning - Future work for cross-instrument learning
  • ❌ Domain adaptation - Future work for cross-exchange transfer
  • ❌ Progressive neural networks - Requires neural architecture
  • ❌ Adapter fine-tuning (noted in Section 5.1) - Future work for LSTM/Transformer models
  • ❌ Full Fisher Information weighting - Simplified to EWC for linear models only
  • ❌ Online/continuous learning - Weekly batch updates only
  • ❌ Cross-asset transfer - Each instrument has independent models

Research & Future Enhancements

This section covers theoretical extensions and research directions documented but not implemented in production.

1. Introduction

1.1 The Challenge of Non-Stationarity in Financial Markets

Financial time series exhibit fundamental properties that challenge conventional machine learning approaches. Unlike domains such as image recognition where the underlying data distribution remains stable, financial markets continuously evolve as participants adapt, regulations change, and macroeconomic conditions shift. This non-stationarity manifests across multiple timescales: intraday patterns driven by algorithmic trading, weekly cycles influenced by options expiration, and longer-term regime shifts associated with monetary policy changes.

Cryptocurrency markets amplify these challenges. The 24/7 trading environment, relatively short history, and high retail participation create exceptionally dynamic conditions. A model trained on 2022 bear market data may perform poorly during a 2024 bull run, while a model optimized for high-volatility periods may generate spurious signals during consolidation phases. Traditional backtesting approaches that assume stationarity systematically overestimate future performance, leading to significant disappointment when models are deployed to production.

1.2 Why Periodic Full Retraining Fails

The naive solution to non-stationarity is periodic full retraining: discard the old model entirely and train a new one on the most recent data. While conceptually simple, this approach suffers from several critical weaknesses:

Catastrophic Forgetting: Full retraining destroys knowledge encoded from historical market conditions. A model trained during a 2025 bull market forgets the patterns it learned from the 2022 bear market, leaving it unprepared when bearish conditions return. Historical market data encodes valuable information about how assets behave under extreme conditions that may not be present in recent training windows.

Sample Inefficiency: Financial data is inherently limited. Even with 3+ years of 4-hour bars, we have approximately 6,500 samples per instrument. Full retraining that discards historical data to focus on recent periods operates with drastically reduced sample sizes, increasing variance and overfitting risk.

Computational Burden: Complete retraining requires regenerating all features, running hyperparameter optimization, and validating from scratch. For production systems requiring weekly updates, this overhead becomes prohibitive, potentially consuming hours of compute time for marginal improvements.

Validation Complexity: Each full retrain requires establishing new validation procedures. Without continuity, there is no baseline against which to measure improvement, making it difficult to distinguish genuine adaptation from noise fitting.

1.3 Transfer Learning as an Alternative Paradigm

Transfer Learning offers a principled framework for maintaining model performance while adapting to evolving conditions. Rather than discarding previous knowledge, TL preserves a frozen baseline model that encodes proven patterns while training an adaptive component on recent data. The ensemble of OLD and NEW models combines historical robustness with contemporary relevance.

This approach aligns with how successful quantitative traders operate: they maintain core strategies with long track records while continuously developing and integrating new signals. The frozen baseline serves as a strategic anchor, preventing wholesale abandonment of working approaches during temporary regime shifts.

The key insight is that market patterns exhibit persistence at varying timescales. Short-term microstructure patterns may change rapidly, but fundamental relationships between volatility regimes, momentum, and mean reversion tend to persist. A well-designed TL system can preserve the durable patterns while adapting the transient ones.

2. Problem Formulation

2.1 Mathematical Framework

Let Xt∈RdX_t \in \mathbb{R}^d represent the feature vector at time tt, and yt∈Ry_t \in \mathbb{R} represent the forward return target. We seek a function fθ:Rd→Rf_\theta: \mathbb{R}^d \rightarrow \mathbb{R} parameterized by θ\theta that maximizes the Information Coefficient (IC):

IC=CorrS(fθ(Xt),yt)IC = \text{Corr}_S(f_\theta(X_t), y_t)

where CorrS\text{Corr}_S denotes Spearman rank correlation, chosen for robustness to outliers prevalent in financial data.

The non-stationarity assumption states that the joint distribution P(Xt,yt)P(X_t, y_t) varies over time, creating a distribution shift between training period Ttrain\mathcal{T}_\text{train} and deployment period Tdeploy\mathcal{T}_\text{deploy}:

PTtrain(X,y)≠PTdeploy(X,y)P_{\mathcal{T}_\text{train}}(X, y) \neq P_{\mathcal{T}_\text{deploy}}(X, y)

2.2 Concept Drift and Regime Changes

We distinguish between two forms of distribution shift:

Gradual Drift: The distribution changes slowly over time, allowing models to maintain relevance with periodic updates. Example: slowly evolving correlations between Bitcoin and traditional risk assets.

Sudden Regime Change: The distribution shifts abruptly, typically associated with major market events. Example: the October 2025 Bitcoin ETF approval triggered rapid revaluation of crypto-equity correlations.

Both forms degrade model performance, but regime changes present the greater challenge as they can render models ineffective within days. The TL protocol must handle both scenarios.

2.3 The Catastrophic Forgetting Problem

Let θOLD\theta_\text{OLD} denote model parameters trained on historical data DOLD\mathcal{D}_\text{OLD}, and suppose we wish to adapt to new data DNEW\mathcal{D}_\text{NEW}. Standard gradient-based training minimizes:

L(θ)=∑(x,y)∈DNEWℓ(fθ(x),y)\mathcal{L}(\theta) = \sum_{(x,y) \in \mathcal{D}_\text{NEW}} \ell(f_\theta(x), y)

The resulting parameters θ∗\theta^* may satisfy θ∗≉θOLD\theta^* \not\approx \theta_\text{OLD} even for components where the OLD model performed well. This is catastrophic forgetting: the optimization process has no incentive to preserve previously learned patterns.

Elastic Weight Consolidation (EWC) addresses this by adding a penalty term:

LEWC(θ)=L(θ)+λ2∑iFi(θi−θOLD,i)2\mathcal{L}_\text{EWC}(\theta) = \mathcal{L}(\theta) + \frac{\lambda}{2} \sum_i F_i (\theta_i - \theta_{\text{OLD},i})^2

where FiF_i approximates the Fisher information, indicating which parameters are important for the OLD task. Our protocol implements similar principles adapted to tree-based ensemble models.

2.4 Success Metrics

We define success across multiple dimensions:

Metric Threshold Description
Information Coefficient (IC) ≥0.05\geq 0.05 Rank correlation between predictions and returns
p-value <0.15< 0.15 Statistical significance (relaxed for small samples)
Sharpe Ratio >0.5> 0.5 Risk-adjusted return from backtest
Maximum Drawdown <20%< 20\% Worst peak-to-trough decline
Feature Mismatch 0 Exact feature name and order match between OLD/NEW
Training Time ~60 min Total weekly pipeline duration (8 steps)

The IC threshold of 0.05 represents strong predictive power for cryptocurrency markets, where noise levels are substantially higher than in equity markets. For context:

IC Range Quality Interpretation
0.00-0.01 Noise No predictive power
0.01-0.02 Weak Marginal signal
0.02-0.04 Moderate Typical for crypto
0.04-0.06 Strong Excellent for crypto
0.06+ Very Strong Verify not overfit

3. Methodology: The 3-Phase TL Protocol

The Transfer Learning Protocol consists of three distinct phases, each serving a specific purpose in the model lifecycle.

3.1 Protocol Overview

            +---------------------------+
            |     Phase 1: Validation   |
            |        (ONE-TIME)         |
            |     Prove TL Works        |
            +------------+--------------+
                         |
                         | If IC >= 0.05
                         v
            +---------------------------+
            |   Phase 2: Production     |
            |        (ONE-TIME)         |
            |   Train on 100% Data      |
            +------------+--------------+
                         |
                         | Deploy to K3S
                         v
            +---------------------------+
            |    Phase 3: Weekly        |
            |       (ONGOING)           |
            |   Incremental Updates     |
            +---------------------------+
                  ^              |
                  |              |
                  +--------------+
                   Every Sunday
Phase Purpose Frequency Duration
Phase 1: Validation Prove TL approach works ONE-TIME ~20 min
Phase 2: Production Train on 100% data ONE-TIME ~10 min
Phase 3: Weekly Incremental updates Every Sunday ~60 min (full pipeline)

3.2 Phase 1: Validation

Phase 1 is the ONE-TIME validation that proves the Transfer Learning approach generalizes to unseen data. This phase must pass before proceeding to production deployment.

3.2.1 Data Split Strategy

The validation methodology employs a strict temporal split to ensure unbiased evaluation:

Timeline
|---------------------------------------------------------->
|  Training (75%)  |  Purge Gap  |   Holdout (25%)   |
|   2022-01       |   200 bars   |      2025-11      |
      ^                              ^
      |                              |
   Boruta Selection              NEVER SEEN
   OLD/NEW Training              during training

Key Principle: The holdout period (last 25% of data) is NEVER seen during training, Boruta selection, or hyperparameter tuning. This ensures unbiased validation.

3.2.2 The 200-Bar Purge Gap

The purge gap prevents data leakage from look-ahead bias in feature computation. At 4-hour bars, 200 bars equals approximately 33 days.

Why 200 bars? This value derives from the maximum lookback window used in feature computation:

  • Rolling rank features: 200-bar window for stable percentile rankings
  • Exponential moving averages: longest EMA uses 200-bar span
  • Volatility clustering: 200 bars captures typical regime persistence

Without the purge gap, features computed near the split boundary would contaminate information from the holdout period. This is a subtle but critical source of overfitting that many practitioners overlook.

3.2.3 Validation Workflow

def run_phase1_validation():
    """
    Phase 1: Transfer Learning Validation (ONE-TIME)
    Proves TL approach works using proper holdout validation.
    """
    for instrument in ['BTCUSDT', 'ETHUSDT', 'SOLUSDT']:
        # Load 3 years of data
        df_full = load_instrument_data(instrument)

        # Split: 75% training, 25% holdout
        split_idx = int(len(df_full) * 0.75)
        df_train = df_full.iloc[:split_idx].copy()
        df_holdout = df_full.iloc[split_idx:].copy()

        # Step 1: Run Boruta feature selection on TRAINING data only
        locked_features = run_boruta_selection(
            df=df_train,
            instrument=instrument,
            target='forward_returns',
            max_features=15  # Typically 9-11 selected
        )

        # Save locked features for Phase 2 and Phase 3
        save_locked_features(instrument, locked_features)

        # Step 2: Train OLD model on early portion of training data
        old_split = int(len(df_train) * 0.5)
        df_old = df_train.iloc[:old_split]
        old_model = train_model(df_old, locked_features)

        # Step 3: Train NEW model on all training data
        new_model = train_model(df_train, locked_features)

        # Step 4: Generate ensemble predictions
        X_holdout = df_holdout[locked_features].values
        old_preds = old_model.predict(X_holdout)
        new_preds = new_model.predict(X_holdout)
        ensemble_preds = 0.4 * old_preds + 0.6 * new_preds

        # Step 5: Validate on holdout (NEVER SEEN!)
        y_holdout = df_holdout['forward_returns'].values
        ic, p_value = spearmanr(ensemble_preds, y_holdout)

        # Decision gate
        if ic >= 0.05 and p_value &#x3C; 0.15:
            print(f"{instrument}: APPROVED for Phase 2")
        else:
            print(f"{instrument}: FAILED - use fallback model")

3.3 Phase 2: Production Training

Phase 2 trains production models on 100% of available data. Since Phase 1 already proved the TL approach generalizes, holding out data would waste valuable signal information.

3.3.1 Rationale for 100% Data Utilization

Why train on 100% data after validation?

  1. Phase 1 Already Validated: The holdout test proved the methodology generalizes. Repeating validation wastes data.

  2. Maximum Signal Power: More data improves model robustness and reduces variance. Each additional bar contributes to estimating feature importance more accurately.

  3. Institutional Standard: This follows the Renaissance Technologies principle: "Validate once, then maximize." Once you have confidence in your approach, you want the best possible model.

  4. Production Deployment: Models deployed to production need maximum predictive power to justify transaction costs.

3.3.2 OLD Model Baseline

The OLD models serve as immutable baselines containing historical market knowledge:

Instrument Training Period Purpose
BTCUSDT 2022-09 to 2025-07 Bull/bear cycle patterns
ETHUSDT 2022-09 to 2025-07 ETH-specific dynamics
SOLUSDT 2022-09 to 2025-07 Alt-coin behavior

CRITICAL: OLD models are NEVER modified. They contain proven alpha patterns from 3 years of market history across bull, bear, and ranging markets. Modifying them would destroy institutional knowledge and invalidate the Transfer Learning approach.

3.4 Phase 3: Weekly Incremental Updates

Phase 3 implements continuous model improvement through weekly incremental updates. The NEW model is retrained each Sunday while the OLD baseline remains frozen.

3.4.1 The 8-Step Weekly Pipeline

# Complete weekly update pipeline (every Sunday)
make weekly-tl-update

# Expands to 8 sequential steps:
# 1. fetch-ohlcv-data           (~3 min)  - Get latest market data
# 2. build-features-only        (~5 min)  - Feature engineering
# 3. train-tl-only              (~20 min) - Train TL models
# 4. run-boruta-only            (~5 min)  - Validate features unchanged
# 5. train-rl-only              (~15 min) - RL position sizing agents
# 6. generate-signals-only      (~3 min)  - Generate trading signals
# 7. run-backtest-only          (~5 min)  - Validate performance
# 8. export-mlflow-only         (~1 min)  - Export for deployment

Total weekly update time: approximately 60 minutes, fully automated.

3.4.2 Sample Weighting Strategy

Weekly updates use adaptive sample weighting to emphasize recent data while preserving historical context:

def compute_sample_weights(df, regime_break_date='2025-10-15'):
    """
    Compute adaptive sample weights for TL training.

    Weight Strategy:
    - Historical (pre-2025): 1x weight
    - 2025 pre-regime: 2x weight
    - Post-regime break: 5x weight
    """
    weights = np.ones(len(df))
    timestamps = df.index

    # 2025 data gets 2x weight
    mask_2025 = timestamps >= '2025-01-01'
    weights[mask_2025] *= 2.0

    # Post-regime break gets 5x weight
    mask_regime = timestamps >= regime_break_date
    weights[mask_regime] *= 2.5  # 2.0 * 2.5 = 5.0 total

    # Normalize
    weights = weights / weights.sum() * len(weights)

    return weights

The exponential decay formulation provides a continuous alternative:

w(t)=exp⁡(−λ⋅agedays)w(t) = \exp(-\lambda \cdot \text{age}_\text{days})

With λ=0.005\lambda = 0.005, data from 3 months ago receives approximately 64% of the weight of current data, balancing recency emphasis with historical preservation.

3.4.3 Warm-Start Architecture for Tree Ensembles

The Transfer Learning architecture uses XGBoost/RandomForest warm-start to add new trees while preserving OLD trees:

def transfer_learning_train(old_model, X_train, y_train, sample_weights):
    """
    Transfer Learning via warm-start tree addition.

    Architecture:
    - OLD model: 100 trees (FROZEN)
    - NEW trees: 50 additional trees (TRAINED)
    - Total: 150 trees in ensemble
    """
    # Clone OLD model
    tl_model = clone(old_model)

    # Enable warm-start
    tl_model.warm_start = True

    # Increase tree count (OLD: 100, NEW: +50)
    tl_model.n_estimators = old_model.n_estimators + 50

    # Train (only NEW trees get fitted)
    tl_model.fit(X_train, y_train, sample_weight=sample_weights)

    return tl_model

The first 100 trees remain exactly as they were in the OLD model. The additional 50 trees are trained on current data with sample weighting, learning to complement the existing predictions.

4. Feature Selection with Boruta

4.1 The Shadow Feature Algorithm

Boruta identifies statistically significant features by comparing their importance against randomized "shadow" features. The algorithm operates as follows:

  1. Create Shadow Features: For each original feature, create a shuffled copy (shadow feature). This establishes a null distribution of importance scores.

  2. Train Ensemble: Train a Random Forest on original + shadow features.

  3. Compare Importance: For each original feature, compare its importance score to the maximum importance among all shadow features.

  4. Statistical Test: Run binomial test to determine if feature consistently beats shadow features across multiple iterations.

  5. Classify Features: Features are classified as "Confirmed" (significant), "Rejected" (not significant), or "Tentative" (inconclusive).

from boruta import BorutaPy
from sklearn.ensemble import RandomForestRegressor

def run_boruta_selection(df, target='forward_returns', max_features=15):
    """
    Run Boruta feature selection to identify important features.
    """
    # Initialize Random Forest for Boruta
    rf = RandomForestRegressor(
        n_estimators=100,
        max_depth=5,  # Shallow to prevent overfitting
        random_state=42,
        n_jobs=-1
    )

    # Initialize Boruta
    boruta = BorutaPy(
        estimator=rf,
        n_estimators='auto',
        max_iter=100,
        random_state=42,
        verbose=2
    )

    # Run selection
    X = df[feature_cols].values
    y = df[target].values
    boruta.fit(X, y)

    # Extract confirmed features
    selected = [
        feature_cols[i]
        for i, selected in enumerate(boruta.support_)
        if selected
    ]

    return selected

4.2 Why 9-11 Features Per Instrument

Boruta consistently selects 9-11 features across instruments, a number that balances signal strength with parsimony:

Advantages of Parsimonious Feature Sets:

  1. Reduced Overfitting: Fewer features mean fewer opportunities to fit noise. With 6,500 samples and 10 features, we have 650 samples per feature, well above the minimum of 30 typically recommended.

  2. Interpretability: Each feature has clear economic meaning (momentum, volatility, volume). This allows for sanity checking and debugging.

  3. Stability: With fewer features, the model is less sensitive to minor variations in feature computation. This improves production robustness.

  4. Computational Efficiency: Feature computation, model training, and inference all scale with feature count. Parsimonious models are faster.

4.3 Locked Feature Sets for Consistency

CRITICAL: Boruta runs ONLY in Phase 1 on training data. The resulting locked_features.json is used unchanged in Phase 2 and all Phase 3 weekly updates.

{
  "feature_columns": [
    "vvix_change_1d",
    "momentum_rl_rank",
    "trend_strength_rank",
    "lagged_volume",
    "turnover",
    "volume_z",
    "obv_signal_rank"
  ],
  "boruta_run_id": "abc123def456",
  "training_timestamp": "2025-11-26T12:00:00Z",
  "instrument": "BTCUSDT"
}

Why lock features?

Re-running Boruta would invalidate the validation because:

  • Different features could be selected with slight data changes
  • NEW model would use different features than OLD model
  • Feature order could change, causing silent prediction errors (see Section 6)

The locked feature set ensures that OLD and NEW models operate on identical inputs, making their predictions directly comparable and safely combinable.

5. Model-Specific Transfer Strategies

The Transfer Engine implements four model-specific TL strategies, each optimized for the characteristics of different model architectures.

5.1 Strategy Selection

class TransferEngine:
    """
    Model-Specific Transfer Learning Strategies.
    """

    def __init__(self):
        self.supported_methods = {
            'warm_start': ['RandomForest', 'GradientBoosting', 'ExtraTrees'],
            'booster': ['XGBoost', 'LightGBM'],
            'ewc': ['Ridge', 'Lasso', 'ElasticNet'],
            'adapter_ft': []  # Future: LSTM, Transformer
        }

    def detect_tl_method(self, model):
        """Auto-detect appropriate TL method based on model type."""
        model_class = model.__class__.__name__

        for method, supported in self.supported_methods.items():
            if any(cls in model_class for cls in supported):
                return method

        # Fallback
        if hasattr(model, 'warm_start'):
            return 'warm_start'
        elif hasattr(model, 'get_booster'):
            return 'booster'
        else:
            return 'ewc'

5.2 Warm-Start Strategy (Tree Ensembles)

For RandomForest and similar ensemble methods, the warm-start strategy adds new trees to the existing ensemble:

Process:

  1. Clone OLD model
  2. Set warm_start=True
  3. Grid search n_new_estimators on Tune block
  4. Train with best hyperparameters
  5. Optional: Add calibration head

Hyperparameters Tuned:

  • n_new_estimators: [50, 100, 150, 200, 250]

The Tune block provides unbiased hyperparameter selection:

def warm_start_strategy(old_model, X_train, y_train, X_tune, y_tune, weights):
    """Warm-Start Strategy for Tree-Based Models."""

    old_n_estimators = old_model.n_estimators
    best_model, best_ic = None, -1

    # Grid search on Tune block
    for n_new in [50, 100, 150, 200, 250]:
        tl_model = clone(old_model)
        tl_model.warm_start = True
        tl_model.n_estimators = old_n_estimators + n_new

        # Fit on Train (OLD frozen, NEW trained)
        tl_model.fit(X_train, y_train, sample_weight=weights)

        # Evaluate on Tune (hyperparameter selection)
        pred_tune = tl_model.predict(X_tune)
        ic_tune, _ = spearmanr(pred_tune, y_tune)

        if ic_tune > best_ic:
            best_ic = ic_tune
            best_model = tl_model
            best_n_new = n_new

    return best_model, {'n_new': best_n_new, 'tune_ic': best_ic}

5.3 Booster Continuation Strategy (XGBoost/LightGBM)

For gradient boosting models, the booster continuation strategy extends training from the OLD booster:

Process:

  1. Extract OLD booster
  2. Grid search: learning rate reduction + n_new_rounds
  3. Continue training from OLD booster
  4. Use early stopping on Tune block

Hyperparameters Tuned:

  • lr_factor: [0.1, 0.3, 0.5] (reduce learning rate for fine-tuning)
  • n_new_rounds: [25, 50, 75]

Learning rate reduction is critical: training continuation at the original learning rate often overshoots and destroys OLD knowledge.

5.4 Elastic Weight Consolidation (Linear Models)

For linear models (Ridge, Lasso), EWC adds a penalty term preserving OLD weights:

L(θ)=MSEnew(θ)+λ1∣∣θ∣∣2+λ2∣∣θ−θOLD∣∣2\mathcal{L}(\theta) = \text{MSE}_\text{new}(\theta) + \lambda_1 ||\theta||^2 + \lambda_2 ||\theta - \theta_\text{OLD}||^2

The λ2\lambda_2 term penalizes deviation from OLD weights, preventing catastrophic forgetting.

Implementation via Data Augmentation:

def ewc_strategy(old_model, X_train, y_train, X_tune, y_tune, weights):
    """EWC Strategy for Linear Models."""

    theta_old = old_model.coef_
    n_features = X_train.shape[1]

    # Grid search lambda2 on Tune block
    best_model, best_ic = None, -1

    for lambda2 in [0.1, 0.3, 1.0, 3.0]:
        # Augment data with EWC penalty
        X_aug = np.vstack([
            X_train.values,
            np.sqrt(lambda2) * np.eye(n_features)
        ])

        y_aug = np.concatenate([
            y_train.values,
            theta_old
        ])

        weights_aug = np.concatenate([
            weights,
            np.ones(n_features)
        ])

        # Train with EWC penalty
        tl_model = Ridge(alpha=old_model.alpha)
        tl_model.fit(X_aug, y_aug, sample_weight=weights_aug)

        # Evaluate on Tune
        pred_tune = tl_model.predict(X_tune)
        ic_tune, _ = spearmanr(pred_tune, y_tune)

        if ic_tune > best_ic:
            best_ic = ic_tune
            best_model = tl_model

    return best_model, {'lambda2': best_lambda2, 'tune_ic': best_ic}

5.5 Foundation Model Transfer Learning

A fundamentally different approach to Transfer Learning has emerged with the rise of time series foundation models: pre-trained transformers that learn universal temporal patterns from billions of observations across diverse domains (weather, traffic, electricity, sales), then adapt to specific tasks like cryptocurrency forecasting through fine-tuning.

This paradigm mirrors the revolution that BERT and GPT brought to NLP: instead of training models from scratch on limited domain-specific data, we can leverage massive pre-training to capture fundamental temporal dynamics, then specialize to crypto markets with minimal additional training.

5.5.1 Pre-Trained Time Series Models

Three major foundation models emerged in 2024, each offering unique advantages for financial forecasting:

Chronos (Amazon Science, ICML 2024)

Chronos adapts the T5 language model architecture by treating time series as discrete token sequences. Continuous values are scaled and quantized into 4,096 discrete tokens, allowing sequence-to-sequence language modeling approaches to be applied to forecasting.

  • Architecture: T5 encoder-decoder with tokenization
  • Model Sizes: 20M (tiny) to 710M (large) parameters
  • Training Data: Diverse public datasets + synthetic Gaussian processes
  • Key Innovation: Language model tokenization for time series
  • Zero-Shot Performance: Matches or exceeds per-dataset tuned models on 42 benchmarks

Moirai (Salesforce AI Research, ICML 2024)

Moirai introduces "Any-Variate Attention" that handles forecasts across any number of variables (from univariate to hundreds of features) without architectural changes. Trained on LOTSA dataset containing 27 billion observations across 9 domains.

  • Architecture: Masked transformer with mixture-of-experts extension
  • Model Sizes: 14M (small), 91M (base), 311M (large) parameters
  • Training Data: 27B observations spanning energy, transportation, climate, sales, economics
  • Key Innovation: Any-variate attention mechanism for variable feature dimensions
  • Performance: Competitive zero-shot forecasting, 17% improvement with MoE variant

TimesFM (Google Research, ICML 2024)

TimesFM adopts a decoder-only architecture similar to GPT language models, treating forecasting as autoregressive generation with patching (groups of contiguous time points as tokens).

  • Architecture: GPT-style decoder-only with PatchTST-inspired patching
  • Model Size: 200M parameters
  • Training Data: 100 billion real-world time points
  • Key Innovation: Decoder-only design with extended 16K context (v2.5)
  • Performance: Rank #1 on GIFT-Eval for zero-shot forecasting (both point and probabilistic)

MOMENT (Carnegie Mellon, ICML 2024)

MOMENT is designed for multi-task time series analysis, supporting forecasting, anomaly detection, classification, and imputation with a single pre-trained model.

  • Architecture: Multi-task encoder architecture
  • Model Size: 125M parameters
  • Training Data: Time Series Pile (diverse tasks)
  • Key Innovation: Single model for forecasting, anomaly detection, classification, imputation
  • Advantage: Valuable for trading systems requiring multiple capabilities beyond forecasting

Comparison Matrix:

Model Parameters Training Data Zero-Shot Open Weights Key Strength
Chronos-Large 710M Diverse + Synthetic Yes Yes Language model approach
Moirai-Large 311M 27B observations Yes Yes Any-variate flexibility
TimesFM 200M 100B time points Yes Partial Decoder-only + 16K context
MOMENT-Large 125M Multi-task pile Yes Yes Multi-task capabilities

5.5.2 Parameter-Efficient Fine-Tuning

Foundation models contain hundreds of millions of parameters, making full fine-tuning computationally expensive and prone to overfitting on limited crypto data. Parameter-Efficient Fine-Tuning (PEFT) techniques adapt only a small subset of parameters while keeping the pre-trained backbone frozen.

LoRA (Low-Rank Adaptation)

LoRA injects trainable low-rank decomposition matrices into transformer layers while freezing pre-trained weights:

W=W0+ΔW=W0+BAW = W_0 + \Delta W = W_0 + BA

where W0∈Rd×kW_0 \in \mathbb{R}^{d \times k} is the frozen pre-trained weight, and B∈Rd×rB \in \mathbb{R}^{d \times r}, A∈Rr×kA \in \mathbb{R}^{r \times k} are trainable low-rank matrices with rank r≪min⁡(d,k)r \ll \min(d, k).

Advantages:

  • Train only 0.1-1% of total parameters (e.g., 710M → 7M trainable)
  • Modest training speedup: ~10-30% faster than full fine-tuning (primary benefit is memory, not speed)
  • Significant memory reduction: ~70% lower GPU requirements
  • Prevents catastrophic forgetting: Pre-trained knowledge preserved in frozen W0W_0

QLoRA (Quantized LoRA)

QLoRA extends LoRA with 4-bit quantization of the frozen pre-trained model, enabling fine-tuning of large models on consumer GPUs:

  • Quantizes W0W_0 to 4-bit (NF4 format)
  • Trains LoRA adapters in full precision
  • Achieves 16x memory reduction (710M model: 2.8GB vs 45GB)
  • Minimal accuracy degradation (<2% typical)

Fine-Tuning Strategy Comparison:

Method Trainable Params Training Speed GPU Memory Forgetting Risk
Full Fine-Tuning 100% 1x (baseline) 100% High
Frozen Backbone <1% (head only) 10x faster 20% None
LoRA 0.1-1% ~1.2x faster 30% Low
QLoRA 0.1-1% ~1.2x faster 15% Low

Note: LoRA's primary advantage is memory efficiency, not training speed. Research shows training speed improvements are modest (~10-30%) due to sequential adapter processing overhead. The dramatic benefit is enabling fine-tuning on smaller GPUs.

Implementation Example (LoRA for TimesFM):

from peft import LoraConfig, get_peft_model
import timesfm

# Load pre-trained TimesFM
base_model = timesfm.TimesFM(
    context_len=168,  # 7 days at 4H bars
    horizon_len=6,    # 6 steps ahead (24H)
)

# Configure LoRA
lora_config = LoraConfig(
    r=16,                # Rank of low-rank matrices
    lora_alpha=32,       # Scaling factor
    target_modules=["q_proj", "v_proj"],  # Apply to attention layers
    lora_dropout=0.1,
    bias="none",
)

# Wrap model with LoRA adapters
model = get_peft_model(base_model, lora_config)

# Only LoRA parameters trainable
trainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
total_params = sum(p.numel() for p in model.parameters())
print(f"Trainable: {trainable_params:,} / {total_params:,} ({100*trainable_params/total_params:.2f}%)")
# Output: Trainable: 2,097,152 / 200,000,000 (1.05%)

5.5.3 XGBoost TL vs Foundation Model TL

Trade-Matrix currently uses XGBoost with warm-start Transfer Learning. Foundation models offer a qualitatively different approach:

Aspect XGBoost TL Foundation Model TL
Pre-Training Data None (train from scratch) 100B+ time points from diverse domains
Crypto-Specific Training 3 years OHLCV (~6,500 samples) Fine-tune on same 3 years
Transfer Mechanism Warm-start: Add 50-250 new trees LoRA/QLoRA: Adapt 0.1-1% of parameters
Inference Latency 0.5-1.0ms (production requirement: <5ms) 10-50ms (requires quantization for <5ms)
Sample Efficiency 1,000+ samples required for stability 10-100 samples sufficient (leverages pre-training)
Multi-Horizon Single-step (forward returns) Native 1-96 step forecasting
Uncertainty Quantification External calibration needed Native quantile outputs (10th, 50th, 90th percentiles)
Feature Engineering Manual Boruta selection (9-11 features) Automatic learned representations
Multivariate Modeling Limited (independent instruments) Native (Moirai any-variate attention)
Interpretability Feature importance scores Attention weights, variable selection networks
Training Time (Weekly) ~20 min (Step 3: train-tl-only) ~45-90 min with LoRA fine-tuning
Computational Cost CPU-friendly (no GPU required) GPU-dependent (RTX 3090 / A100 recommended)
Overfitting Risk Moderate (6,500 samples, 10 features) Low (pre-trained on billions of samples)
Regime Adaptability Relies on sample weighting + ensemble Potentially better (learned diverse regime patterns)
Production Maturity Proven (Nov 2025 deployment, IC 0.05-0.10) Experimental (no published crypto trading results)

Key Trade-Offs:

  1. Latency: XGBoost meets <5ms requirement natively; foundation models require INT8 quantization (ONNX Runtime, TensorRT) to approach 10ms, potentially 5ms with aggressive optimization.

  2. Data Efficiency: Foundation models excel in low-data regimes (useful for new instruments with limited history), but Trade-Matrix has 3 years of data where XGBoost performs well.

  3. Multi-Horizon Capability: Foundation models natively forecast multiple horizons (1-6 steps), valuable for RL position sizing that considers multi-bar price trajectories. XGBoost requires separate models for each horizon.

  4. Infrastructure Complexity: XGBoost runs on CPU with minimal dependencies; foundation models require GPU infrastructure, CUDA, PyTorch, and specialized serving (TorchServe, Triton).

  5. Interpretability: XGBoost provides clear feature importance; foundation models offer attention weights (less intuitive but potentially revealing feature interactions).

5.5.4 Why Trade-Matrix Uses XGBoost TL (Not Foundation Models)

Despite the excitement around foundation models, Trade-Matrix's production deployment uses XGBoost Transfer Learning for pragmatic reasons:

1. Latency Constraint

Trade-Matrix has a hard <5ms inference latency requirement to maintain event-driven architecture responsiveness. Current benchmarks:

  • XGBoost (current): 0.5-1.0ms ✓
  • Chronos-Large: 50-100ms ✗
  • Moirai-Large: 30-80ms ✗
  • TimesFM v2.5: 20-60ms ✗
  • MOMENT-Large: 15-40ms ✗

Even the smallest foundation models (MOMENT: 125M parameters) exceed the latency budget by 3-8x. While INT8 quantization can achieve 2-4x speedup, reaching <5ms would require aggressive optimization (distillation, pruning, custom kernels) that hasn't been validated for financial forecasting accuracy.

2. Pre-Training Domain Mismatch

Foundation models are pre-trained primarily on physical-world time series (weather, electricity, traffic) with characteristics fundamentally different from cryptocurrency markets:

Characteristic Physical Time Series Crypto Markets
Stationarity Relatively stable dynamics Non-stationary, frequent regime breaks
Noise-to-Signal Ratio Low (physics-based processes) Very high (efficient market hypothesis)
Predictability High (seasonal patterns) Low (adversarial participants)
Outliers Rare Frequent (fat-tailed distributions)
Data Frequency Often regular intervals 24/7 continuous with irregular gaps
Seasonal Patterns Strong (weather, traffic) Weak (crypto trades around the clock)

The transfer learning hypothesis assumes source domain knowledge transfers to target domain. Weather forecasting patterns may not transfer meaningfully to predicting cryptocurrency price movements driven by market microstructure, sentiment, and macroeconomic shocks.

3. Production Validation Gap

As of January 2025, there are no published academic studies or production systems demonstrating foundation model effectiveness for cryptocurrency trading. Literature shows:

  • Weather forecasting: Excellent foundation model performance
  • Traffic prediction: Strong foundation model results
  • Stock prediction (daily): Promising GPT-4 results (Lopez-Lira & Tang)
  • Crypto (4H bars, IC-based signals): No documented evidence

Trade-Matrix would be an early adopter, accepting significant deployment risk without validation benchmarks.

4. Infrastructure Simplicity

XGBoost's CPU-based inference integrates seamlessly with Trade-Matrix's Docker/K3s architecture:

  • No GPU scheduling complexity
  • No CUDA version management
  • No PyTorch dependency conflicts
  • Lower memory footprint (10MB vs 500MB-2GB)
  • Simpler CI/CD pipelines (no model quantization steps)

Foundation models would require GPU infrastructure (RTX 3090 / A100), TensorRT optimization, and specialized serving frameworks (Triton Inference Server), increasing operational complexity.

5. Proven Performance

XGBoost Transfer Learning has demonstrated production-validated performance:

  • IC: 0.05-0.10 consistently across instruments
  • Weekly updates: ~60 min full pipeline (~20 min training)
  • October 2025 regime recovery: 1 week (vs 4-6 weeks full retrain)
  • Zero production incidents since November 2025 deployment

Replacing a proven system with an experimental approach requires compelling evidence of superiority—evidence that doesn't yet exist for foundation models in crypto trading.

Being Honest About Limitations

Foundation models are not ready for Trade-Matrix production as of January 2025, but they represent a promising research direction:

  • Latency gap: 3-8x too slow for current architecture
  • Domain transfer uncertainty: Pre-training on weather/traffic may not help crypto forecasting
  • Validation gap: No academic or industry evidence for crypto trading
  • Complexity cost: GPU infrastructure, quantization pipelines, specialized serving

However, foundation models offer compelling long-term advantages:

  • Sample efficiency: Could enable faster instrument onboarding (new altcoins)
  • Multi-horizon: Native 1-96 step forecasting valuable for RL agents
  • Uncertainty quantification: Built-in probabilistic predictions
  • Feature learning: Automatic representation learning vs manual Boruta selection

Future Roadmap

Trade-Matrix should pursue foundation models as a parallel research track, not a production replacement:

  1. Phase 1 (Research, 8 weeks): Benchmark MOMENT-Large on 3-year backtest

    • Evaluate IC, Sharpe, maximum drawdown
    • Measure fine-tuning sample efficiency (100 samples vs 1,000 vs full dataset)
    • Quantify latency with INT8 quantization

  2. Phase 2 (Optimization, 6 weeks): If Phase 1 shows IC >= 0.10 (vs current 0.05-0.10)

    • Distillation: Compress 125M MOMENT to 20M student model
    • Pruning: Remove low-importance attention heads
    • Custom kernels: Optimize attention computation for financial data

  3. Phase 3 (Deployment, 4 weeks): If Phase 2 achieves <5ms latency

    • A/B test: 10% traffic to foundation model, 90% to XGBoost
    • Monitor for degradation: Weekly IC validation gates
    • Gradual rollout: Increase to 50% if superior over 4 weeks

Conservative Estimate: Foundation models won't be production-ready for Trade-Matrix until Q3-Q4 2025 at earliest, pending:

  • Published validation on crypto trading benchmarks
  • Proven quantization/distillation techniques maintaining accuracy
  • Open-source implementations with financial-specific pre-training

Until then, XGBoost Transfer Learning remains the pragmatic choice: proven, fast, and effective.

6. Preventing Catastrophic Forgetting

6.1 Feature Order Validation

CRITICAL: Sklearn models validate feature names AND ORDER at prediction time. If features are provided in different order than training, predictions are silently incorrect.

# Training time: Features in Boruta-selected order
X_train = df[['rsi_14', 'macd', 'bb_width', 'volume_ma_ratio']]
model.fit(X_train, y_train)

# Inference time: Features in DIFFERENT order (SILENT FAILURE!)
X_inference = df[['bb_width', 'macd', 'rsi_14', 'volume_ma_ratio']]
prediction = model.predict(X_inference)
# WRONG PREDICTIONS! No error raised - silently incorrect results

The model internally maps features by position, not name. If bb_width (actually position 2 during training) is provided at position 0 during inference, the model uses bb_width values where it expects rsi_14 values.

The Solution: Every model artifact includes locked_features.json preserving the exact training order. Feature extraction during inference must use this exact order.

def validate_feature_compatibility(old_run_id, new_run_id):
    """Validate OLD and NEW models use same features in same order."""

    old_features = load_locked_features(old_run_id)
    new_features = load_locked_features(new_run_id)

    # Check exact match (names AND order)
    if old_features != new_features:
        raise ValueError(
            f"Feature mismatch!\n"
            f"OLD: {old_features}\n"
            f"NEW: {new_features}\n"
        )

    return True

6.2 Ensemble Design

The ensemble architecture combines OLD and NEW model predictions with fixed weights:

signal=0.4×OLD+0.6×NEW\text{signal} = 0.4 \times \text{OLD} + 0.6 \times \text{NEW}

Weight Rationale:

  • 40% OLD: Preserves historical knowledge, provides stability
  • 60% NEW: Emphasizes recent adaptation, captures current regime

This weighting was determined empirically through backtesting across multiple regime changes. Higher NEW weights improve short-term responsiveness but increase volatility; higher OLD weights improve stability but reduce adaptation speed.

6.3 Sample Weighting for Recency

Exponential sample weighting provides continuous emphasis on recent data:

w(t)=exp⁡(−λ⋅agedays)w(t) = \exp(-\lambda \cdot \text{age}_\text{days})
λ\lambda Half-life Interpretation
0.001 693 days Very slow decay
0.005 139 days Moderate decay (recommended)
0.010 69 days Fast decay
0.020 35 days Very fast decay

The recommended λ=0.005\lambda = 0.005 provides:

  • Recent data (last month): ~85% weight
  • 3-month-old data: ~64% weight
  • 6-month-old data: ~41% weight
  • 1-year-old data: ~16% weight

This decay profile ensures historical data contributes meaningfully while recent data dominates.

7. Walk-Forward Validation

7.1 Why Simple Holdout Fails

Simple train/test splits suffer from high variance in financial applications:

Dataset: 6,500 bars
Train: 4,875 bars (75%)
Test: 1,625 bars (25%)

Single IC estimate: 0.055

But: Small change in split point could yield IC = 0.03 or IC = 0.08
Variance is too high for reliable deployment decisions!

7.2 Stationary Bootstrap

Walk-Forward Validation with stationary bootstrap provides robust IC estimates with lower variance:

Algorithm (Politis & Romano, 1994):

  1. Generate bootstrap samples by randomly selecting starting points
  2. Extend from each starting point with blocks of random length
  3. Block length determines autocorrelation preservation
  4. Calculate IC for each bootstrap sample
  5. Report mean, standard deviation, and confidence intervals
def stationary_bootstrap(predictions, actuals, n_bootstrap=100):
    """
    Stationary bootstrap for robust IC estimation.
    """
    n = len(predictions)
    block_length = max(5, int(n ** (1/3)))  # Optimal for time series
    p = 1.0 / block_length  # Probability of starting new block

    bootstrap_ics = []

    for _ in range(n_bootstrap):
        # Generate bootstrap indices
        indices = []
        current_idx = np.random.randint(0, n)

        while len(indices) &#x3C; n:
            indices.append(current_idx)
            if np.random.random() &#x3C; p:
                current_idx = np.random.randint(0, n)
            else:
                current_idx = (current_idx + 1) % n

        # Calculate IC for bootstrap sample
        boot_pred = predictions[indices[:n]]
        boot_actual = actuals[indices[:n]]
        ic, _ = spearmanr(boot_pred, boot_actual)
        bootstrap_ics.append(ic)

    return {
        'mean': np.mean(bootstrap_ics),
        'std': np.std(bootstrap_ics),
        'ci_lower': np.percentile(bootstrap_ics, 2.5),
        'ci_upper': np.percentile(bootstrap_ics, 97.5)
    }

7.3 Validation Results

Typical Phase 1 validation results:

Instrument Bootstrap IC Std 95% CI Features Status
BTCUSDT 0.065-0.070 0.015 [0.04, 0.10] 9 APPROVED
ETHUSDT 0.055-0.060 0.012 [0.03, 0.08] 13 APPROVED
SOLUSDT 0.050-0.055 0.014 [0.02, 0.08] 11 APPROVED

The confidence intervals provide deployment decision boundaries:

  • If CI_lower >= 0.03: Strong confidence in positive IC
  • If CI_lower >= 0.00 but CI_upper >= 0.05: Marginal, deploy with monitoring
  • If CI_upper < 0.05: Consider skip or fallback model

8. Implementation Details

8.1 MLflow Integration

All models are tracked in MLflow with comprehensive metadata:

def register_to_mlflow(model, metrics, locked_features, instrument, phase):
    """Register model to MLflow with comprehensive artifacts."""

    mlflow.set_experiment(f"TL_Training_{instrument}")

    with mlflow.start_run(run_name=f"{instrument}_{phase}"):
        # Parameters
        mlflow.log_param("instrument", instrument)
        mlflow.log_param("phase", phase)
        mlflow.log_param("n_features", len(locked_features))

        # Metrics
        mlflow.log_metric("ic", metrics['ic'])
        mlflow.log_metric("p_value", metrics['p_value'])
        mlflow.log_metric("sharpe", metrics['sharpe'])

        # CRITICAL: Log locked features
        with open('locked_features.json', 'w') as f:
            json.dump({'feature_columns': locked_features}, f)
        mlflow.log_artifact('locked_features.json')

        # Log model
        mlflow.sklearn.log_model(
            model,
            artifact_path="model",
            registered_model_name=f"{instrument}_TL_production"
        )

        return mlflow.active_run().info.run_id

8.2 Model Loading Architecture

A 4-tier resilient loading system ensures production reliability:

Tier 0: MLflow Registry (registered model)
    |
    v (if fails)
Tier 1: Run ID + Artifact Path
    |
    v (if fails)
Tier 2: Direct S3/MinIO access
    |
    v (if fails)
Tier 3: Local filesystem fallback

class ResilientModelLoader:
    """4-Tier Resilient Model Loading System."""

    def load_model(self):
        """Attempt loading with automatic fallback."""

        # Tier 0: MLflow Registry
        try:
            return mlflow.sklearn.load_model(
                f"models:/{self.model_name}/Production"
            )
        except Exception:
            pass

        # Tier 1: Run ID + Artifact
        try:
            return mlflow.sklearn.load_model(
                f"runs:/{self.run_id}/model"
            )
        except Exception:
            pass

        # Tier 2: Direct S3
        try:
            return self._load_from_s3()
        except Exception:
            pass

        # Tier 3: Local fallback
        return self._load_from_local()

8.3 Weekly Automation Pipeline

The complete weekly pipeline executes automatically:

# .github/workflows/weekly-update.yml
name: Weekly TL Update

on:
  schedule:
    - cron: "0 6 * * 0" # Every Sunday at 6 AM UTC
  workflow_dispatch: # Manual trigger

jobs:
  weekly-update:
    runs-on: ubuntu-latest
    steps:
      - name: Fetch OHLCV Data
        run: make fetch-ohlcv-data

      - name: Build Features
        run: make build-features-only

      - name: Train TL Models
        run: make train-tl-only

      - name: Train RL Agents
        run: make train-rl-only

      - name: Generate Signals
        run: make generate-signals-only

      - name: Run Backtest
        run: make run-backtest-only

      - name: Export to MLflow
        run: make export-mlflow-only

      - name: Deploy to Production
        if: success()
        run: make deploy-models

9. Results and Validation

9.1 Training Time Comparison

Approach Weekly Duration Computational Cost
Full Retrain 2-4 hours High (full optimization)
TL Protocol ~60 minutes Low (incremental only)
Speedup 2-4x Significant savings

The dramatic speedup comes from:

  • Skipping Boruta selection (uses locked features)
  • Warm-start training (only fit new trees)
  • No hyperparameter optimization (uses validated settings)

9.2 Performance Across Regimes

The October 2025 regime break provided a natural experiment:

Model Type Pre-Break IC Post-Break IC Recovery Time
OLD Baseline 0.02-0.09 <0.02 N/A (never recovers)
Full Retrain 0.02-0.05 0.01-0.03 4-6 weeks
TL Ensemble 0.02-0.08 0.05-0.10 1 week

The TL approach recovered within a single weekly update because:

  • OLD component preserved non-regime-specific patterns
  • NEW component rapidly adapted to post-break dynamics
  • Sample weighting emphasized post-break data

9.3 Comparative Validation Results

The adaptive deployment decision framework compares NEW vs OLD models on holdout data:

Instrument CV IC NEW Holdout IC OLD Holdout IC Decision
BTCUSDT 0.027 0.32 0.34 (+6.8%) DEPLOY_NEW
ETHUSDT 0.055 0.05 -0.01 DEPLOY_NEW
SOLUSDT 0.068 0.06 0.16 (+147%) DEPLOY_OLD

The decision framework prevents deploying inferior models:

  • BTCUSDT: NEW had strong holdout despite weak CV (exception case)
  • ETHUSDT: NEW clearly superior to OLD
  • SOLUSDT: OLD significantly outperformed NEW on recent data

10. Trade-Matrix Integration

10.1 Production Architecture

The TL models integrate into Trade-Matrix's production pipeline:

                     +-----------------+
                     |  Market Data    |
                     |   (Bybit API)   |
                     +--------+--------+
                              |
                              v
                     +--------+--------+
                     | Feature Engine  |
                     | (Locked Boruta) |
                     +--------+--------+
                              |
                              v
              +--------------+--------------+
              |                             |
              v                             v
    +---------+--------+         +---------+--------+
    |   OLD Model      |         |   NEW Model      |
    | (Frozen Baseline)|         | (Weekly Updated) |
    +--------+---------+         +--------+---------+
              |                             |
              +-------------+---------------+
                            |
                            v
                   +--------+--------+
                   |    Ensemble     |
                   | 0.4*OLD+0.6*NEW |
                   +--------+--------+
                            |
                            v
                   +--------+--------+
                   | RL Position     |
                   | Sizing Agent    |
                   +--------+--------+
                            |
                            v
                   +--------+--------+
                   |  Risk Manager   |
                   | (Circuit Breaker)|
                   +--------+--------+
                            |
                            v
                   +--------+--------+
                   | Order Execution |
                   +------------------+

10.2 RL Position Sizing Integration

The TL signal feeds into an RL position sizing agent:

4-Tier Fallback Cascade:

Tier Condition Position Size
1: FULL_RL High IC (>=0.05) + High Confidence 100% RL
2: BLENDED Medium IC or Medium Confidence 50% RL + 50% Kelly
3: PURE_KELLY Low IC or IC failure 100% Kelly
4: EMERGENCY_FLAT Circuit Breaker OPEN 0% position

10.3 Regime Detection Integration

A 2-state MS-GARCH (Markov-Switching GARCH) model with 4-state Kelly mapping provides regime context:

Regime Characteristics Kelly Fraction
Bull Positive trend, low volatility 67%
Bear Negative trend, rising volatility 25%
Neutral No clear trend, stable volatility 50%
Crisis High volatility, correlation breakdown 17%

The regime state modifies position sizing, reducing exposure during adverse conditions regardless of TL signal strength.

Weekly Regime Retraining (v1.9.5):

As of January 2026, regime models are retrained weekly as part of the automated pipeline:

  • Step 3.7: Retrain MS-GARCH parameters on rolling 1200-bar window (~2 min)
  • Training window: Latest 1200 bars of 4H OHLCV per instrument
  • Output: 3 model files saved to models/regime/
  • MLflow: Logged to regime_detection experiment

This ensures regime parameters adapt to recent market behavior evolution, particularly important after volatility regime changes.

Code References:

Real-time regime detection:
  services/regime_detection/realtime_regime_detector.py

Weekly retraining:
  scripts/ml/retrain_regime_models.py

Workflow definition:
  .claude/workflows/weekly-tl-update.yaml (Step 3.7)

11. Conclusion

The 3-Phase Transfer Learning Protocol provides a production-validated methodology for maintaining ML model performance in non-stationary cryptocurrency markets. Key achievements include:

  1. Computational Efficiency: Weekly updates complete in approximately 60 minutes (full 8-step pipeline), a 2-4x improvement over full retraining requiring 2-4 hours.

  2. Preservation of Knowledge: The frozen OLD model baseline prevents catastrophic forgetting of proven alpha patterns across market cycles.

  3. Rapid Adaptation: Sample weighting and warm-start training enable quick recovery from regime breaks, typically within one weekly update cycle.

  4. Robust Validation: Walk-Forward Validation with stationary bootstrap provides lower-variance IC estimates compared to simple holdout approaches.

  5. Production Reliability: 4-tier model loading, locked feature sets, and automated pipelines ensure stable production operation.

The methodology generalizes beyond cryptocurrency to other domains exhibiting non-stationarity: FX markets, commodities, and equity factors. The core principles of preserving baseline knowledge while enabling incremental adaptation apply wherever learned patterns exhibit varying persistence timescales.

Future work includes extending the adapter fine-tuning strategy to neural architectures (LSTM, Transformer) and investigating online learning approaches for continuous adaptation between weekly updates.

References

  1. Pan, S. J., & Yang, Q. (2010). A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345-1359.

  2. Lopez de Prado, M. (2018). Advances in Financial Machine Learning. Wiley.

  3. Kirkpatrick, J., et al. (2017). Overcoming Catastrophic Forgetting in Neural Networks. Proceedings of the National Academy of Sciences, 114(13), 3521-3526.

  4. Politis, D. N., & Romano, J. P. (1994). The Stationary Bootstrap. Journal of the American Statistical Association, 89(428), 1303-1313.

  5. Kursa, M. B., & Rudnicki, W. R. (2010). Feature Selection with the Boruta Package. Journal of Statistical Software, 36(11), 1-13.

  6. Guida, T., & Coqueret, G. (2020). Machine Learning for Factor Investing. Chapman and Hall/CRC.

  7. Gu, S., Kelly, B., & Xiu, D. (2020). Empirical Asset Pricing via Machine Learning. The Review of Financial Studies, 33(5), 2223-2273.

  8. Feng, G., Giglio, S., & Xiu, D. (2020). Taming the Factor Zoo: A Test of New Factors. The Journal of Finance, 75(3), 1327-1370.

  9. Harvey, C. R., Liu, Y., & Zhu, H. (2016). ... and the Cross-Section of Expected Returns. The Review of Financial Studies, 29(1), 5-68.

  10. Bailey, D. H., Borwein, J. M., Lopez de Prado, M., & Zhu, Q. J. (2014). Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance. Notices of the AMS, 61(5), 458-471.