Issues & Limitations

Data constraints, methodology caveats, and research gaps

A. Data Limitations

Crisis Database Coverage
  • Laeven-Valencia database covers systemic banking crises from 1970-2017
  • Manual extension to 2023 adds US (SVB, Signature, First Republic) and Switzerland (Credit Suisse)
  • No standardized crisis database exists for 2018-2022 period
  • Crisis severity is not captured (binary classification only)
Geographic & Sectoral Coverage
  • 19 countries analyzed - primarily developed economies + major emerging markets
  • Many developing countries excluded due to data gaps
  • Sector limited to 'P' (Private non-financial) - households and non-financial corps not analyzed separately
  • Some countries have incomplete time series (gaps in early years)
Data Timeliness
  • BIS data lag: ~6 month publication delay for credit-to-GDP data
  • Real-time implementation would use provisional/estimated data
  • Quarterly frequency limits early detection (monthly data not available)

B. Methodology Caveats

HP Filter Parameters
  • Lambda = 400,000 is BIS convention, not econometrically derived
  • Originally designed for business cycles (lambda=1,600 for quarterly), adapted for credit cycles
  • One-sided filter avoids future information but has endpoint bias
  • Alternative trend extraction methods (e.g., Hamilton filter) not explored
Signal Extraction Assumptions
  • Pre-crisis window = 12 quarters (3 years) is researcher choice
  • Optimal threshold (4 pp) derived in-sample, may not generalize
  • Binary signal (gap >= threshold) ignores gap magnitude
  • Country-specific thresholds not explored
Train/Test Split
  • 70/30 chronological split - no temporal cross-validation
  • Test set concentrated in recent period (includes GFC)
  • No expanding window or walk-forward validation

C. Model Limitations

Class Imbalance
  • Only 4.4% pre-crisis observations (164 out of 3,719)
  • Standard accuracy metrics are misleading
  • No resampling techniques (SMOTE, undersampling) applied
  • Precision is low by design (many false positives)
Feature Engineering
  • 12 features are researcher choices - not exhaustive
  • Features: credit_gap, 4 lags, 2 momentum, 2 volatility, mean, std, z-score
  • No external variables (interest rates, property prices, debt service ratio)
  • Feature selection based on domain knowledge, not automated
Logistic Regression Dominance
  • Logistic (AUC=0.694) >> Tree models (AUC~0.5)
  • Suggests near-linear decision boundary in feature space
  • Complex models may overfit sparse signal
  • Hyperparameter tuning was limited for tree models
Validation Constraints
  • No external validation - all results are in-sample or single test set
  • 2023 crisis not included in training/testing
  • Country-specific performance not reported
  • Confidence intervals not computed for all metrics

D. Future Research Directions

Recommended Extensions
  1. Real-time evaluation: Test using only data available at each point in time
  2. Country-specific models: Compare pooled vs individual country thresholds
  3. Alternative crisis definitions: Severity-weighted, multi-class (mild/severe)
  4. Additional EWS indicators: Property prices, debt service ratio, bank leverage
  5. Out-of-sample testing: Evaluate on 2023 banking crisis events
  6. Ensemble approaches: Combine signal extraction with ML predictions
  7. Interpretability: SHAP values for tree models, calibration curves
  8. Temporal dynamics: Regime-switching models, time-varying thresholds
Known Technical Issues
  • Chart PDFs generated at fixed resolution (may pixelate when zoomed)
  • Panel data pagination is client-side (full data loaded in browser)
  • No automated data refresh (manual pipeline execution required)
  • Literature search limited to OpenAlex (no Scopus/WoS integration)