Alternative EOI G - Foundation-Model-Free Forecast Calibration
Title (paste-ready): Calibrated Confidence Intervals for AI-Driven Economic Forecasts: Open Tooling Without Foundation-Model Training
EOI BODY (paste-ready)
Foundation-model forecasts now reach every economic prediction surface from inflation to commodity prices. Reviewers from the Bank for International Settlements and the European Central Bank report that the calibration of these forecasts is uneven. Confidence intervals tend to under-cover during stress regimes. Pre-deployment validation rarely tests tail behaviour. Supervisors care about coverage and tail risk far more than headline accuracy. The gap is reachable without training a new foundation model.
We propose calibrate-now, an open MIT-licensed library for post-hoc calibration of AI-driven economic forecasts. The library wraps any forecast source and returns calibrated quantile intervals plus tail-risk diagnostics. The methodology layers conformal prediction, extreme-value-theory tail diagnostics, and Bayesian regime-aware re-weighting. The methodological precedent draws on Stephen’s distribution-theory work at AUS, FRG24 price-impact analysis, and Joerg’s 2025 retail-customer distress early-warning paper. Foundation-model time-series tools (TimesFM, Lag-Llama) serve as input forecasters via off-the-shelf inference. No foundation-model training is in scope.
Specific aims: (1) Develop conformal-EVT-Bayesian calibration methods. (2) Release the calibrate-now library under MIT by year 2. (3) Build an open coverage benchmark across crypto, commodity, and macro forecasts. (4) Validate against two published forecast series with known tail under-coverage. (5) Train one UAE-based PhD plus host two workshops.
Application-readiness arc. Months 0-6 cover the Applied baseline with conformal-prediction primitives and the EVT tail layer. Months 6-12 produce a working prototype on crypto and commodity forecasts. Months 12-18 reach Tech Development with the Bayesian regime-aware re-weighting added. Months 18-24 demonstrate the first Validation step. A named user from Joerg’s ING advisory or a COST 19130 European supervisor runs calibrate-now on a live forecast stream by month 24.
PI Stephen Chan is at AUS Mathematics & Statistics in Sharjah, with a distribution-theory backbone and the FRG23 plus FRG24 substrate in place by May 2025. Co-PI Joerg Osterrieder chairs COST Action 19130 across 51 countries and coordinates the MSCA-DN Industrial Doctoral Network. Co-PI Youcheng Sun at MBZUAI Computer Science leads verification of the calibration guarantees through neural-network testing methods (ISSTA 2024) and the Verifi framework (TDSC 2024). UAE delivery anchors at AUS plus MBZUAI through Co-PI presence. Two UAE-based PhDs are recruited, one at AUS under Stephen and one at MBZUAI under Youcheng. Capacity building includes a graduate course on Calibration for Economic AI plus workshops at AUS in year 1 and MBZUAI in year 2. ADGM RegLab and CBUAE Fintech Office are the regulator-side target outreach during the May full-proposal phase. All code MIT, methods/datasets CC-BY where licence allows.