AI-Based Forecast Bust Detection for Medium-Range Weather Forecasts
Ministry of Earth Sciences (MoES) · Disaster Management · Software
A genuinely clean supervised problem with free real data, an objectively computable target and a fair baseline — rare on this portal — so define your bust threshold honestly and make beating ensemble spread the whole claim.
What it actually is
Weather forecasts occasionally go badly wrong, and they tend to do it during exactly the situations that matter — monsoon depressions, heavy rain events, cyclones, heatwaves. Forecasters cannot tell in advance which of today's forecasts is one of those. The ask is a system that looks at a forecast and says how much to trust it, region by region and day by day.
What to build
A forecast confidence system that learns from history what a forecast about to fail looks like: an archive-building stage pairing past forecasts with the verifying analysis to compute the error field for each region and lead time, a model predicting the probability of a large error from the forecast's own state and the atmospheric situation it describes rather than from the outcome, outputs covering the four the statement names — a region-wise confidence map for days one to ten, a bust probability, identification of error-prone areas, and an explanation naming the meteorological reason confidence is low — benchmarked against ensemble spread, which is the operational proxy for uncertainty and the baseline you must beat to have contributed anything.
Smallest thing that wins the room
Pick a documented forecast bust the model never trained on, show it flagging low confidence over the right region days ahead, and open the explanation naming the situation that drove the flag.
How crowded this one gets
A guess, projected from the 2025 statements — the last year where both the submission counts and the winners were published.
Quieter than 42% of the 226 · #132 of 226 by expected field
A normal-sized field. Your idea has to be good, not miraculous.
Why: central ministry statements sat below the average.
This is a guess, not a fact
Nobody has published 2026’s numbers yet. This is an analysed estimate from last year’s pattern, so please do not take it as the truth — check the live counter on the SIH portal before you decide anything. The range covers the middle half of likely outcomes, so one statement in two lands outside it. Entry closes at 500 ideas per statement, so no range goes past that — a statement that reaches the cap fills and shuts rather than drawing an unlimited crowd. The model reads only three things a team can see before choosing — software or hardware, the theme, and what kind of body posted it — and those explain about a quarter of the variation in last year’s field sizes (R² 0.25 on held-out statements). Trust the band more than the number, and the ordering more than either. It cannot see how good your idea is, which is the part that actually decides it.
The scores
The number is the shorthand. The line under it is the reason.
Acceptance potential
4/5This is a clean, well-posed supervised problem with genuinely free data, an honest hindcast validation against documented busts, and a fair baseline in ensemble spread — and forecast verification is an unglamorous corner that will attract very few teams despite being operationally valuable.
Feasibility
4/5The data position is genuinely good — global forecast archives are openly distributed, research ensemble archives are freely available, and reanalysis provides the verifying truth, so you can construct a real history of forecast errors and learn from it rather than simulating anything.
Innovation scope
4/5The statement names the outputs and nothing about method, so how you represent a forecast's state, what features indicate impending failure and how you distinguish genuine predictability limits from model deficiencies are all yours to work out.
Clarity
4/5Short but well posed — the four expected outputs are enumerated precisely and the framing that you compare current forecast patterns against historical error behaviour is exactly the right formulation, though no accuracy target or definition of how large an error counts as a bust is given.
Effort
HeavyBuilding an aligned forecast-and-verification archive across ten lead times is substantial data engineering, and on top of it sit the error model, the explainability layer, the ensemble spread baseline and a dashboard.
Demo-ability
MediumFlagging a real historical bust before it happened is a genuinely satisfying moment and it is verifiable against the record, but the product is confidence maps and probabilities, which need framing before a general judge appreciates them.
In its favour
- Green flag: Historical forecast archives and reanalysis are both freely available, so unlike almost everything else in this block you train and validate on real data rather than on a simulator
- Green flag: The target variable is objectively computable — you know exactly how wrong every past forecast was — which makes this a genuinely clean supervised problem with no label ambiguity
- Green flag: Ensemble spread gives you an operational baseline that is honest and non-trivial, so demonstrating improvement over it is a real result rather than a claim
- Green flag: Forecast verification is unglamorous and will attract very few teams, yet it is directly useful to the sponsor's day-to-day operations
Against it
- Red flag: Ensemble spread already encodes much of the predictable uncertainty and is a stronger baseline than it looks, so a model that merely correlates with spread has added nothing and you must show the increment
- Red flag: Busts are by definition rare, so the class imbalance is severe and a model optimised on accuracy will confidently declare every forecast reliable
- Red flag: Bust is never defined in the statement, so you set the error threshold that determines your own positive class — pick it from operational impact rather than from what makes your numbers look good, and say which you did
- Red flag: Some forecast failures are genuine limits of atmospheric predictability rather than detectable model weaknesses, so there is a ceiling on this task and claiming to have beaten it should make you suspicious of your own evaluation
What you will be writing
- forecast-analysis paired error archive construction
- gradient boosting or CNN error probability model
- ensemble spread as uncertainty baseline
- SHAP attribution for low-confidence reasons
- ERA5 verifying analysis
- lead-time-stratified reliability diagrams
- Forecast verification
- Uncertainty quantification
- Operational meteorology
Prior art to read before you start
predicting forecast error before verification · forecast confidence and reliability estimation · explainable uncertainty attribution
Analysed by Claude Opus. Every score above is a judgment call with its reasoning attached — kindly cross-check this against the official statement on the SIH portal before your team commits to it.