disease¶
Climate suitability for disease outbreaks (e.g. malaria). Entry point:
DiseaseRiskUseCase.
Use case¶
use_case ¶
DiseaseRiskUseCase ¶
Bases: BaseUseCase
Entry point for the climate-driven disease surveillance domain.
Minimal config (all optional — defaults match Kisumu County, Kenya 2021–2023):
{ "aoi_geojson": {"type": "Polygon", "coordinates": [...]}, "gee_project": "your-gee-project-id", "start_date": "2021-01-01", "end_date": "2023-12-31", "model_type": "gbm", # "gbm" | "xgboost" | "ensemble" "n_pixels": 3000, "scale": 1000, # metres — MODIS / CHIRPS native "output_dir": "outputs", "prefix": "disease", }
Source code in disease/use_case.py
fetch_data ¶
Authenticate GEE and parse the AOI geometry (see _fetch_from_dict).
Source code in disease/use_case.py
preprocess ¶
Build the feature stack and sample training pixels (see _preprocess_raw).
run_model ¶
Train models, export the risk COG, and package an AnalysisOutput.
Source code in disease/use_case.py
run ¶
Single AOI analysis — full pipeline in one call.
Source code in disease/use_case.py
run_date_ranges ¶
Run the same AOI over multiple date-range configurations in parallel.
run_multi_regions ¶
Features¶
features ¶
fetch_rainfall_4w ¶
CHIRPS 28-day cumulative rainfall ending on end_date. Returns Dataset with variable 'rainfall_4w'.
Source code in disease/features.py
fetch_lst_mean ¶
MODIS Terra MOD11A2 daytime LST mean over [start, end] in °C. Returns Dataset with variable 'temp_mean'.
Source code in disease/features.py
fetch_ndwi ¶
Sentinel-2 MNDWI = (Green − SWIR1) / (Green + SWIR1) median composite. Returns Dataset with variable 'ndwi'.
Source code in disease/features.py
fetch_elevation ¶
USGS SRTM 30 m elevation. Returns Dataset with variable 'elevation'.
Source code in disease/features.py
fetch_pop_density ¶
fetch_pop_density(aoi: Geometry, year: int = 2020, scale: int = 1000, country: str | None = None) -> xr.Dataset
Population density, log-transformed: log(1 + pop). Returns Dataset with variable 'pop_density'.
Delegates to core.population.fetch_population_count for the underlying count raster (Earth Engine's WorldPop mirror primary, worldpop.org's own catalog as fallback — see that function's docstring) rather than querying GEE directly, so this feature benefits from the same fallback and doesn't duplicate the WorldPop query logic. Deliberately not wrapped in fetch_population_count_safe: pop_density is a required FEATURE_COLS entry, so a total fetch failure (both sources unavailable) should propagate rather than silently produce an incomplete training set.
Source code in disease/features.py
fetch_ndvi_mean ¶
MODIS MOD13A3 monthly NDVI mean over [start, end] (scale factor 0.0001). Returns Dataset with variable 'ndvi'.
Source code in disease/features.py
fetch_landcover ¶
ESA WorldCover v200 normalised to [0, 1]. Returns Dataset with variable 'land_cover'.
Source code in disease/features.py
build_feature_datasets ¶
Download all seven disease feature bands from GEE in parallel. The seven fetch calls are independent HTTP requests; they run concurrently via DaskEngine.run_io_parallel (ThreadPoolExecutor) sharing the GEE session. Returns a dict keyed by band group name, consumed by cog_export.
Source code in disease/features.py
build_gee_feature_stack ¶
Assemble the 6-band GEE image used for stratified pixel sampling (FEATURE_COLS minus pop_density). geometries=True preserved in sampling call so centroids are available for DBSCAN hotspot detection.
pop_density is deliberately not part of this GEE stack: it's joined into the sampled points afterwards, by nearest-neighbour lookup against the Earth-Engine-primary/worldpop.org-fallback raster already fetched by build_feature_datasets — see sample_training_data. It used to be merged in here as a seventh GEE band (WorldPop/GP/100m/pop), but that GEE mirror has coverage gaps for some countries/AOIs (verified live for parts of Uganda), which surfaced as a "No data was available for: population density" error from sample_training_data even though the other six bands sampled fine.
Source code in disease/features.py
295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 | |
fetch_monthly_timeseries ¶
Fetch monthly area-mean NDVI, rainfall, and LST time series in parallel. The three GEE queries are independent and run concurrently via DaskEngine.run_io_parallel (ThreadPoolExecutor) sharing the GEE session. Returns dict keyed by variable name with a DatetimeIndex DataFrame.
Source code in disease/features.py
sample_training_data ¶
sample_training_data(feature_stack: Image, aoi: Geometry, pop_density_da: DataArray, n_pixels: int = 3000, scale: int = 1000, seed: int = 42) -> pd.DataFrame
Sample n_pixels from the 6-band GEE feature stack, join in pop_density by nearest-neighbour lookup against pop_density_da, and assign 3-class disease risk labels. Labels: fixed absolute thresholds on the composite risk score (0-100 scale). 0 = Low Risk (score < 33) 1 = Medium Risk (33 <= score < 66) 2 = High Risk (score >= 66) Returns DataFrame with FEATURE_COLS + ['lon', 'lat', 'risk_score', 'label'].
pop_density_da (from build_feature_datasets's "population" fetch — see fetch_pop_density) is sampled at each point separately from the other six features rather than being merged into feature_stack itself as a GEE band — see build_gee_feature_stack's docstring for why.
Source code in disease/features.py
519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 | |
align_datasets ¶
align_datasets(datasets: dict[str, Dataset], ref_key: str = 'elevation', method_continuous: InterpOptions = 'linear', method_categorical: InterpOptions = 'nearest') -> dict[str, xr.Dataset]
Interpolate all datasets onto the reference grid. Land cover is treated as categorical (nearest-neighbour). population_count is passed through entirely unchanged, at its own native ~100 m resolution — it is never interpolated onto this (coarser) reference grid. Summing population within a risk zone requires the opposite: upsampling the (categorical) classification onto population's native grid — see core.population.population_exposure, which every cog_export.py uses for this instead of relying on align_datasets. Interpolating population here would silently corrupt totals (verified against live GEE data — see core.population.fetch_population_count's docstring).
Each dataset is chunked before interpolation so that xarray produces Dask-backed lazy arrays; dask.compute() then materialises them all in a single parallel scheduler pass.
Source code in disease/features.py
Model¶
model ¶
DiseaseModel ¶
Orchestrates the full ML pipeline for a single disease surveillance analysis. config['model_type'] selects which model(s) are trained and which predictions drive the primary risk distribution: "gbm" — Gradient Boosting only (default, highest accuracy per lab) "xgboost" — XGBoost only "ensemble" — both, combined as mean softmax probabilities of GBM + XGBoost Only the selected model type is trained. Trained models and scaler are stored on self for use by cog_export (whichever model wasn't needed stays None).
Source code in disease/model.py
predict ¶
predict(df: DataFrame, timeseries: dict[str, DataFrame] | None = None, config: dict | None = None) -> dict
Parameters¶
df : DataFrame with FEATURE_COLS + ['lon', 'lat', 'risk_score', 'label'] timeseries : dict of monthly DataFrames (ndvi, rain, lst) from fetch_monthly_timeseries config : optional dict; reads 'model_type' (default 'gbm')
Returns¶
dict with keys: stats, charts
Source code in disease/model.py
335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 | |
train_gbm ¶
train_gbm(X_train: ndarray, y_train: ndarray, cv_folds: int = 5) -> tuple[GradientBoostingClassifier, dict]
Fit Gradient Boosting Classifier with sample weights. Returns (model, metadata).
Source code in disease/model.py
train_xgb ¶
Fit XGBoost multi-class classifier via the low-level Booster API with sample weights.
Uses xgboost.train() rather than the XGBClassifier sklearn wrapper because the wrapper requires every class in DISEASE_CLASSES to appear in y_train (it infers num_class from np.unique(y_train) and rejects gaps). Some AOIs/time windows genuinely have zero samples of a given risk class, which is a legitimate state, not invalid input, so class count is fixed via XGB_PARAMS instead.
Returns (booster, metadata).
Source code in disease/model.py
pad_gbm_proba ¶
Expand GBM's predict_proba output to the full DISEASE_CLASSES width.
GradientBoostingClassifier only emits a column per class it actually saw during fit (via gbm.classes_), whereas train_xgb's Booster always outputs len(DISEASE_CLASSES) columns regardless of what the AOI's data contained. Without padding, an AOI missing one risk class produces mismatched shapes (e.g. (n, 2) vs (n, 3)) the moment GBM and XGBoost probabilities are combined for the ensemble.
Source code in disease/model.py
evaluate_models ¶
evaluate_models(gbm: GradientBoostingClassifier | None, xgb: Booster | None, X_test: ndarray, y_test: ndarray) -> dict
Evaluate whichever of GBM/XGBoost were actually trained (both, for an "ensemble" selection; exactly one otherwise) on the held-out test set, plus their mean-proba ensemble when both are present.
Source code in disease/model.py
compute_shap_importance ¶
TreeExplainer SHAP values for whichever model was actually selected/trained — see core.shap_utils for the axis-handling details.
Source code in disease/model.py
detect_hotspots ¶
detect_hotspots(df: DataFrame, pred_labels: ndarray, eps: float = DBSCAN_EPS, min_samples: int = DBSCAN_MIN_SAMPLES) -> list[dict]
DBSCAN spatial hotspot detection on High Risk (class 2) pixel centroids. Returns a list of cluster dicts with cluster_id, size, lon, lat. Requires df to contain 'lon' and 'lat' columns (preserved from GEE sample geometries).
Source code in disease/model.py
build_disease_charts ¶
build_disease_charts(eval_result: dict, shap_payload: dict, timeseries: dict[str, DataFrame], hotspots: list[dict], model_type: str = 'gbm') -> dict
Assemble frontend-ready chart payloads for the disease surveillance module.
Source code in disease/model.py
COG export¶
cog_export ¶
export_disease_cog ¶
export_disease_cog(gbm_model: GradientBoostingClassifier | None, xgb_model: Booster | None, scaler: StandardScaler, datasets: dict[str, Dataset], output_dir: str, prefix: str, model_type: str = 'gbm', aoi_geojson: dict | None = None) -> DiseaseCogResult
Build the full-AOI feature matrix from in-memory xarray Datasets, run inference with the selected model, classify into 3 risk classes, and write a Cloud Optimised GeoTIFF.
Parameters¶
gbm_model : trained GradientBoostingClassifier xgb_model : trained XGBoost Booster scaler : fitted StandardScaler (from DiseaseModel.scaler) datasets : dict returned by build_feature_datasets output_dir : local directory for COG output prefix : file prefix model_type : "gbm" | "xgboost" | "ensemble"
Returns¶
dict with keys: 'disease_risk' → path string 'risk_pct' → [low_pct, medium_pct, high_pct] over the AOI's valid (non-nodata) pixels, matching DISEASE_CLASSES order. This reflects every pixel actually painted on the map, unlike the training-sample-based stats DiseaseModel.predict() computes, which only covers the few hundred sampled points and can diverge sharply from the full-AOI distribution.
Source code in disease/cog_export.py
38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 | |