Predicting Severe Acute Respiratory Illness (SARI) Mortality with Scikit-Learn Halving Search

When dealing with public health data, computational efficiency and rapid experimentation are critical factors. In this tutorial, we will build an end-to-end Machine Learning pipeline using Python and Scikit-Learn to predict the clinical outcome of patients notified with Severe Acute Respiratory Illness (SARI/SRAG) in Brazil.

We will work with a large-scale real-world dataset from DataSUS, clean and preprocess clinical features, handle class imbalance, and use an efficient search strategy called Successive Halving to optimize our Random Forest hyperparameters in a fraction of the time required by exhaustive Grid Search.

Severe Acute Respiratory Illness (SARI) remains a major global public health concern. Predicting patient outcomes (Recovery vs. Death) based on initial demographics, symptoms, and comorbidities assists healthcare managers in risk stratification and resource allocation.

Our dataset contains over 260,000 patient records from 2025 in Brazil. We will focus on key variables such as:

CS_SEXO) and Age ( NU_IDADE_N). UTI) and Ventilatory support ( SUPORT_VEN). Because raw public health datasets contain dozens of unused epidemiological columns, reading uncompressed CSVs consumes excessive memory. We use the Apache Parquet format to enable columnar selection and fast I/O:

import pandas as pd
import numpy as np

file_path = 'INFLUD25-data.parquet'
target_columns = [
    'CS_SEXO', 'NU_IDADE_N', 'EVOLUCAO', 'UTI', 'SUPORT_VEN', 
    'CARDIOPATI', 'DIABETES', 'OBESIDADE', 'RENAL', 'ASMA', 'IMUNODEPRE'
]

df = pd.read_parquet(file_path, columns=target_columns)
print(f"Dataset Dimensions: {df.shape}")

Public health records contain ambiguous missing flags. We filter out unknown outcomes to build our binary target (TARGET), where:

0 = Recovery (Cura)1 = Mortality (Óbito) We then normalize categorical comorbidity features into strict binary (1 = Yes, 0 = No/Unknown) values:

df['EVOLUCAO'] = pd.to_numeric(df['EVOLUCAO'], errors='coerce')
df_clean = df[df['EVOLUCAO'].isin([1, 2])].copy()
df_clean['TARGET'] = df_clean['EVOLUCAO'].map({1: 0, 2: 1})
df_clean = df_clean.drop('EVOLUCAO', axis=1)

comorbidity_cols = [
    'UTI', 'SUPORT_VEN', 'CARDIOPATI', 'DIABETES', 
    'OBESIDADE', 'RENAL', 'ASMA', 'IMUNODEPRE'
]
for col in comorbidity_cols:
    df_clean[col] = pd.to_numeric(df_clean[col], errors='coerce')
    df_clean[col] = df_clean[col].apply(lambda x: 1 if x == 1 else 0)

df_clean['NU_IDADE_N'] = pd.to_numeric(df_clean['NU_IDADE_N'], errors='coerce')
df_clean['NU_IDADE_N'] = df_clean['NU_IDADE_N'].fillna(df_clean['NU_IDADE_N'].median())
df_clean['CS_SEXO'] = df_clean['CS_SEXO'].map({'M': 0, 'F': 1, 'm': 0, 'f': 1}).fillna(0)

Because mortality is an imbalanced class (~8% positive occurrences), standard random partitioning can cause class distribution drift. We apply a stratified split:

from sklearn.model_selection import train_test_split

X = df_clean.drop('TARGET', axis=1)
y = df_clean['TARGET']

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=42, stratify=y
)

Traditional GridSearchCV evaluates every parameter combination across the entire training dataset. For large datasets, this approach is computationally prohibitive.

Instead, we use Successive Halving via HalvingRandomSearchCV:

factor=3), and surviving models are re-evaluated.

from sklearn.experimental import enable_halving_search_cv
from sklearn.model_selection import HalvingRandomSearchCV
from sklearn.ensemble import RandomForestClassifier

param_distributions = {
    'n_estimators': [50, 100, 200, 300],
    'max_depth': [10, 20, 30, None],
    'min_samples_split': [2, 5, 10],
    'min_samples_leaf': [1, 2, 4],
    'criterion': ['gini', 'entropy']
}

rf_base = RandomForestClassifier(random_state=42, n_jobs=-1)

halving_search = HalvingRandomSearchCV(
    estimator=rf_base,
    param_distributions=param_distributions,
    factor=3,
    resource='n_samples',
    scoring='accuracy',
    random_state=42,
    cv=3,
    verbose=1
)

halving_search.fit(X_train, y_train)
best_model = halving_search.best_estimator_
print(f"Optimal Parameters: {halving_search.best_params_}")

With our optimized ensemble model persisted, we can evaluate arbitrary, unseen patient records via direct inference:

high_risk_patient = {
    'CS_SEXO': 1,
    'NU_IDADE_N': 68,
    'UTI': 1,
    'SUPORT_VEN': 1,
    'CARDIOPATI': 1,
    'DIABETES': 1,
    'OBESIDADE': 1,
    'RENAL': 0,
    'ASMA': 0,
    'IMUNODEPRE': 0
}

input_df = pd.DataFrame([high_risk_patient])
mortality_probability = best_model.predict_proba(input_df)[0][1]

print(f"Calculated Risk Probability: {mortality_probability * 100:.2f}%")

Using HalvingRandomSearchCV dramatically reduces iteration time on large datasets without sacrificing model performance. When combined with rigorous binary feature normalization and stratified sampling, developers can build robust risk-stratification pipelines ready for healthcare operational workflows.