{"slug": "predicting-severe-acute-respiratory-illness-sari-mortality-with-scikit-learn", "title": "Predicting Severe Acute Respiratory Illness (SARI) Mortality with Scikit-Learn Halving Search", "summary": "A developer built an end-to-end Scikit-Learn pipeline to predict mortality versus recovery for Severe Acute Respiratory Illness (SARI) patients using over 260,000 Brazilian DataSUS records from 2025. The pipeline reads selected columns from Parquet files, normalizes comorbidities to binary flags, applies a stratified train-test split to handle roughly 8% mortality imbalance, and tunes a Random Forest with HalvingRandomSearchCV successive halving instead of exhaustive grid search.", "body_md": "When dealing with public health data, computational efficiency and rapid experimentation are critical factors. In this tutorial, we will build an end-to-end Machine Learning pipeline using Python and Scikit-Learn to predict the clinical outcome of patients notified with Severe Acute Respiratory Illness (SARI/SRAG) in Brazil.\n\nWe will work with a large-scale real-world dataset from DataSUS, clean and preprocess clinical features, handle class imbalance, and use an efficient search strategy called **Successive Halving** to optimize our Random Forest hyperparameters in a fraction of the time required by exhaustive Grid Search.\n\nSevere Acute Respiratory Illness (SARI) remains a major global public health concern. Predicting patient outcomes (Recovery vs. Death) based on initial demographics, symptoms, and comorbidities assists healthcare managers in risk stratification and resource allocation.\n\nOur dataset contains over 260,000 patient records from 2025 in Brazil. We will focus on key variables such as:\n\n`CS_SEXO`) and Age (` NU_IDADE_N`).` UTI`) and Ventilatory support (` SUPORT_VEN`).\nBecause raw public health datasets contain dozens of unused epidemiological columns, reading uncompressed CSVs consumes excessive memory. We use the **Apache Parquet format** to enable columnar selection and fast I/O:\n\n``` python\nimport pandas as pd\nimport numpy as np\n\nfile_path = 'INFLUD25-data.parquet'\ntarget_columns = [\n    'CS_SEXO', 'NU_IDADE_N', 'EVOLUCAO', 'UTI', 'SUPORT_VEN', \n    'CARDIOPATI', 'DIABETES', 'OBESIDADE', 'RENAL', 'ASMA', 'IMUNODEPRE'\n]\n\ndf = pd.read_parquet(file_path, columns=target_columns)\nprint(f\"Dataset Dimensions: {df.shape}\")\n```\n\nPublic health records contain ambiguous missing flags. We filter out unknown outcomes to build our binary target (`TARGET`), where:\n\n`0` = Recovery (Cura)`1` = Mortality (Óbito)\nWe then normalize categorical comorbidity features into strict binary (`1` = Yes, `0` = No/Unknown) values:\n\n```\n# Filter valid clinical endpoints\ndf['EVOLUCAO'] = pd.to_numeric(df['EVOLUCAO'], errors='coerce')\ndf_clean = df[df['EVOLUCAO'].isin([1, 2])].copy()\ndf_clean['TARGET'] = df_clean['EVOLUCAO'].map({1: 0, 2: 1})\ndf_clean = df_clean.drop('EVOLUCAO', axis=1)\n\n# Normalize clinical risk factors to binary\ncomorbidity_cols = [\n    'UTI', 'SUPORT_VEN', 'CARDIOPATI', 'DIABETES', \n    'OBESIDADE', 'RENAL', 'ASMA', 'IMUNODEPRE'\n]\nfor col in comorbidity_cols:\n    df_clean[col] = pd.to_numeric(df_clean[col], errors='coerce')\n    df_clean[col] = df_clean[col].apply(lambda x: 1 if x == 1 else 0)\n\n# Impute and clean demographics\ndf_clean['NU_IDADE_N'] = pd.to_numeric(df_clean['NU_IDADE_N'], errors='coerce')\ndf_clean['NU_IDADE_N'] = df_clean['NU_IDADE_N'].fillna(df_clean['NU_IDADE_N'].median())\ndf_clean['CS_SEXO'] = df_clean['CS_SEXO'].map({'M': 0, 'F': 1, 'm': 0, 'f': 1}).fillna(0)\n```\n\nBecause mortality is an imbalanced class (~8% positive occurrences), standard random partitioning can cause class distribution drift. We apply a stratified split:\n\n``` python\nfrom sklearn.model_selection import train_test_split\n\nX = df_clean.drop('TARGET', axis=1)\ny = df_clean['TARGET']\n\nX_train, X_test, y_train, y_test = train_test_split(\n    X, y, test_size=0.3, random_state=42, stratify=y\n)\n```\n\nTraditional `GridSearchCV` evaluates every parameter combination across the entire training dataset. For large datasets, this approach is computationally prohibitive.\n\nInstead, we use **Successive Halving** via `HalvingRandomSearchCV`:\n\n`factor=3`), and surviving models are re-evaluated.\n\n``` python\nfrom sklearn.experimental import enable_halving_search_cv\nfrom sklearn.model_selection import HalvingRandomSearchCV\nfrom sklearn.ensemble import RandomForestClassifier\n\nparam_distributions = {\n    'n_estimators': [50, 100, 200, 300],\n    'max_depth': [10, 20, 30, None],\n    'min_samples_split': [2, 5, 10],\n    'min_samples_leaf': [1, 2, 4],\n    'criterion': ['gini', 'entropy']\n}\n\nrf_base = RandomForestClassifier(random_state=42, n_jobs=-1)\n\nhalving_search = HalvingRandomSearchCV(\n    estimator=rf_base,\n    param_distributions=param_distributions,\n    factor=3,\n    resource='n_samples',\n    scoring='accuracy',\n    random_state=42,\n    cv=3,\n    verbose=1\n)\n\nhalving_search.fit(X_train, y_train)\nbest_model = halving_search.best_estimator_\nprint(f\"Optimal Parameters: {halving_search.best_params_}\")\n```\n\nWith our optimized ensemble model persisted, we can evaluate arbitrary, unseen patient records via direct inference:\n\n```\nhigh_risk_patient = {\n    'CS_SEXO': 1,\n    'NU_IDADE_N': 68,\n    'UTI': 1,\n    'SUPORT_VEN': 1,\n    'CARDIOPATI': 1,\n    'DIABETES': 1,\n    'OBESIDADE': 1,\n    'RENAL': 0,\n    'ASMA': 0,\n    'IMUNODEPRE': 0\n}\n\ninput_df = pd.DataFrame([high_risk_patient])\nmortality_probability = best_model.predict_proba(input_df)[0][1]\n\nprint(f\"Calculated Risk Probability: {mortality_probability * 100:.2f}%\")\n```\n\nUsing `HalvingRandomSearchCV` dramatically reduces iteration time on large datasets without sacrificing model performance. When combined with rigorous binary feature normalization and stratified sampling, developers can build robust risk-stratification pipelines ready for healthcare operational workflows.", "url": "https://wpnews.pro/news/predicting-severe-acute-respiratory-illness-sari-mortality-with-scikit-learn", "canonical_source": "https://dev.to/larissa_vitoria/predicting-severe-acute-respiratory-illness-sari-mortality-with-scikit-learn-halving-search-21f1", "published_at": "2026-11-04 00:42:40+00:00", "updated_at": "2026-11-04 00:52:14.368499+00:00", "lang": "en", "topics": ["machine-learning", "ai-research"], "entities": ["Scikit-Learn", "DataSUS", "Brazil", "Random Forest", "HalvingRandomSearchCV", "Apache Parquet"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/predicting-severe-acute-respiratory-illness-sari-mortality-with-scikit-learn", "markdown": "https://wpnews.pro/news/predicting-severe-acute-respiratory-illness-sari-mortality-with-scikit-learn.md", "text": "https://wpnews.pro/news/predicting-severe-acute-respiratory-illness-sari-mortality-with-scikit-learn.txt", "jsonld": "https://wpnews.pro/news/predicting-severe-acute-respiratory-illness-sari-mortality-with-scikit-learn.jsonld"}}