{"slug": "scikit-learn-pipeline-fundamentals-a-titanic-survival-prediction-guide", "title": "Scikit-Learn, Pipeline Fundamentals: A Titanic Survival Prediction Guide", "summary": "A developer published a guide demonstrating how Scikit-Learn's Pipeline and ColumnTransformer modules can be combined into a reproducible machine learning workflow, using the classic Titanic survival dataset as a case study. The tutorial walks through exploratory data analysis, pipeline-based feature engineering, model selection, and 5-fold stratified cross-validation, arguing that the approach prevents code duplication and data leakage during hyperparameter tuning.", "body_md": "Machine learning workflows often suffer from code duplication, data leakage, and hyperparameter tuning complexities. Scikit-Learn's `Pipeline` and `ColumnTransformer` modules simplify this by combining feature preprocessing and model estimation into an integrated, reproducible object.\n\nThis guide demonstrates a complete predictive modeling workflow using the classic **Titanic: Machine Learning from Disaster** dataset, covering exploratory data analysis, pipeline-based feature engineering, model selection, and cross-validation.\n\nBegin by importing the necessary libraries for data manipulation, visualization, preprocessing, modeling, and evaluation.\n\n``` python\nimport pandas as pd\nimport numpy as np\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\n# Model selection & evaluation\nfrom sklearn.model_selection import train_test_split, StratifiedKFold, cross_val_score\nfrom sklearn.metrics import roc_auc_score, classification_report\n\n# Preprocessing & Pipelines\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.impute import SimpleImputer\nfrom sklearn.preprocessing import StandardScaler, OneHotEncoder\nfrom sklearn.compose import ColumnTransformer\n\n# Classifiers\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier\nfrom sklearn.svm import SVC\nfrom xgboost import XGBClassifier\n\n# Load dataset\ntitanic_df = pd.read_csv(\"data/train.csv\")\n```\n\nA quick inspection of the dataset structure reveals the feature types, missing values, and potential columns to drop.\n\n``` python\ndef basic_checks(df):\n    print(\"Shape:\", df.shape)\n    print(\"Missing Values:\\n\", df.isnull().sum())\n    print(\"Data Types:\\n\", df.dtypes)\n\nbasic_checks(titanic_df)\n```\n\n**`Age`**: Missing **19.87%** of data (177 missing values).\n\n**`Embarked`**: Missing **0.22%** of data (2 missing values).\n\n**`Cabin`**: Missing **77.10%** of data (687 missing values).\n\nDue to the extreme proportion of missing data, the `Cabin` column is dropped.\n\n```\ntitanic_df = titanic_df.drop(columns=['Cabin'])\n```\n\nIdentifiers such as `PassengerId`, `Name`, and `Ticket` carry no predictive power for survival and are excluded from the feature set $X$. We split the data using a **80/20 train-test split** stratified by the target column `Survived` to maintain class proportions.\n\n```\n# Separate features and target\nX = titanic_df.drop(columns=['PassengerId', 'Name', 'Ticket', 'Survived'])\ny = titanic_df['Survived']\n\n# Categorize feature types\nnumeric_columns = ['Age', 'SibSp', 'Parch', 'Fare']\ncategorical_columns = ['Pclass', 'Sex', 'Embarked']\n\n# Stratified split\nX_train, X_test, y_train, y_test = train_test_split(\n    X, y, test_size=0.2, random_state=42, stratify=y\n)\n```\n\nTo avoid **data leakage** (such as calculating the mean or standard deviation on the entire dataset before splitting), transformations must be learned only on the training set. Using `Pipeline` and `ColumnTransformer` ensures these transformations are safely applied during cross-validation.\n\n```\n# 1. Numerical Pipeline: Impute missing values with median, then scale\nnumerical_pipeline = Pipeline(steps=[\n    ('imputer', SimpleImputer(strategy='median')),\n    ('scaler', StandardScaler())\n])\n\n# 2. Categorical Pipeline: Impute missing values with mode, then One-Hot Encode\ncategorical_pipeline = Pipeline(steps=[\n    ('imputer', SimpleImputer(strategy='most_frequent')),\n    ('onehot', OneHotEncoder(handle_unknown='ignore'))\n])\n\n# 3. Combine Preprocessing Steps\npreprocessor = ColumnTransformer(transformers=[\n    ('num', numerical_pipeline, numeric_columns),\n    ('cat', categorical_pipeline, categorical_columns)\n])\n```\n\nEvaluate multiple classification algorithms using **5-Fold Stratified Cross-Validation** evaluated on the $F_1$-score metric.\n\n```\n# Define candidate models\nmodels = {\n    'Logistic Regression': LogisticRegression(max_iter=1000),\n    'K-Nearest Neighbors': KNeighborsClassifier(n_neighbors=7, metric='euclidean'),\n    'Random Forest': RandomForestClassifier(n_estimators=300, max_depth=15, random_state=42),\n    'Support Vector Machine': SVC(kernel='rbf', probability=True, random_state=42),\n    'Gradient Boosting': GradientBoostingClassifier(n_estimators=300, max_depth=15, random_state=42)\n}\n\n# Cross-Validation setup\ncv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)\nresults = {}\n\nfor name, model in models.items():\n    # Chain preprocessing and model estimation into a single pipeline\n    full_pipeline = Pipeline(steps=[\n        ('preprocessor', preprocessor),\n        ('model', model)\n    ])\n\n    cv_scores = cross_val_score(full_pipeline, X_train, y_train, cv=cv, scoring='f1', n_jobs=-1)\n    results[name] = {\n        'Mean F1 Score': cv_scores.mean(),\n        'Std Dev': cv_scores.std()\n    }\n\n# Display results in a table\nresults_df = pd.DataFrame(results).T.sort_values(by='Mean F1 Score', ascending=False)\n```\n\n| Model Classifier | Mean $F_1$ Score | Standard Deviation | \n|---|---|---|\n| **Support Vector Machine (SVM)** | **0.7447** | 0.0311 | \n| **Random Forest** | **0.7374** | 0.0200 | \n| **Logistic Regression** | **0.7301** | 0.0217 | \n| **K-Nearest Neighbors** | **0.7194** | 0.0229 | \n| **Gradient Boosting** | **0.7002** | 0.0233 | \n\n**Preventing Data Leakage**: Wrapping feature engineering steps inside a `Pipeline` guarantees that statistics (e.g., mean/median for imputation, standard deviations for scaling) are calculated strictly on training folds during cross-validation.\n\n**Handling Mixed Data Types**: `ColumnTransformer` cleanly routes numerical features to continuous scaling modules and categorical variables to encoding blocks.\n\n**Model Verdict**: The **Support Vector Machine (SVM)** achieved the highest mean $F_1$-score ($0.7447$), with **Random Forest** displaying the highest stability across folds with the lowest variance ($\\sigma = 0.0200$).", "url": "https://wpnews.pro/news/scikit-learn-pipeline-fundamentals-a-titanic-survival-prediction-guide", "canonical_source": "https://dev.to/mark_glemba_962f6bc8a12dd/scikit-learn-pipeline-fundamentals-a-titanic-survival-prediction-guide-2nlp", "published_at": "2026-09-12 20:36:52+00:00", "updated_at": "2026-09-12 21:23:58.453349+00:00", "lang": "en", "topics": ["machine-learning", "developer-tools"], "entities": ["Scikit-Learn", "Titanic: Machine Learning from Disaster", "LogisticRegression", "RandomForestClassifier", "XGBoost"], "alternates": {"html": "https://wpnews.pro/news/scikit-learn-pipeline-fundamentals-a-titanic-survival-prediction-guide", "markdown": "https://wpnews.pro/news/scikit-learn-pipeline-fundamentals-a-titanic-survival-prediction-guide.md", "text": "https://wpnews.pro/news/scikit-learn-pipeline-fundamentals-a-titanic-survival-prediction-guide.txt", "jsonld": "https://wpnews.pro/news/scikit-learn-pipeline-fundamentals-a-titanic-survival-prediction-guide.jsonld"}}