Scikit-Learn, Pipeline Fundamentals: A Titanic Survival Prediction Guide

A developer published a guide demonstrating how Scikit-Learn's Pipeline and ColumnTransformer modules can be combined into a reproducible machine learning workflow, using the classic Titanic survival dataset as a case study. The tutorial walks through exploratory data analysis, pipeline-based feature engineering, model selection, and 5-fold stratified cross-validation, arguing that the approach prevents code duplication and data leakage during hyperparameter tuning.

Machine learning workflows often suffer from code duplication, data leakage, and hyperparameter tuning complexities. Scikit-Learn's Pipeline and ColumnTransformer modules simplify this by combining feature preprocessing and model estimation into an integrated, reproducible object. This guide demonstrates a complete predictive modeling workflow using the classic Titanic: Machine Learning from Disaster dataset, covering exploratory data analysis, pipeline-based feature engineering, model selection, and cross-validation. Begin by importing the necessary libraries for data manipulation, visualization, preprocessing, modeling, and evaluation. python import pandas as pd import numpy as np import seaborn as sns import matplotlib.pyplot as plt Model selection & evaluation from sklearn.model selection import train test split, StratifiedKFold, cross val score from sklearn.metrics import roc auc score, classification report Preprocessing & Pipelines from sklearn.pipeline import Pipeline from sklearn.impute import SimpleImputer from sklearn.preprocessing import StandardScaler, OneHotEncoder from sklearn.compose import ColumnTransformer Classifiers from sklearn.linear model import LogisticRegression from sklearn.neighbors import KNeighborsClassifier from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier from sklearn.svm import SVC from xgboost import XGBClassifier Load dataset titanic df = pd.read csv "data/train.csv" A quick inspection of the dataset structure reveals the feature types, missing values, and potential columns to drop. python def basic checks df : print "Shape:", df.shape print "Missing Values:\n", df.isnull .sum print "Data Types:\n", df.dtypes basic checks titanic df Age : Missing 19.87% of data 177 missing values . Embarked : Missing 0.22% of data 2 missing values . Cabin : Missing 77.10% of data 687 missing values . Due to the extreme proportion of missing data, the Cabin column is dropped. titanic df = titanic df.drop columns= 'Cabin' Identifiers such as PassengerId , Name , and Ticket carry no predictive power for survival and are excluded from the feature set $X$. We split the data using a 80/20 train-test split stratified by the target column Survived to maintain class proportions. Separate features and target X = titanic df.drop columns= 'PassengerId', 'Name', 'Ticket', 'Survived' y = titanic df 'Survived' Categorize feature types numeric columns = 'Age', 'SibSp', 'Parch', 'Fare' categorical columns = 'Pclass', 'Sex', 'Embarked' Stratified split X train, X test, y train, y test = train test split X, y, test size=0.2, random state=42, stratify=y To avoid data leakage such as calculating the mean or standard deviation on the entire dataset before splitting , transformations must be learned only on the training set. Using Pipeline and ColumnTransformer ensures these transformations are safely applied during cross-validation. 1. Numerical Pipeline: Impute missing values with median, then scale numerical pipeline = Pipeline steps= 'imputer', SimpleImputer strategy='median' , 'scaler', StandardScaler 2. Categorical Pipeline: Impute missing values with mode, then One-Hot Encode categorical pipeline = Pipeline steps= 'imputer', SimpleImputer strategy='most frequent' , 'onehot', OneHotEncoder handle unknown='ignore' 3. Combine Preprocessing Steps preprocessor = ColumnTransformer transformers= 'num', numerical pipeline, numeric columns , 'cat', categorical pipeline, categorical columns Evaluate multiple classification algorithms using 5-Fold Stratified Cross-Validation evaluated on the $F 1$-score metric. Define candidate models models = { 'Logistic Regression': LogisticRegression max iter=1000 , 'K-Nearest Neighbors': KNeighborsClassifier n neighbors=7, metric='euclidean' , 'Random Forest': RandomForestClassifier n estimators=300, max depth=15, random state=42 , 'Support Vector Machine': SVC kernel='rbf', probability=True, random state=42 , 'Gradient Boosting': GradientBoostingClassifier n estimators=300, max depth=15, random state=42 } Cross-Validation setup cv = StratifiedKFold n splits=5, shuffle=True, random state=42 results = {} for name, model in models.items : Chain preprocessing and model estimation into a single pipeline full pipeline = Pipeline steps= 'preprocessor', preprocessor , 'model', model cv scores = cross val score full pipeline, X train, y train, cv=cv, scoring='f1', n jobs=-1 results name = { 'Mean F1 Score': cv scores.mean , 'Std Dev': cv scores.std } Display results in a table results df = pd.DataFrame results .T.sort values by='Mean F1 Score', ascending=False | Model Classifier | Mean $F 1$ Score | Standard Deviation | |---|---|---| | Support Vector Machine SVM | 0.7447 | 0.0311 | | Random Forest | 0.7374 | 0.0200 | | Logistic Regression | 0.7301 | 0.0217 | | K-Nearest Neighbors | 0.7194 | 0.0229 | | Gradient Boosting | 0.7002 | 0.0233 | Preventing Data Leakage : Wrapping feature engineering steps inside a Pipeline guarantees that statistics e.g., mean/median for imputation, standard deviations for scaling are calculated strictly on training folds during cross-validation. Handling Mixed Data Types : ColumnTransformer cleanly routes numerical features to continuous scaling modules and categorical variables to encoding blocks. Model Verdict : The Support Vector Machine SVM achieved the highest mean $F 1$-score $0.7447$ , with Random Forest displaying the highest stability across folds with the lowest variance $\sigma = 0.0200$ .