Scikit-Learn, Pipeline Fundamentals: A Titanic Survival Prediction Guide

Machine learning workflows often suffer from code duplication, data leakage, and hyperparameter tuning complexities. Scikit-Learn's Pipeline and ColumnTransformer modules simplify this by combining feature preprocessing and model estimation into an integrated, reproducible object.

This guide demonstrates a complete predictive modeling workflow using the classic Titanic: Machine Learning from Disaster dataset, covering exploratory data analysis, pipeline-based feature engineering, model selection, and cross-validation.

Begin by importing the necessary libraries for data manipulation, visualization, preprocessing, modeling, and evaluation.

import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt

from sklearn.model_selection import train_test_split, StratifiedKFold, cross_val_score
from sklearn.metrics import roc_auc_score, classification_report

from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer

from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.svm import SVC
from xgboost import XGBClassifier

titanic_df = pd.read_csv("data/train.csv")

A quick inspection of the dataset structure reveals the feature types, missing values, and potential columns to drop.

def basic_checks(df):
    print("Shape:", df.shape)
    print("Missing Values:\n", df.isnull().sum())
    print("Data Types:\n", df.dtypes)

basic_checks(titanic_df)

Age: Missing 19.87% of data (177 missing values).

Embarked: Missing 0.22% of data (2 missing values).

Cabin: Missing 77.10% of data (687 missing values).

Due to the extreme proportion of missing data, the Cabin column is dropped.

titanic_df = titanic_df.drop(columns=['Cabin'])

Identifiers such as PassengerId, Name, and Ticket carry no predictive power for survival and are excluded from the feature set $X$. We split the data using a 80/20 train-test split stratified by the target column Survived to maintain class proportions.

X = titanic_df.drop(columns=['PassengerId', 'Name', 'Ticket', 'Survived'])
y = titanic_df['Survived']

numeric_columns = ['Age', 'SibSp', 'Parch', 'Fare']
categorical_columns = ['Pclass', 'Sex', 'Embarked']

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

To avoid data leakage (such as calculating the mean or standard deviation on the entire dataset before splitting), transformations must be learned only on the training set. Using Pipeline and ColumnTransformer ensures these transformations are safely applied during cross-validation.

numerical_pipeline = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='median')),
    ('scaler', StandardScaler())
])

categorical_pipeline = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('onehot', OneHotEncoder(handle_unknown='ignore'))
])

preprocessor = ColumnTransformer(transformers=[
    ('num', numerical_pipeline, numeric_columns),
    ('cat', categorical_pipeline, categorical_columns)
])

Evaluate multiple classification algorithms using 5-Fold Stratified Cross-Validation evaluated on the $F_1$-score metric.

models = {
    'Logistic Regression': LogisticRegression(max_iter=1000),
    'K-Nearest Neighbors': KNeighborsClassifier(n_neighbors=7, metric='euclidean'),
    'Random Forest': RandomForestClassifier(n_estimators=300, max_depth=15, random_state=42),
    'Support Vector Machine': SVC(kernel='rbf', probability=True, random_state=42),
    'Gradient Boosting': GradientBoostingClassifier(n_estimators=300, max_depth=15, random_state=42)
}

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
results = {}

for name, model in models.items():
    full_pipeline = Pipeline(steps=[
        ('preprocessor', preprocessor),
        ('model', model)
    ])

    cv_scores = cross_val_score(full_pipeline, X_train, y_train, cv=cv, scoring='f1', n_jobs=-1)
    results[name] = {
        'Mean F1 Score': cv_scores.mean(),
        'Std Dev': cv_scores.std()
    }

results_df = pd.DataFrame(results).T.sort_values(by='Mean F1 Score', ascending=False)

Preventing Data Leakage: Wrapping feature engineering steps inside a Pipeline guarantees that statistics (e.g., mean/median for imputation, standard deviations for scaling) are calculated strictly on training folds during cross-validation.

Handling Mixed Data Types: ColumnTransformer cleanly routes numerical features to continuous scaling modules and categorical variables to encoding blocks.

Model Verdict: The Support Vector Machine (SVM) achieved the highest mean $F_1$-score ($0.7447$), with Random Forest displaying the highest stability across folds with the lowest variance ($\sigma = 0.0200$).