# Scikit-Learn, Pipeline Fundamentals: A Titanic Survival Prediction Guide

> Source: <https://dev.to/mark_glemba_962f6bc8a12dd/scikit-learn-pipeline-fundamentals-a-titanic-survival-prediction-guide-2nlp>
> Published: 2026-09-12 20:36:52+00:00

Machine learning workflows often suffer from code duplication, data leakage, and hyperparameter tuning complexities. Scikit-Learn's `Pipeline` and `ColumnTransformer` modules simplify this by combining feature preprocessing and model estimation into an integrated, reproducible object.

This guide demonstrates a complete predictive modeling workflow using the classic **Titanic: Machine Learning from Disaster** dataset, covering exploratory data analysis, pipeline-based feature engineering, model selection, and cross-validation.

Begin by importing the necessary libraries for data manipulation, visualization, preprocessing, modeling, and evaluation.

``` python
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt

# Model selection & evaluation
from sklearn.model_selection import train_test_split, StratifiedKFold, cross_val_score
from sklearn.metrics import roc_auc_score, classification_report

# Preprocessing & Pipelines
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer

# Classifiers
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.svm import SVC
from xgboost import XGBClassifier

# Load dataset
titanic_df = pd.read_csv("data/train.csv")
```

A quick inspection of the dataset structure reveals the feature types, missing values, and potential columns to drop.

``` python
def basic_checks(df):
    print("Shape:", df.shape)
    print("Missing Values:\n", df.isnull().sum())
    print("Data Types:\n", df.dtypes)

basic_checks(titanic_df)
```

**`Age`**: Missing **19.87%** of data (177 missing values).

**`Embarked`**: Missing **0.22%** of data (2 missing values).

**`Cabin`**: Missing **77.10%** of data (687 missing values).

Due to the extreme proportion of missing data, the `Cabin` column is dropped.

```
titanic_df = titanic_df.drop(columns=['Cabin'])
```

Identifiers such as `PassengerId`, `Name`, and `Ticket` carry no predictive power for survival and are excluded from the feature set $X$. We split the data using a **80/20 train-test split** stratified by the target column `Survived` to maintain class proportions.

```
# Separate features and target
X = titanic_df.drop(columns=['PassengerId', 'Name', 'Ticket', 'Survived'])
y = titanic_df['Survived']

# Categorize feature types
numeric_columns = ['Age', 'SibSp', 'Parch', 'Fare']
categorical_columns = ['Pclass', 'Sex', 'Embarked']

# Stratified split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)
```

To avoid **data leakage** (such as calculating the mean or standard deviation on the entire dataset before splitting), transformations must be learned only on the training set. Using `Pipeline` and `ColumnTransformer` ensures these transformations are safely applied during cross-validation.

```
# 1. Numerical Pipeline: Impute missing values with median, then scale
numerical_pipeline = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='median')),
    ('scaler', StandardScaler())
])

# 2. Categorical Pipeline: Impute missing values with mode, then One-Hot Encode
categorical_pipeline = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('onehot', OneHotEncoder(handle_unknown='ignore'))
])

# 3. Combine Preprocessing Steps
preprocessor = ColumnTransformer(transformers=[
    ('num', numerical_pipeline, numeric_columns),
    ('cat', categorical_pipeline, categorical_columns)
])
```

Evaluate multiple classification algorithms using **5-Fold Stratified Cross-Validation** evaluated on the $F_1$-score metric.

```
# Define candidate models
models = {
    'Logistic Regression': LogisticRegression(max_iter=1000),
    'K-Nearest Neighbors': KNeighborsClassifier(n_neighbors=7, metric='euclidean'),
    'Random Forest': RandomForestClassifier(n_estimators=300, max_depth=15, random_state=42),
    'Support Vector Machine': SVC(kernel='rbf', probability=True, random_state=42),
    'Gradient Boosting': GradientBoostingClassifier(n_estimators=300, max_depth=15, random_state=42)
}

# Cross-Validation setup
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
results = {}

for name, model in models.items():
    # Chain preprocessing and model estimation into a single pipeline
    full_pipeline = Pipeline(steps=[
        ('preprocessor', preprocessor),
        ('model', model)
    ])

    cv_scores = cross_val_score(full_pipeline, X_train, y_train, cv=cv, scoring='f1', n_jobs=-1)
    results[name] = {
        'Mean F1 Score': cv_scores.mean(),
        'Std Dev': cv_scores.std()
    }

# Display results in a table
results_df = pd.DataFrame(results).T.sort_values(by='Mean F1 Score', ascending=False)
```

| Model Classifier | Mean $F_1$ Score | Standard Deviation | 
|---|---|---|
| **Support Vector Machine (SVM)** | **0.7447** | 0.0311 | 
| **Random Forest** | **0.7374** | 0.0200 | 
| **Logistic Regression** | **0.7301** | 0.0217 | 
| **K-Nearest Neighbors** | **0.7194** | 0.0229 | 
| **Gradient Boosting** | **0.7002** | 0.0233 | 

**Preventing Data Leakage**: Wrapping feature engineering steps inside a `Pipeline` guarantees that statistics (e.g., mean/median for imputation, standard deviations for scaling) are calculated strictly on training folds during cross-validation.

**Handling Mixed Data Types**: `ColumnTransformer` cleanly routes numerical features to continuous scaling modules and categorical variables to encoding blocks.

**Model Verdict**: The **Support Vector Machine (SVM)** achieved the highest mean $F_1$-score ($0.7447$), with **Random Forest** displaying the highest stability across folds with the lowest variance ($\sigma = 0.0200$).