Splitting your dataset into training and testing sets is the only way to detect if your model is actually learning patterns or just memorizing the noise (overfitting). If you evaluate your model on the same data it trained on, your accuracy metrics are essentially a lie.
For a more robust deep dive, I recommend moving from a simple split to K-Fold Cross-Validation, especially when working with smaller datasets where a single 20% slice might not be representative.
For anyone starting from scratch, the standard approach in Python is using train_test_split
from Scikit-Learn. Here is the basic implementation for a real-world AI workflow:
from sklearn.model_selection import train_test_split
import pandas as pd
df = pd.read_csv('data.csv')
X = df.drop('target', axis=1)
y = df['target']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
A few technical nuances that often trip people up:
The Without this, your split changes every time you run the code. Set this to a fixed integer (like 42) to ensure your experiments are reproducible.random_state
parameter:Data Leakage: This is the biggest killer in ML pipelines. You must split your databeforeperforming any scaling or imputation. If you calculate the mean of the entire dataset and then split, information from the test set has "leaked" into the training set.Stratification: If you are dealing with an imbalanced dataset (e.g., 99% "No" and 1% "Yes"), a random split might leave your test set with zero positive cases. Usestratify=y
to maintain the class proportions across both sets.
For a more robust deep dive, I recommend moving from a simple split to K-Fold Cross-Validation, especially when working with smaller datasets where a single 20% slice might not be representative.
Next Promograph: Solving Retail Promotion Waste with GNNs →
All Replies （3） #
Z
Don't forget to shuffle your data first or you might end up with biased splits.
0
L
do u think stratifed splitting is better for imbalanced classes or just stick to random?
0
N
Learned this the hard way after my model hit 99% accuracy but failed in production.
0