If you manage high-volume publishing, content decay is the silent killer of organic traffic. By the time you notice a drop in your dashboard, the damage is already done.
For my FlyRank Machine Learning Internship capstone, I set out to solve this by building an offline analytical pipeline that predicts content decay *before* traffic plummets, using historical search telemetry.
Here is a look under the hood at what I built, the architectural decisions that mattered, and the limitations of predicting the future.
Instead of letting editors blindly guess what to update, the pipeline translates 30,000 anonymized search performance rows into a prioritized action queue (e.g., refresh, monitor, expand_and_refresh).
My final model, an optimized Random Forest, achieved a 0.750 ROC-AUC and a 0.74 Precision@50, massively outperforming the naive recency baseline (which sat at 0.24 Precision@50).
When I started, I thought the hardest part would be tuning hyperparameters. I was wrong. The hardest part was structural hygiene.
My biggest design decision was abandoning random k-fold cross-validation in favor of a strict chronological client_holdout split.
When dealing with time-series search data and rolling-window velocity features (like log_impressions_90d), random splits inevitably leak future information into the training set. By enforcing a temporal embargo gap and grouping data strictly by page entities, I ensured the model’s evaluation metrics were completely leak-free and production-realistic.
Honesty reads as credibility in data science, so here is the major limitation of this build: It calculates statistical probability, not causal certainty.
The model identifies that a page is exhibiting the mathematical signals of decay based on historical telemetry. However, it cannot predict unobserved external shocks—like a sudden core algorithm update from a search engine or a macroeconomic shift in user intent. It is a powerful reviewer aid, but it requires a human editor to make the final publishing decision.
As part of the AI Fluency framework, I used Claude during this build not as a code-generator shortcut, but as a rigorous sounding board. I used it to stress-test my leakage assumptions, audit my validation split logic, and scaffold my documentation, while manually verifying every metric against my local telemetry execution.
You can inspect the full pipeline, evaluation metrics, and my generated action queues here:
If you are working on predictive SEO or time-series classification, I'd love to hear how you handle temporal data leakage in the comments!