This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady
#
🦥 DataJanitor AI — From Raw Data to Real Insights
#
What I Built
Data analysis shouldn't require hours of manual exploration, repetitive plotting, or writing boilerplate machine-learning code.
That's why I'm building DataJanitor AI — an AI-powered data exploration and machine-learning platform that helps users go from raw datasets to meaningful insights with less effort.
The goal is simple: upload your data, understand it, visualize it, and eventually train machine-learning models without writing everything from scratch.
📊 Phase 1: Smart Data Visualization
Users can upload CSV or Excel files and explore their datasets through:
Univariate Analysis: Understand individual variables using histograms, box plots, bar charts, and descriptive statistics. #
Bivariate Analysis: Discover relationships between two variables using scatter plots, correlation analysis, grouped comparisons, and other suitable visualizations. #
Multivariate Analysis: Explore patterns across multiple variables through correlation heatmaps, pair plots, and more advanced visualizations. #
Dataset Profiling: Identify column types, missing values, duplicate records, and potential data-quality issues. #
AI-Powered Insights: Generate plain-language explanations of statistical patterns and visualization results.
🤖 Phase 2: Machine Learning
The next milestone is to let users select their input features (X) and target variable (** y**) and train machine-learning models directly from the interface.
Planned capabilities include:
- Automatic classification and regression task suggestions.
- Data preprocessing and train-test splitting.
- Training and comparing suitable machine-learning models.
- Evaluation metrics and feature-importance visualizations.
- Predictions on new data and downloadable model artifacts.
The idea is to make data exploration and machine learning more accessible to students, developers, analysts, and anyone working with structured data.
#
Code
GitHub Repository: DataJanitor AI
#
How I Built It
I'm building DataJanitor AI with an open-source-first approach.
Planned technology stack:
Frontend: React + Tailwind CSS for an interactive analytics dashboard. #
Backend: Python + FastAPI for dataset processing and analysis APIs. #
Data Processing: Pandas and NumPy for data profiling and statistical calculations. #
Visualizations: Plotly for interactive charts and exploration. #
Machine Learning: Scikit-learn for preprocessing, training, and evaluation. #
Open-Weight AI: Gemma or another compatible open-weight model, served locally through Ollama, for natural-language explanations and dataset insights.
The application will use deterministic statistical calculations for numerical results and machine-learning evaluation. The AI layer will help explain findings in understandable language rather than inventing statistics.
The planned workflow is:
Upload → Profile → Visualize → Understand → Train → Predict
#
Why Does Open Innovation Matter?
Data can be sensitive. A dataset may contain business records, customer information, financial figures, or research data that users don't want to send to an external service.
Open innovation gives DataJanitor AI the flexibility to support local inference, model choice, and transparent data-processing workflows.
By using open-source libraries and compatible open-weight models, the project aims to:
- Keep sensitive datasets on the user's machine when using the local workflow.
- Reduce reliance on proprietary AI APIs for dataset explanations.
- Allow users to experiment with different models.
- Make core statistical analysis and machine-learning workflows accessible without depending on a paid AI service.
For this project, AI should make analysis easier — not make users dependent on a black box.
#
My Agent Session
Optional — I'll add a DevRelay agent session or development trace here if available.
#
The Lazy Payoff
Imagine up a spreadsheet and getting an initial understanding of its structure, distributions, relationships, and potential data-quality problems without manually creating every chart.
Instead of spending time on repetitive analysis setup, users can focus on interpreting results and making decisions.
The longer-term goal is to make the journey from a raw dataset to a trained predictive model much simpler.
#
Prize Categories
The following are the partner categories I'm considering for DataJanitor AI. I'll include only the categories supported by the final implementation.
Best Use of Gemma — Use Google's open-weight Gemma model for dataset explanations and AI-generated insights. #
Best Use of TabPFN — Integrate TabPFN for suitable tabular classification or prediction tasks in the machine-learning phase. #
Best Use of DigitalOcean — Deploy the application or serve its AI component using DigitalOcean infrastructure.
- **Best Use of Tinker ** — Fine-tune an open-weight model for chart recommendations or structured data-analysis planning and evaluate it against the base model.
#
Tags
#
devchallenge #hf26challenge #ai #machinelearning #datascience #opensource #python #react #exploratorydataanalysis #automl