{"slug": "i-was-learning-pytorch-then-i-accidentally-started-building-my-own-ai-training", "title": "I Was Learning PyTorch, Then I Accidentally Started Building My Own AI Training Framework", "summary": "A developer building Tensorless, an AI training framework, started with two PyTorch models to learn the basics, then created a lightweight framework that automates training setup. The project evolved from a PyTorch wrapper into a custom neural/training system with CUDA optimization, TPU support, and automatic configuration, tested with a cat chatbot named CatTongue.", "body_md": "Honestly, **Tensorless was not supposed to become this big.**\n\nI was learning PyTorch.\n\nI wanted to understand how neural networks and model training actually worked, so before starting Tensorless, I built **two AI models with PyTorch**.\n\nThose two projects were mainly for learning. I wanted to understand the basics by actually building things instead of only reading tutorials.\n\nAnd after doing that, I started thinking about something pretty simple:\n\nWhy does training a model have to involve so much setup every time?\n\nIf I already have the training data, why can't I just give it to a framework and let it figure out most of the boring stuff?\n\nThat was the beginning of Tensorless.\n\nBefore Tensorless existed, I spent time learning PyTorch by actually building **two AI models**.\n\nThat gave me a basic understanding of how the whole process worked:\n\nBuilding those two models was important because I wasn't starting Tensorless completely blind.\n\nI had already experienced the process myself.\n\nAnd after doing it twice, I started noticing how much of the setup was repetitive.\n\nSometimes I don't want to spend a huge amount of time configuring everything.\n\nSometimes I just want to get a result quickly.\n\nSo I started thinking:\n\nWhat if most of this could just happen automatically?\n\nMy initial goal wasn't to create another huge machine-learning framework.\n\nI wanted something more like:\n\nGive Tensorless your training data\n\n↓\n\nTensorless figures out the setup\n\n↓\n\nTrain\n\n↓\n\nGet your model\n\nThe idea was basically:\n\nPut data in → go.\n\nIf you're a beginner, you shouldn't need to understand every training parameter before you can train a model.\n\nBut if you're an advanced programmer, you should still be able to configure everything yourself.\n\nSo the philosophy became:\n\nSimple by default. Configurable when you need it.\n\nThe First Tensorless Version Used PyTorch\n\nThe first version of Tensorless actually used PyTorch underneath.\n\nI wasn't trying to reinvent neural networks at that point.\n\nI was trying to automate the training process.\n\nThe configuration was mostly determined automatically using fixed conditions in the code.\n\nSo instead of manually deciding every parameter, Tensorless would make those decisions for you.\n\nThat was the first version.\n\nAnd honestly, it worked surprisingly well for what I wanted.\n\nBut then I ran into another problem.\n\nI Wanted It To Be Small\n\nOne of the reasons I started thinking differently about the architecture was the amount of stuff involved in a PyTorch-based workflow.\n\nWhen I was experimenting with the project, the dependencies and modules could become 500+ MB.\n\nAnd I started thinking:\n\n\"I'm trying to make training simpler and lightweight, but I'm bringing in a huge stack just to do it.\"\n\nI didn't want Tensorless to become a massive package full of components that users might never need.\n\nI wanted the package to contain the things actually required for its own workflow.\n\nSo I decided to go much further.\n\nThen I Made A Slightly Insane Decision\n\nI started building more of the system myself.\n\nInstead of keeping Tensorless as a PyTorch wrapper forever, I started working toward my own neural/training system.\n\nThat meant getting into things I originally hadn't planned to touch.\n\nI started working on:\n\nNeural network components\n\nTraining\n\nCUDA optimization\n\nData optimization\n\nData pipelines\n\nTPU support\n\nAutomatic configuration\n\nTokenization\n\nBPE\n\nModel saving/loading\n\nAnd other training infrastructure\n\nAt this point, Tensorless had changed completely.\n\nIt wasn't just:\n\n\"PyTorch, but with easier configuration.\"\n\nIt was becoming an attempt to build a lightweight training system around the workflow I wanted.\n\nThen I Needed Something Real To Test It On\n\nSo I built a small chatbot.\n\nA cat chatbot.\n\nI called it CatTongue. 🐈\n\nThe model was trained on CPU, and with the PyTorch-based version of my training system, I was getting results that were honestly pretty fun.\n\nFor example:\n\nYou: \"Do you trust me?\"\n\nCat: \"only if you walk toward the kitchen. 😼🐾\"\n\nYou: \"Who is your enemy?\"\n\nCat: \"empty boxes. 📦😾 they are useful for exactly zero seconds.\"\n\nYou: \"Who is your favorite human?\"\n\nCat: \"are you holding food? 🤨🍗\"\n\nYou: \"Do you have a best friend?\"\n\nCat: \"the vacuum cleaner. 😾🧹 we have unfinished business.\"\n\nIt wasn't perfect.\n\nBut it was understandable.\n\nThe model was actually producing responses that made sense.\n\nAnd I thought:\n\nOkay. This is working.\n\nThen I Changed The Training System Again\n\nI continued developing my own training implementation.\n\nAnd then I trained again.\n\nThis time...\n\nOH MY GOD.\n\nThe results were terrible. 😭\n\nI started getting things like:\n\nYou: Hello\n\nCat: yes. � I understand the ancient languagnacke.\n\nYou: do you want food?\n\nCat: food. �� � I have interrupt the sound forctinking.\n\nYou: fish?\n\nCat: food. � � � suspicious.\n\nYou: who is your enemy?\n\nCat: yes. � I amain but jobs.\n\nYou: meow\n\nCat: okay. � I superiore your fish.\n\nYou: kitchen\n\nCat: fish. � They are you mrrp.\n\nAnd my reaction was basically:\n\nFAAAAAAAAAAAAAAAAAAAAH.\n\n😂\n\nThere were recognizable words.\n\nThere were sometimes recognizable meanings.\n\nBut the language wasn't behaving properly.\n\nAnd this was actually one of the most important moments in the project.\n\nRunning a Neural Network Isn't The Same As Making It Learn Properly\n\nI had already learned how to make the system train.\n\nI had a model.\n\nI had data.\n\nI had a training loop.\n\nI had the infrastructure.\n\nBut getting something to actually learn useful language representations is a completely different problem.\n\nThat experiment showed me that my implementation still had a lot to improve.\n\nAnd that's where I started understanding the difference between:\n\n\"The code runs.\"\n\nand\n\n\"The model works well.\"\n\nThose are VERY different things.\n\nBut The API Was Becoming What I Wanted\n\nEven while the underlying system still needed work, the interface was becoming extremely simple.\n\nFor example, pretraining and fine-tuning can look like:\n\nimport tensorless as tl\n\nprint(\"Pretraining base model...\")\n\nbase_model = tl.pretrain(\n\nout=\"pretrained_base.tl\",\n\nmax_seq_len=30\n\n)\n\nprint(\"Base model pretrained -> pretrained_base.tl\")\n\nprint(\"Fine-tuning on cat conversations...\")\n\nmodel = tl.train(\n\n\"data/conversations.json\",\n\npretrained=\"pretrained_base.tl\",\n\nout=\"cat.tl\",\n\n)\n\nprint(\"CatTongue trained successfully!\")\n\nThat's basically the experience I'm trying to create.\n\nYou don't need to write a massive training script.\n\nYou don't need thousands of lines just to get started.\n\nYou provide your data.\n\nTensorless handles the setup.\n\nAnd if you want to customize things, you can.\n\nBeginner vs Advanced User\n\nThis is something I really want Tensorless to get right.\n\nIf you're a beginner\n\nYou shouldn't have to understand every parameter.\n\nYou should be able to do something like:\n\nmodel = tl.train(\"data/\")\n\nand get started.\n\nIf you're an advanced programmer\n\nYou should be able to configure the training yourself.\n\nI want all the important configuration to be accessible in one place rather than making someone jump through a huge project just to change one setting.\n\nSo the idea is:\n\nOne simple function for beginners. Deep configuration for advanced users.\n\nI Also Wanted It To Stay Lightweight\n\nAnother thing I'm trying to maintain is the overall package size.\n\nTensorless is currently under roughly 30–40 MB overall, rather than pulling in a huge collection of unnecessary components.\n\nThat's important to me because the original reason for this project was partly:\n\nI want to experiment without needing a giant stack just to get started.\n\nI don't want someone to install Tensorless for a small experiment and feel like they installed an entire operating system. 😭\n\nTensorless Is Now Actually Open Source\n\nThe project is available on GitHub and PyPI.\n\nGitHub: [https://github.com/DeveloperPuneet/Tensorless](https://github.com/DeveloperPuneet/Tensorless)\n\nPyPI: [https://pypi.org/project/tensorless/](https://pypi.org/project/tensorless/)\n\nAnd CatTongue is also open source:\n\nCatTongue: [https://github.com/DeveloperPuneet/CatTongue](https://github.com/DeveloperPuneet/CatTongue)\n\nThe CatTongue experiment is one of the things I'm using to test whether the training system is actually improving.\n\nAnd Now I'm Stuck On The Hard Stuff\n\nThis is where the project is today.\n\nThe basic system works.\n\nThe API works.\n\nTraining works to a certain extent.\n\nThe automatic setup works.\n\nBut now I'm running into problems that aren't as easy as writing another Python function.\n\nCUDA\n\nI've been working on CUDA support and optimization.\n\nThe system can detect the GPU and show GPU-related activity, but I'm having problems getting the actual training workload to behave properly on the GPU.\n\nBasically:\n\nThe GPU is detected, but it doesn't always mean the training is actually being accelerated the way it should be.\n\nThat's something I really want to fix.\n\nTPU\n\nI've also started working toward TPU support.\n\nAgain, detection is one thing.\n\nActually getting the training workload to execute correctly on the TPU is another.\n\nSo TPU support is currently something I want to improve substantially.\n\nMulti-GPU\n\nI've also been working on multi-GPU support.\n\nThe problem is that detecting multiple GPUs doesn't automatically mean the training workload is correctly distributed across them.\n\nI want multi-GPU training to actually make use of the available devices rather than simply detecting them.\n\nTraining Results\n\nThis is probably the biggest issue.\n\nThe same training data can behave differently depending on the implementation.\n\nMy current training implementation doesn't always produce results comparable to established training frameworks.\n\nThe CatTongue experiment made that very obvious.\n\nThe model can learn words.\n\nIt can sometimes learn relationships between words.\n\nBut getting consistently good results is another level.\n\nAutomatic Configuration\n\nThe automatic configuration is also something I want to improve.\n\nRight now, some decisions are based on predefined logic.\n\nThat's useful, but it isn't flexible enough for every dataset or model.\n\nI want the system to become smarter about determining what configuration makes sense.\n\nThis Is Where I Need Help\n\nAnd honestly, this is the main reason I'm writing this article.\n\nI'm still learning.\n\nI started this entire project because I was learning PyTorch.\n\nI built two models with PyTorch.\n\nThen I started automating the process.\n\nThen I started building my own system.\n\nAnd now I've reached areas where there are people who know way more than I do.\n\nIf you have experience with:\n\nCUDA\n\nGPU optimization\n\nTPU/XLA\n\nmulti-GPU training\n\nneural network implementation\n\ntraining stability\n\ndata pipelines\n\nmodel optimization\n\nBPE/tokenization\n\nmachine-learning infrastructure\n\nI'd genuinely love feedback.\n\nYou don't have to write code.\n\nEven finding a problem and explaining why something is wrong would help.\n\nAnd if you want to contribute code, even better.\n\nThe project is open source and the issues are there for exactly this reason.\n\nWhat I Want To Build Next\n\nI have a pretty ambitious list of things I'd like to experiment with.\n\nSome of the ideas currently on my roadmap are:\n\nBetter CUDA optimization\n\nProper TPU training\n\nReliable multi-GPU training\n\nSmarter automatic configuration\n\nImproved data pipelines\n\nMore training/data formats\n\nAdditional model architectures\n\nGame integration\n\nCLI-based AI interaction\n\nUI-based AI interaction\n\nUser-vs-AI experiences\n\nReinforcement learning\n\nQ-learning\n\nDeep Q-learning\n\nAnd more\n\nThe goal isn't necessarily to implement everything immediately.\n\nI want to keep expanding the system as I learn.\n\nFrom Two PyTorch Models To Tensorless\n\nLooking back, the progression is kind of funny.\n\nI started with:\n\n\"I want to learn PyTorch.\"\n\nThen I built two models with PyTorch.\n\nThen:\n\n\"Why do I have to configure all this stuff every time?\"\n\nSo I started automating it.\n\nThen:\n\n\"Why is this getting so heavy?\"\n\nSo I started trying to make it lightweight.\n\nThen:\n\n\"What if I build more of the system myself?\"\n\nThen came:\n\nCUDA.\n\nTPUs.\n\nData pipelines.\n\nBPE.\n\nMulti-GPU.\n\nAnd then eventually:\n\n\"Why does my cat suddenly speak ancient alien language?\" 💀\n\nThat's basically how Tensorless happened.\n\nI didn't start this project because I thought I could build something better than PyTorch.\n\nI started it because I wanted to make one specific thing easier:\n\nTake training data → configure as little as possible → train a model.\n\nI wanted something where beginners could just give it their data and go, while advanced users could still control the system when they needed to.\n\nAnd now I'm trying to figure out how far I can actually take that idea.\n\nIf You Want To Check It Out\n\nTensorless\n\n[https://github.com/DeveloperPuneet/Tensorless](https://github.com/DeveloperPuneet/Tensorless)\n\n[https://pypi.org/project/tensorless/](https://pypi.org/project/tensorless/)\n\nCatTongue\n\n[https://github.com/DeveloperPuneet/CatTongue](https://github.com/DeveloperPuneet/CatTongue)\n\nIf you're interested in the project, try it, break it, report something, suggest something, or contribute.\n\nEspecially if you understand the parts I'm currently struggling with.\n\nI'm still learning.\n\nThat's kind of the whole reason this project exists.\n\nI started by learning PyTorch.\n\nNow I'm learning what it actually takes to build the stuff underneath it.\n\nAnd honestly?\n\nI'm curious to see where this goes. 🚀", "url": "https://wpnews.pro/news/i-was-learning-pytorch-then-i-accidentally-started-building-my-own-ai-training", "canonical_source": "https://dev.to/puneetkumar2010/i-was-learning-pytorch-then-i-accidentally-started-building-my-own-ai-training-framework-55a9", "published_at": "2026-08-23 10:48:35+00:00", "updated_at": "2026-08-23 11:14:30.284178+00:00", "lang": "en", "topics": ["machine-learning", "developer-tools", "ai-infrastructure"], "entities": ["Tensorless", "PyTorch", "CatTongue"], "alternates": {"html": "https://wpnews.pro/news/i-was-learning-pytorch-then-i-accidentally-started-building-my-own-ai-training", "markdown": "https://wpnews.pro/news/i-was-learning-pytorch-then-i-accidentally-started-building-my-own-ai-training.md", "text": "https://wpnews.pro/news/i-was-learning-pytorch-then-i-accidentally-started-building-my-own-ai-training.txt", "jsonld": "https://wpnews.pro/news/i-was-learning-pytorch-then-i-accidentally-started-building-my-own-ai-training.jsonld"}}