My first ai model A high-school student's informal guide to getting started in machine learning recommends fine-tuning a 350M LFM model with r32 a64 LoRA rather than QLoRA, then moving to models such as qwen3.5-800M and the new gemma4, and capping full fine-tuning at 230M-4B parameters and pretraining under 250M. For a first pretraining run, the author suggests roughly 50B tokens on a 120M-150M parameter model, citing papers that put 141M at an efficient point, and scaling down architectures like qwen3-600M or SmolLM2's 135M. The author, who describes themselves as a year-10 high-school student, also advises against taking technical advice from LLMs, saying "I learnt it the hard way. hey man, thats really cool you are getting into ML. but look, we have several types of models like: CNNs, RNNs, transformers if you are a student like me, I reccomend getting started with transformers. also one more thing. if you ask help from a chatbot, they will say you can train big models like 30B+ models, but please stick to 230M-4B LoRA training for FT, and under 250M for PT. also there are two main types of attention: bi directional and casual. I recommend finetuning a 350M LFM model with r32 a64 LoRA not QLoRA. you have enough compute . then you should ry more models like qwen3.5-800M and maybe the new gemma4 models as well then once you are comfortable with things like loss masking, you should do a small PT run with around 50B tokens. try training a 120M model it has just enough parameters to not dissapoint you if u didnt plan ur data quite right, so its just neough. maybe bumping it to 150M will make a considerable change as well. although some papers i read said that 141M is efficient, so go from somewhere around 120-150M for the PT architecture, use an LLM for the math and code. try to learn the high level things like how you calculate loss for PT, context length and position encoding. for the architecture, use something like qwen3-600M and scale it down smollm2’s 135M model’s arch works as well since you have an A100, you can train this really quickly so you can comfortably learn things yourself. something with this field is that no matter how good you are with the math or science, if you havnt don anything practical, knowing just math and science wont help you. then you should slowly write your own code and math. I also like this books I am in HS yr10, but if u are above HS yr8 ish you should read this book the ebook is free, but the physical on is a bit expensive . wait, i actually have two books if u like reading. if you know some math skim this a bit: book – second link but I reccomend you to read both these together and for the love of god, please dont get advice from LLMs I learnt it the hard way BTW, you have a lot of compute, but dont waste it on big models, try to work on smaller models and then once you know what you are doing, scale up to bigger models. here is the list of models I like FT: also here is a fun challeng before doing PT, try doing CPT continuid pre training on smolLM2 135M to expand its context length you got the VRAM for it so dont worry about it. Then do a chat FT to chat with it. and also you can try doing a reasoning FT as well, but try doing the normal user/assistant one, then you can later redo the FT on the CPTed model for the reasoning dataset which has