The book 'Build a DeepSeek Model (From Scratch)' by Raj Abhijit Dandekar, Rajat Dandekar, Sreedath Panat, and Naman Dwivedi rebuilds DeepSeek's innovations on consumer hardware, covering key-value cac…
Researchers propose Thought-Aware Attention Matching (TAM), a KV cache compaction method that segments chain-of-thought reasoning into blocks, allocates compression budgets adaptively, and protects pi…
New research shows that incentivizing diversity in AI exploration through a representation-based bonus derived from hidden states of pre-trained models boosts verifier efficiency by over 50% for the Q…
Researchers introduced TREK, a method that improves AI problem-solving by using verified output trajectories to extend model learning. TREK boosted Qwen3-8B's performance on AIME 2024 from 36.9 to 40.…
FIPO (Future-Impact-based Policy Optimization) is a reinforcement learning method that improves LLM reasoning by assigning token-level credit based on each token's future impact on the policy, rather …