AI builder essentials: tokens, context windows and RAG 101

A developer and @greggyb recorded a video explaining foundational AI concepts including tokens, retrieval-augmented generation (RAG), context windows, and tokenmaxxing. The video aims to demystify how large language models process input and output, and how providing more context can improve responses.

Tokens are an important foundational concept that underlie how we interact with LLMs. @greggyb https://dev.to/greggyb and myself recorded a quick video to demystify what a token actually is and unpack several other related concepts. What's in the video - What is a token? Tokens are the basic unit of input and output that LLMs operate on. - What is retrieval-augmented generation AKA RAG ? Providing a model with extra data that isn't already in the model's training dataset, and is relevant to a specific query. - What are context windows? Context windows are how many tokens a model can use for input and output. Context window size varies from model to model, but it has increased a lot in recent years. - What is tokenmaxxing? Does using more tokens really mean you're being more productive? Spoiler alert: no. To paint a bigger picture: more context you give a large language model, the better your responses are likely to be. The design patterns for providing that context have evolved at a rapid pace, and are likely to continue doing so. What AI terms should we demystify next? Let us know