ByManish ShahiSoftware Engineer • AI Developer
Table of contents(29) #
I run manish.sh. I write about AI tools, how LLMs behave, and the product work that makes people stay.
This post opens the ** Inside AI Products** series — I read what a company published, explain it in plain English, and tell the story in first person. If you prefer interviewing a model and then checking papers, that is
—
Qwen.
I did not find Character.ai through a research paper.
I typed c.ai.
The browser sent me to character.ai. Short domain, long rabbit hole. I tried a few characters, then one more turn, then another hour. The question for this series is simple: what engineering makes people stay?
60-second TL;DR
How I found it: Typedc.ai
→ redirected to Character.ai → stayed for long role-play chats.What it is: A consumer chat app for talking to (and creating) characters — companions, fiction, practice — not a coding API.Problem it solves: Generic chatbots forget the plot and break character; Character.ai optimises for long, in-character sessions.Why them: Co-founded byandNoam ShazeerDaniel De Freitas(ex-Google;Transformer+ Meena/LaMDA — seeCharacter.ai on Wikipedia). Chat and cheap serving were already their craft.Model:Kaiju— densetransformersat 13B / 34B / 110B, tuned for speed and fun chats, not only exams.Speed tricks:MQA,sliding-window attention, sharedKV cache,INT8+QAT, sticky sessions.Stickiness: Story Memory + Facts +Lorebook+ a “good writing” scorecard.Safety:SFT→ onlineDPO→ classifier-guidedbeam search.** Sources:**the ten Character.ai posts underResearch— company blogs, not an independent audit.
How to read this post
Read it like a documentary. Each chapter usually goes:
Short scene— what happened / what the company said** Hard words**— a** Like you are fivegloss when terms get dense Picture or table**— mermaid diagram or quick comparison when it helps** Callout**— Key Idea / Why This Matters / Common Misconception (when useful)** Takeaway → Next**— one line to keep, then a hook into the next chapter
Blue/glossary links open fuller definitions. Jump around via Chapters. Sources sit at the bottom under Research.
Chapters
Part 1 — How I got here
- How I found the product
- Introduction to the product
- What problem it solves
- What this series is
- Who founded it — and why it works
Part 2 — The model (Kaiju + training)
Part 3 — Product loops
Part 4 — Scale & open future
How I found the product #
I was looking for a quick character chat. Someone had mentioned “c.ai” the way people mention a meme domain — short, easy to share, easy to type from memory.
So I opened c.ai
.
Redirect. Landing page. Character.ai.
That redirect matters. Short domains get people through the door. What keeps them there is the chat: fast replies, a character that remembers the conversation, and a story that stays coherent for a long time.
I live in India. I care about how these systems are built — not only demos. After enough long chats, I went looking for Character.ai’s own engineering posts. They publish a surprising amount: Kaiju, inference, Squinch, memory updates, Slonk, open-source fine-tuning. This article is my first-person walkthrough of that public trail.
Introduction to the product #
Character.ai is a consumer chat product: you talk to characters — celebrities, fictional people, tutors, custom personas you create — in a threaded chat UI. Sign-up is light (Google / Apple / email). The homepage pitch is millions of characters and “start in seconds.”
It is not primarily a developer API playground or a coding assistant. The unit of value is a session: many turns with one character, often role-play or companion chat, sometimes study or practice.
Like you are five: Imagine a huge costume party of chatbots. You pick one costume (a character), then talk for as long as the story stays fun.
Under the hood (from their public posts) that product sits on:
| Layer | What you feel | What they talk about publicly |
|---|---|---|
| Model | Replies feel “in character” | |
KV cache,MQA,INT8servingLorebookSFT→DPO→ guided decode## What problem it solves
Most general chatbots are built for short Q&A: ask, answer, done. Character chat fails that pattern in three boring ways:
Continuity— After twenty turns the bot forgets your name, the plot, or the tone.** Character**— Replies drift into generic “helpful assistant” voice.** World**— Shared lore (places, rules, side characters) either vanishes or has to be pasted into every prompt.
Character.ai’s public story is a bet that people want companions and stories, not only answers. So the product problem is:
Keep a fictional (or companion) conversation
alivefor a long time — fast enough to stay immersive, consistent enough to feel real, safe enough to ship at scale.
Like you are five: A good bedtime storyteller remembers what happened yesterday. A bad one asks your name every page. Character.ai is trying to be the first kind — for millions of chats at once.
That is why later chapters obsess over sticky KV caches, Story Memory, Lorebook, and a “compelling writing” scorecard instead of only MMLU points.
What this series is #
Inside LLMs asks a model about itself, then checks papers.
Inside AI Products asks: what did the company say it built, and how does that explain how the app feels?
Character.ai is a good first pick because:
- People live in the product, not only in an API playground.
- Their public posts cover model → serving → memory → infra. - The hook is clear: long role-play and companion chats.
I will keep it technical but easy. New terms get a short definition and a glossary link.
Who founded it — and why it works #
Character.ai was co-founded by ** Noam Shazeer** and
Daniel De Freitas, both ex-Google. (Shazeer has his own Wikipedia page; De Freitas is mainly on the
Character.aipage and
Wikidata.)
This is not a random founder story. It explains why they could build a chat product that holds under heavy load.
** Noam Shazeer** helped write
Attention Is All You NeedTransformer. At Google he also worked on big serving systems. Later Character.ai ideas like
MQA, smaller
KV caches, and
Squinchfit that same goal: make big models cheaper to run.
Like you are five: Shazeer helped invent the popular “brain recipe” (Transformer) and also cared about making the recipe run without melting the kitchen (serving cost).
** Daniel De Freitas** helped design Google’s chat models that started as
Meena and became
. In short: years of dialogue work, not only demos.
LaMDALike you are five: De Freitas spent years teaching computers to hold a conversation, not only finish a sentence on a quiz.
They left Google around 2021 and built Character.ai end to end — train or fine-tune, tune for character chat, then ship to users worldwide. In 2024 both went back to Google DeepMind in a licensing deal; Character.ai kept running with the remaining team. This post still follows the public engineering they left behind.
So when they talk about INT8 inference, sticky KV caches, and ~20,000 queries per second, it does not sound like empty marketing. They already knew the hard truth: chat models stay expensive unless you invent serving tricks.
Kaiju: the in-house LLM family #
Main source: Inside Kaiju.
** Kaiju** is Character.ai’s own family of large language models.
Like you are five:
- An is a very big word-guessing brain. You type words; it guesses what comes next, again and again, until a reply appears.LLM - A is that brain’s wiring — lots of tiny number dials connected together.neural network - A is a tiny text brick (a short word or part of a word).token are those number dials. Training turns the dials so guesses get better.Parameters/weights
Three production sizes:
| Variant | Size |
|---|---|
| Small | 13 billion parameters |
| Medium | 34B |
| Large | 110B |
Like you are five: “13 billion parameters” means the model has a huge pile of dials — enough to remember lots of language patterns. Bigger piles can be smarter, but they also cost more to run.
Many labs chase exam leaderboards first. Character.ai says Kaiju was built for fast, fun, safer chats. Think “does this feel good at 2am in a long role-play?” more than “can we gain one more exam point?”
Kaiju is a dense transformer.
Like you are five:
- A transformer is the “brain recipe” most chat AIs use. It reads your words, decides what matters, and guesses the next little piece of text. - A token is one of those little pieces — often a short word or part of a word. Dense means: for every next guess, the whole kitchen stays open. Every cook (every layer) works on the order.Feed-forward block is one workbench inside each layer — the place that mixes and reshapes what the model just “heard.” In a dense model, that whole workbench stays on.Sparse is a different kitchen. You hire many specialist cooks (“experts”), but for each token only a few wake up. The others sleep. That can save cost, but it is a different design.Mixture of Experts(MoE)
Kaiju uses the dense style. That fits the founders’ background — they helped popularise dense transformers, and Kaiju follows that path.
flowchart TD
A["Your message"] --> B["Kaiju dense transformer"]
B --> C["Next-token stream"]
C --> D["Safety-aware decode"]
D --> E["Reply in chat"]
classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
class A,B,C,D,E box
Architecture tricks that make chats feel instant #
Long chats cost a lot. A big reason is the ** KV cache**.
Like you are five:
is “what should I look at right now?” — like pointing at the important words in a story.AttentionKey / Value (K and V) are sticky notes the model saves about words it already saw, so it can look them up later.- A is the box of those sticky notes. Without it, the model would rewrite every sticky note from scratch every turn (slow and expensive).KV cache means “using the trained model to chat” — not the training homework phase.Inference
Character.ai worked hard to make that sticky-note box smaller and cheaper.
1. Multi-Query Attention (MQA)
Normal multi-head attention gives every head its own Keys and Values. ** MQA** shares K and V across heads. Less to store → smaller KV cache → more chats per GPU and faster inference.
Like you are five: Imagine many flashlights (heads) reading a page. Old way: each flashlight keeps its own photocopy of the page. MQA: they share one photocopy. Same reading, less paper in the backpack.
2. Sliding Window Attention
Most layers only look at a recent window (1024 tokens in their write-up), with a few full “global” layers mixed in (about 5:1 or 6:1). That is ** sliding-window attention**: cheaper on long chats, still some long-range reach. Current Kaiju does not use
Like you are five:
Sliding window= mostly read only the last few pages of the story, not the whole library every time.** Global layers**= every so often, peek at the whole book so you do not forget the beginning.= a trick where the very first words keep grabbing attention forever. Kaiju says it does not lean on that trick right now.Attention sink
3. Cross-layer KV cache sharing
About 2–3 neighbouring layers share one KV cache. Same idea as MQA, applied up the stack: less memory, little quality loss in their reports.
Like you are five: Floors 3, 4, and 5 of the building share one filing cabinet instead of each buying their own.
4. INT8 + Quantization-Aware Training
Weights, activations, and KV values use ** INT8**. On modern chips, INT8 matmuls are roughly
2× faster than bf16-style compute. They train with
so the model learns to stay accurate at that precision — not only
QATquantizeat the end and hope.
They also name pre-layer RMSNorm (normalise before the heavy work) and
dynamic clamping so low precision stays stable.
Like you are five:
= draw with fewer crayons. A fine picture uses many colour shades; a smaller box of crayons is lighter to carry.Quantization= an 8-crayon box for numbers (small and fast).INT8bf16 / FP32= bigger crayon boxes (more detail, heavier).** Matmul**= the model’s “times tables” on huge number grids — the heavy gym work GPUs do.= practise drawing with the small crayon boxQATduring training, so the picture still looks good later.= gently rescale the numbers so they do not get too wild before the next step.RMSNormDynamic clamping= “no number may jump higher than this fence” — and the fence can move a bit so training stays safe.
| Trick | What you feel in chat |
|---|---|
| MQA | Smaller memory per token |
| Sliding window | Long chats cost less |
| Cross-layer KV share | Even less KV memory |
| INT8 + QAT | Faster replies without a quality cliff |
Training stack: Squinch and friends #
Inside Kaiju covers training briefly. The longer story is Optimizing Large-Scale Pretraining — written after they stopped big pretraining and started using open-source bases more.
Like you are five:
Pretraining= the long “read lots of text” homework before the model becomes a chat buddy.** Open-source base**= someone else’s finished homework you may reuse, then add your own tutoring.** GPU / H100**= a very fast number engine.
Before that shift, the early training team (led in part by Shazeer) shared a few practical tricks.
Hardware. NVIDIA H100 GPUs. Model parallelism inside a node (tensor + sequence). ** FSDP** across nodes.
Like you are five: The giant Lego set does not fit on one table, so friends each hold a piece (model parallelism). ** FSDP** also shares the instruction booklet and bricks across tables.
Mixed precision. Rough map:
| Piece | Precision |
|---|---|
| Forward weights / KV | INT8 |
| Activations / local gradients | |
Like you are five: Cheap crayons for most drawing; keep one fancy master colouring book (FP32) so colours stay neat.
** Squinch.** A 6-bit way to compress gradients by Noam Shazeer. Gradients move between machines in small blocks (8 numbers each). Less network traffic, still accurate enough.
Like you are five: A gradient is a “turn this dial left/right” note after practice. Squinch folds many notes into a tiny postcard so machines can mail them fast.
Other names from that era: Attention Z-Reg (keeps attention scores stable in bfloat16), Visibility Mask (compact rules for what can attend to what in packed chat data), Virtual Scalars (Bungee) (helps small INT8 models stay stable), and ternary weight updates (send only 0 / 1 / −1 in some cases).
Like you are five:
Attention Z-Reg= do not let attention numbers get so huge they go blurry.** Visibility Mask**= a seating chart for which words may look at which words.** Bungee**= elastic bands so tiny INT8 models do not snap.** Ternary updates**= sometimes only shout “stay / up / down.”
Data. Two mixes: “MMLU Max” for exams, “Production Max” for engagement. Web text, code, synthetic data — then ** pretraining annealing** near the end (shift toward higher-quality / instruction data).
Like you are five: Quiz books vs fun-chat practice. Annealing = near the end, switch to cleaner practice so good manners stick.
flowchart LR
A["Data mix"] --> B["Dense transformer train"]
B --> C["Squinch grads across nodes"]
C --> D["QAT-aware INT8 model"]
D --> E["Annealing → post-training"]
classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
class A,B,C,D,E box
Safety pipeline #
Also from Inside Kaiju. Before a model goes live:
on safety and instruction data.SFT- Modified online from swipe / feedback data — a lighter path than classicDPORLHF. Classifier training, with an optional head that can score safety per** token**.
At inference they use classifier-guided beam search: try a few candidate replies, steer with safety scores, keep the chat creative without going fully bland.
Like you are five:
= show many good example answers and say “copy this style.”SFTSwipe / feedback= users tapping “this reply was better / worse.”= learn from those better/worse picks without building a giant separate “score teacher” first.DPO= a heavier version of “humans score answers, then the model practises to please the score.”RLHFClassifier= a safety referee that gives thumbs up / down signals.= keep a few possible next sentences in play, not only one, then pick a good safe path.Beam searchClassifier-guided= the referee helps choose among those paths.
flowchart TD
A["Base Kaiju"] --> B["SFT safety + instructions"]
B --> C["Online DPO from swipes"]
C --> D["Safety classifier head"]
D --> E["Classifier-guided beam search"]
E --> F["User-facing reply"]
classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
class A,B,C,D,E,F box
Memory and Lorebook #
Product meets habit here. Two posts matter most: Smarter Memory for Smarter Chats and April Update: New Model, Memory, and Lorebook.
Like you are five: Chat AIs forget easily. Memory features are sticky notes so the character does not ask “what is your name?” every five minutes.
Story Memory and Facts
Story Memory tracks the plot and emotional thread. You can pin what matters so it is less likely to drop. Facts cover appearance, quirks, relationships, hobbies, and similar stable details. The Memory page got a cleaner layout, more categories (hairstyle, eye colour, quirks, and more), in-chat “saved” notices, and Memory Usage bars so you see when space is running low.
These upgrades were meant to run on newer styles like PipSqueak 2 (PSQ2) and improved DeepSqueak — more consistent, less drift, fewer cutoffs. DeepSqueak 2 was said to be training in the April update.
Like you are five:
Story Memory= “what happened in our adventure.”** Facts**= “your hair is blue; you love mystery books.”** Pin**= a gold star sticky note that should not get thrown away.** Memory Usage bar**= a juice glass — when it is almost full, old notes may get squeezed shorter.** PipSqueak / DeepSqueak**= names of chat “engines” (model styles) on the app — like different car engines under the same car body.
Lorebook
** Lorebook** was the community’s #1 request for a long time. In the April update, Character.ai described it as a reusable
keyword-triggered world database — lore that appears when the chat mentions that place, person, or rule.
You write entries for locations, side characters, rules, items. They only load when keywords match. So you can share one world across characters without stuffing everything into every prompt. c.ai+ creators got first access, then wider rollout.
Like you are five: Lorebook is a backpack of world cards. You only pull out the “dragon castle” card when someone says “dragon” or “castle” — you do not dump the whole backpack on the table every turn.
| Feature | Job |
|---|---|
| Story Memory | Keep the plot |
| Facts | Keep stable details |
| Memory Usage | See what is filling space |
| Lorebook | Load world lore only when needed |
Compelling writing evaluation #
Most labs score exams and coding. Character.ai’s post Evaluating Our Models Using Principles of Compelling Writing takes another path: a Compelling Writing Evaluation Framework built with professional writers.
They break “good chat writing” into measurable pieces: plot shapes (like the Hero’s Journey), archetypes, pacing, “show don’t tell”, dialogue quality, emotional intelligence, and novelty. Offline LLM judges score writer-labelled data; online user ratings sit beside that.
Like you are five:
Exam benches (like MMLU)= quiz marks.** Compelling writing eval**= “was this story fun and clear?” report card.** LLM-as-judge**= another AI teacher marks homework with the writers’ checklist.** Hero’s Journey**= a famous adventure shape (leave home → trouble → grow → return).
Simple reason: if people role-play for hours, story quality is a real metric, not a side hobby.
Inference at scale #
Character.ai has said they serve about 20,000 queries per second (see their earlier inference posts). That only works if prefill and decode stay cheap.
Like you are five:
Queries per second= how many “please reply” knocks arrive every second.= read your whole message first (set the table).PrefillDecode= say the reply word by word (serve the food).** Kernel**= a tiny ultra-fast recipe written for the GPU.= a clever way to do attention with less memory traffic.FlashAttention= send you back to the same waiter so your sticky-note box (KV cache) is already there.Sticky session= how long until the first word appears.TTFT= how long between the next words.TPOT= an even smaller crayon box than INT8 for some serving work.FP8DP / TP / EP= ways to split one giant model across many GPUs (data / tensor / expert splits).= a popular engine for serving chat models efficiently.vLLMp90= “for 90 out of 100 requests, we were at least this fast” — a fairness check on speed.
From outside, traffic tools show the same habit: huge monthly visits and long sessions. Here is a Similarweb snapshot I saved (an estimate, not Character.ai’s official numbers). Notice visit length and pages per visit — people stay and chat.
Serving ingredients they name:
- Custom INT8 attention kernels FlashAttentionchanges for MQA- Parallel work across query heads (~10–30% faster in prefill/decoding vs older kernels) — same chat stays on the same server so KV cache hits stay high (they have cited ~95%)Sticky sessions
They also posted Technical Deep Dive: DigitalOcean + AMD, 2× production inference. On MI300X / MI325X, for a large MoE like Qwen3-235B, they report about 2× throughput with DP2/TP4/EP4, FP8, careful GPU placement, vLLM tuning, and strict p90 ** TTFT** /
targets — with cost per token down by a similar factor in that write-up.
TPOT| Metric | What you feel | |---|---| | TTFT | How fast the first word appears | | TPOT | How smooth the stream feels after that | | Sticky KV hits | Less cold start mid-chat |
Slonk: Slurm on Kubernetes #
From Slonk:
Researchers like ** Slurm** (
sbatch
, squeue
, fair share). Ops teams like for healing and containers.
KubernetesSlonk = Slurm on Kubernetes. Familiar research commands on resilient K8s StatefulSets (controller, workers, login pods), health checks, auto-fix, Git-sync, and a custom operator for machines. Training and inference can share the pool; production can take priority when needed.
Like you are five:
= the school timetable for GPU homework (“your turn on the big computer”).Slurm= robot caretakers that restart broken classroom PCs and keep apps alive.KubernetesSlonk= keep the familiar timetable, but let the robot caretakers run the building.** StatefulSet / pod**= a named computer seat that Kubernetes can replace if it breaks.** Preempt**= “production chat needs this GPU now — training homework, please .”
If you have seen a cluster suffer because the scheduler and the container layer do not agree, this design will feel familiar.
Open-source shift #
In Breaking News: Our Open-Source Models Are A Lot of Fun!, Character.ai explained the move: take open-source bases, tune them for fun and emotional quality, and ship them as Character models — using their user feedback loop, smarter prompting, and post-training (SFT, DPO, RL, QAT).
Like you are five:
Open-source model= a shared toy brain you can download and improve.** Post-training**= extra tutoring after the big homework (SFT / DPO / RL / QAT).** RL**= practise by trial and reward (“that reply earned a cookie”).** Ensemble inference**= ask a small team of models / settings and blend the vibe.** Retention**= people coming back tomorrow.
Their own test numbers:
22% more time spent and13% more sessions14% better retention for lightly engaged users60% more messages rated positive (plus jumps in “funny”, “interesting”, “helpful”)
Then they rolled out PipSqueak more widely. Treat the numbers as company launch metrics — but the direction is clear: entertainment over pure exam scores.
Whether the base is Kaiju or a fine-tuned open model, the product still needs memory, safety, and fast serving.
pipeling-sft and Agent SDK #
Two posts show what comes after pure in-house pretraining.
pipeling-sft
Character.AI Open Sources pipeling-sft is a toolkit for full-parameter SFT of large MoE models (DeepSeek V3–class). It supports multi-level parallelism, BF16 and experimental FP8, Hugging Face checkpoints, and helpers for stable long runs. Still experimental — meant for teams who do not want to rebuild MoE fine-tuning from scratch.
Like you are five: A shared recipe book for teaching a giant “many-cooks” (MoE) brain new manners, without inventing the oven from scratch.
Agent SDK
Agent SDK: Teaching Machines to Use Tools with Claude Code looks past pure chat. Their backend is Go; Anthropic’s Claude Code Agent SDK was Python-first — so they built a Go port with tools, streaming, hooks, MCP, and SSE. Internal tools (they mention AgentX) use it for multi-step work, provider fallbacks, and quality checks. Direction: products that can use tools, not only talk.
Like you are five:
SDK= a toolbox of ready-made Lego bricks for builders.= the AI can press buttons (search, generate, check) instead of only talking.Tool calling= a shared plug standard so tools fit many AIs.MCPSSE / streaming= words (and events) arrive like a live ticker, not one big sealed envelope.** Go / Python**= two different building languages; their servers speak Go.
Why users stay hooked #
Back to that first c.ai
redirect.
I stayed because the chat felt alive enough — fast enough to keep immersion, consistent enough to remember the plot, creative enough that the next turn felt worth typing.
The public engineering story maps to that feeling:
flowchart TD
A["Fast INT8 + MQA + sticky KV"] --> E["Immersion holds"]
B["Story Memory + Facts + Lorebook"] --> E
C["Compelling writing eval"] --> E
D["SFT + DPO + guided beam safety"] --> E
E --> F["Long sessions / return visits"]
classDef box fill:#f7f5f0,stroke:#2a2a2a,color:#2a2a2a,stroke-width:1px
class A,B,C,D,E,F box
Character.ai is not only “building models.” In their words, they are building companions and stories — and every trick from Squinch to Lorebook serves that loop.
If you have used it: what keeps you coming back — speed, memory, a favourite character voice, or Lorebook worlds? Do tell — it will help shape the next Inside AI Products entry.
Research #
Primary Character.ai sources for this walkthrough (checked live):
Inside Kaiju — building conversational models at scale— in-house LLM architecture; conversation over benchmarks; MQA; sliding-window attention; KV cache sharing; INT8 + QAT; training overview; safety pipeline.Optimizing Large-Scale Pretraining at Character.AI— distributed training; Squinch; data/training optimisations; scaling transformers (pre–open-source-foundations era).Evaluating Our Models Using Principles of Compelling Writing— fiction-writer-style eval instead of only MMLU/coding benches.Breaking News: Our Open-Source Models Are A Lot of Fun!— shift to open-source foundations; fine-tuning for entertainment; engagement metrics.Smarter Memory for Smarter Chats— Story Memory; Facts; Memory Usage; long chats.April Update: New Model, Memory, and Lorebook— PipSqueak 2 / DeepSqueak; memory UX; Lorebook.Technical Deep Dive: How DigitalOcean and AMD Delivered a 2x Production Inference Performance Increase for Character.ai— GPU / inference serving; production latency; AMD + DigitalOcean.Slonk: Slurm on Kubernetes for ML Research at Character.ai— Kubernetes; GPU scheduling; research clusters.Character.AI Open Sources pipeling-sft— fine-tuning MoE models (DeepSeek V3–class); SFT pipeline.Agent SDK: Teaching Machines to Use Tools with Claude Code— tool-using agents beyond pure conversation (GitHub: character-ai/claude-agent-sdk-go).
Related context (founders / architecture lineage, not in the list above):
Attention Is All You Need· GoogleLaMDA· WikipediaNoam Shazeer·Character.ai· WikidataDaniel De Freitas- Earlier serving write-ups that mention ~20k QPS / sticky KV caching: Optimizing AI Inference·Part Deux
Also used in this post:
- Series: Inside AI Products· sister series:Inside LLMs - Site: manish.sh - Images: homepage·Similarweb traffic snapshot· concept diagrams:dense vs MoE·KV / MQA·safety pipeline·memory + Lorebook·habit loop - Related glossary: Character.ai·Kaiju·LLM·Transformer·MQA·Sliding Window Attention·KV Cache·INT8·QAT·RMSNorm·Squinch·DPO·SFT·Beam Search·Lorebook·TTFT·TPOT·Slurm·Kubernetes·vLLM·FlashAttention·FSDP·MoE·MCP·full glossary
Pinch or double-tap to zoom · tap outside to close
FAQ
Frequently asked questions #
How did you end up writing about Character.ai? #
I typed c.ai, the domain redirected me to character.ai, and I stayed for the long role-play chats. This post unpacks the public engineering behind that habit. I write from India.
Is this an interview of the model? #
No. Unlike the Inside LLMs series, this piece is built from Character.ai’s public technical and product blog posts — not from interviewing a chat model about itself.
What is Kaiju? #
Kaiju is Character.ai’s own LLM family: dense transformers in Small (13B), Medium (34B), and Large (110B), tuned for fast, fun, safer chats.
What is Lorebook? #
A keyword-triggered world database. Entries for places, side characters, rules, and items only load when matching keywords appear — shared worlds without stuffing every prompt.
Why do chats feel so sticky? #
Fast replies (KV cache + INT8 tricks), Story Memory and Facts for continuity, Lorebook for world consistency, and a writing scorecard aimed at good dialogue — not only exam scores.
Who founded Character.ai? #
Noam Shazeer and Daniel De Freitas, both ex-Google. Shazeer co-authored the Transformer paper; together they worked on chat models that became Meena / LaMDA. That is why a full chat stack at this scale makes sense.
Where do the numbers come from? #
From Character.ai’s public engineering posts listed in the Research section (Kaiju, pretraining/Squinch, compelling writing, open-source shift, memory, Lorebook, AMD/DigitalOcean inference, Slonk, pipeling-sft, Agent SDK). Treat them as vendor-published, not an independent audit. Traffic charts from Similarweb are third-party estimates.
Research this topic further
- Click a tool below (ChatGPT, Perplexity, Claude, or Gemini).
- We copy the
full article + companion guide prompt to your clipboard. - A new chat tab opens — if the message box is empty or only shows a short note, press
Ctrl+V(Windows/Linux) orCmd+V(Mac) to paste. - Send the message. The model already has the article text — it does not need to open this website.
Related posts
Comments
Share a thought on this post — keep it useful and kind. Comments are moderated before they appear.
comments…